2024-09-17T16:14:33.458 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T16:14:33.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T16:14:33.560 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060 branch: wip-bharath10-testing-2024-09-17-1348 description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-hybrid} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} email: skanta@redhat.com first_in_suite: false job_id: '7909060' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath10-testing-2024-09-17-1348 ceph: conf: client: debug monc: 20 debug ms: 1 global: debug ms: 1 debug objecter: 20 debug rados: 20 mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 5 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(TOO_FEW_PGS\) - \(POOL_APP_NOT_ENABLED\) - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_FULL\) - \(SLOW_OPS\) - \(MON_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - slow request sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath10-testing-2024-09-17-1348 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 3421 sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath10-testing-2024-09-17-1348 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPY1BFaEPASCia2fh3TnVU5B8nPB8spKzkfQsiOerydGXcBsEjpi2e6o6xdty0K1u0kv51u6uAvJ/ixdbhiFVI8= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDLD/HWlkC4pe/aVgYesztpUNYj13mT4gSGojjieF8/JL0teng+0g0v5ptgsPH4m4d3ZrzikpaSCUevJWW2pHk= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 store_thrash: true thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - rados/test.sh unit_test_scan teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_15:01:33 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T16:14:33.560 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa; will attempt to use it 2024-09-17T16:14:33.561 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa/tasks 2024-09-17T16:14:33.561 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T16:14:33.562 INFO:teuthology.task.internal:Checking packages... 2024-09-17T16:14:33.583 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1889a165b5128787ec693450a3d1c5df5d6dc203' 2024-09-17T16:14:33.583 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T16:14:33.583 INFO:teuthology.packaging:ref: None 2024-09-17T16:14:33.583 INFO:teuthology.packaging:tag: None 2024-09-17T16:14:33.583 INFO:teuthology.packaging:branch: wip-bharath10-testing-2024-09-17-1348 2024-09-17T16:14:33.583 INFO:teuthology.packaging:sha1: 1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:14:33.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath10-testing-2024-09-17-1348 2024-09-17T16:14:33.728 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4975-g1889a165-1jammy 2024-09-17T16:14:33.730 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T16:14:33.737 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T16:14:33.737 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T16:14:33.756 INFO:teuthology.task.internal:Saving configuration 2024-09-17T16:14:33.772 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T16:14:33.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T16:14:33.802 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 16:08:53.696940', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILt1iVxDfc4CFJzCMqU7h7fyNRt0J5Z5/nlMRvfnS8/t'} 2024-09-17T16:14:33.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 16:08:53.698892', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDZh4vtazx3mtO29uBuiQMchWbRtAsex0qDV4mGV0lzd'} 2024-09-17T16:14:33.824 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T16:14:33.852 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-09-17T16:14:33.852 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-17T16:14:33.852 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T16:14:33.942 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f14c46afbe0>, signals=[15]) 2024-09-17T16:14:33.943 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T16:14:33.948 INFO:teuthology.task.internal:Opening connections... 2024-09-17T16:14:33.948 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:14:33.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:14:34.024 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:14:34.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:14:34.102 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T16:14:34.108 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-09-17T16:14:34.115 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-09-17T16:14:34.116 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-09-17T16:14:34.162 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T16:14:34.162 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Ubuntu" 2024-09-17T16:14:34.162 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="22.04" 2024-09-17T16:14:34.162 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_CODENAME=jammy 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:ID=ubuntu 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE=debian 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T16:14:34.163 INFO:teuthology.orchestra.run.smithi105.stdout:UBUNTU_CODENAME=jammy 2024-09-17T16:14:34.163 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-09-17T16:14:34.182 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-09-17T16:14:34.187 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-09-17T16:14:34.188 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-09-17T16:14:34.233 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T16:14:34.233 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Ubuntu" 2024-09-17T16:14:34.233 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="22.04" 2024-09-17T16:14:34.233 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_CODENAME=jammy 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:ID=ubuntu 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE=debian 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T16:14:34.234 INFO:teuthology.orchestra.run.smithi160.stdout:UBUNTU_CODENAME=jammy 2024-09-17T16:14:34.235 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-09-17T16:14:34.257 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T16:14:34.267 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T16:14:34.273 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T16:14:34.273 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-09-17T16:14:34.276 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-09-17T16:14:34.282 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T16:14:34.288 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T16:14:34.288 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-09-17T16:14:34.324 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-09-17T16:14:34.401 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T16:14:34.458 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T16:14:34.497 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T16:14:34.497 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T16:14:34.498 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T16:14:34.498 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T16:14:34.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:14:34.503 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-17T16:14:34.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:14:34.504 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-09-17T16:14:34.550 INFO:teuthology.orchestra.run.smithi160.stdout:5.15.0-56-generic 2024-09-17T16:14:34.550 INFO:teuthology.task.kernel:Running kernel on smithi160: 5.15.0-56-generic 2024-09-17T16:14:34.550 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get clean 2024-09-17T16:14:34.551 INFO:teuthology.orchestra.run.smithi105.stdout:5.15.0-56-generic 2024-09-17T16:14:34.552 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.15.0-56-generic 2024-09-17T16:14:34.552 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get clean 2024-09-17T16:14:34.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-09-17T16:14:34.691 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get update 2024-09-17T16:14:34.808 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:14:34.831 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T16:14:34.834 INFO:teuthology.orchestra.run.smithi160.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T16:14:34.900 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T16:14:34.922 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T16:14:34.944 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:14:35.026 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T16:14:35.041 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T16:14:35.051 INFO:teuthology.orchestra.run.smithi160.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-17T16:14:35.118 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T16:14:35.156 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T16:14:35.157 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-17T16:14:35.159 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T16:14:35.159 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T16:14:35.160 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T16:14:35.162 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-17T16:14:35.164 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T16:14:35.164 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-17T16:14:35.181 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T16:14:35.185 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T16:14:35.185 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T16:14:35.186 INFO:teuthology.orchestra.run.smithi160.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T16:14:35.186 INFO:teuthology.orchestra.run.smithi160.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-17T16:14:35.189 INFO:teuthology.orchestra.run.smithi160.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T16:14:35.189 INFO:teuthology.orchestra.run.smithi160.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-17T16:14:35.191 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T16:14:35.195 INFO:teuthology.orchestra.run.smithi160.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-17T16:14:35.199 INFO:teuthology.orchestra.run.smithi160.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-17T16:14:35.200 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T16:14:35.201 INFO:teuthology.orchestra.run.smithi160.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T16:14:35.201 INFO:teuthology.orchestra.run.smithi160.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T16:14:35.202 INFO:teuthology.orchestra.run.smithi160.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T16:14:35.203 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-17T16:14:35.203 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T16:14:35.219 INFO:teuthology.orchestra.run.smithi160.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T16:14:35.219 INFO:teuthology.orchestra.run.smithi160.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T16:14:35.226 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T16:14:35.227 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T16:14:35.227 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T16:14:35.227 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T16:14:35.227 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T16:14:35.228 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T16:14:35.231 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T16:14:35.231 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T16:14:35.248 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T16:14:35.249 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T16:14:35.249 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T16:14:35.377 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-17T16:14:35.383 INFO:teuthology.orchestra.run.smithi160.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T16:14:35.522 INFO:teuthology.orchestra.run.smithi160.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T16:14:35.718 INFO:teuthology.orchestra.run.smithi160.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T16:14:35.800 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-17T16:14:35.816 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-17T16:14:35.825 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T16:14:35.826 INFO:teuthology.orchestra.run.smithi105.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-17T16:14:35.879 INFO:teuthology.orchestra.run.smithi160.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T16:14:35.892 INFO:teuthology.orchestra.run.smithi160.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T16:14:35.893 INFO:teuthology.orchestra.run.smithi160.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T16:14:35.923 INFO:teuthology.orchestra.run.smithi105.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T16:14:35.924 INFO:teuthology.orchestra.run.smithi105.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-17T16:14:35.935 INFO:teuthology.orchestra.run.smithi105.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T16:14:35.935 INFO:teuthology.orchestra.run.smithi105.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-17T16:14:35.965 INFO:teuthology.orchestra.run.smithi160.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T16:14:35.965 INFO:teuthology.orchestra.run.smithi160.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T16:14:35.976 INFO:teuthology.orchestra.run.smithi105.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-17T16:14:35.982 INFO:teuthology.orchestra.run.smithi160.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T16:14:35.983 INFO:teuthology.orchestra.run.smithi160.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T16:14:35.993 INFO:teuthology.orchestra.run.smithi105.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-17T16:14:35.996 INFO:teuthology.orchestra.run.smithi105.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T16:14:35.996 INFO:teuthology.orchestra.run.smithi105.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T16:14:35.997 INFO:teuthology.orchestra.run.smithi105.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T16:14:36.012 INFO:teuthology.orchestra.run.smithi160.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T16:14:36.032 INFO:teuthology.orchestra.run.smithi160.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T16:14:36.037 INFO:teuthology.orchestra.run.smithi160.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-17T16:14:36.043 INFO:teuthology.orchestra.run.smithi160.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T16:14:36.044 INFO:teuthology.orchestra.run.smithi160.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T16:14:36.062 INFO:teuthology.orchestra.run.smithi105.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T16:14:36.063 INFO:teuthology.orchestra.run.smithi105.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T16:14:36.116 INFO:teuthology.orchestra.run.smithi160.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T16:14:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T16:14:36.117 INFO:teuthology.orchestra.run.smithi160.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T16:14:36.118 INFO:teuthology.orchestra.run.smithi160.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T16:14:36.120 INFO:teuthology.orchestra.run.smithi160.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T16:14:36.125 INFO:teuthology.orchestra.run.smithi160.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T16:14:36.125 INFO:teuthology.orchestra.run.smithi160.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T16:14:36.126 INFO:teuthology.orchestra.run.smithi160.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T16:14:36.207 INFO:teuthology.orchestra.run.smithi160.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T16:14:36.207 INFO:teuthology.orchestra.run.smithi160.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T16:14:41.467 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 16.2 MB in 2s (7,614 kB/s) 2024-09-17T16:14:41.564 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 16.2 MB in 2s (7,439 kB/s) 2024-09-17T16:14:42.825 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:14:42.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T16:14:42.896 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:14:42.990 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:14:43.007 DEBUG:teuthology.orchestra.run.smithi160:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T16:14:43.082 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-09-17T16:14:43.083 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-09-17T16:14:43.110 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:14:43.251 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:14:43.252 INFO:teuthology.orchestra.run.smithi105.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:14:43.252 INFO:teuthology.orchestra.run.smithi105.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:14:43.252 INFO:teuthology.orchestra.run.smithi105.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:14:43.252 INFO:teuthology.orchestra.run.smithi105.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T16:14:43.252 INFO:teuthology.orchestra.run.smithi105.stdout: python2.7-minimal 2024-09-17T16:14:43.253 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:14:43.254 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2024-09-17T16:14:43.254 INFO:teuthology.orchestra.run.smithi105.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T16:14:43.254 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T16:14:43.254 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T16:14:43.255 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2024-09-17T16:14:43.256 INFO:teuthology.orchestra.run.smithi105.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T16:14:43.256 INFO:teuthology.orchestra.run.smithi105.stdout:Recommended packages: 2024-09-17T16:14:43.256 INFO:teuthology.orchestra.run.smithi105.stdout: thermald 2024-09-17T16:14:43.304 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2024-09-17T16:14:43.304 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T16:14:43.304 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T16:14:43.305 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T16:14:43.306 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages will be upgraded: 2024-09-17T16:14:43.306 INFO:teuthology.orchestra.run.smithi105.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T16:14:43.317 INFO:teuthology.orchestra.run.smithi160.stdout:Building dependency tree... 2024-09-17T16:14:43.318 INFO:teuthology.orchestra.run.smithi160.stdout:Reading state information... 2024-09-17T16:14:43.358 INFO:teuthology.orchestra.run.smithi105.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T16:14:43.358 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 113 MB of archives. 2024-09-17T16:14:43.359 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T16:14:43.359 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T16:14:43.499 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:14:43.499 INFO:teuthology.orchestra.run.smithi160.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:14:43.499 INFO:teuthology.orchestra.run.smithi160.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:14:43.499 INFO:teuthology.orchestra.run.smithi160.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:14:43.500 INFO:teuthology.orchestra.run.smithi160.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T16:14:43.500 INFO:teuthology.orchestra.run.smithi160.stdout: python2.7-minimal 2024-09-17T16:14:43.500 INFO:teuthology.orchestra.run.smithi160.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:14:43.501 INFO:teuthology.orchestra.run.smithi160.stdout:The following additional packages will be installed: 2024-09-17T16:14:43.501 INFO:teuthology.orchestra.run.smithi160.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T16:14:43.501 INFO:teuthology.orchestra.run.smithi160.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T16:14:43.501 INFO:teuthology.orchestra.run.smithi160.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T16:14:43.503 INFO:teuthology.orchestra.run.smithi160.stdout:Suggested packages: 2024-09-17T16:14:43.503 INFO:teuthology.orchestra.run.smithi160.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T16:14:43.503 INFO:teuthology.orchestra.run.smithi160.stdout:Recommended packages: 2024-09-17T16:14:43.503 INFO:teuthology.orchestra.run.smithi160.stdout: thermald 2024-09-17T16:14:43.549 INFO:teuthology.orchestra.run.smithi160.stdout:The following NEW packages will be installed: 2024-09-17T16:14:43.549 INFO:teuthology.orchestra.run.smithi160.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T16:14:43.549 INFO:teuthology.orchestra.run.smithi160.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T16:14:43.549 INFO:teuthology.orchestra.run.smithi160.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T16:14:43.550 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages will be upgraded: 2024-09-17T16:14:43.551 INFO:teuthology.orchestra.run.smithi160.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T16:14:43.604 INFO:teuthology.orchestra.run.smithi160.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T16:14:43.604 INFO:teuthology.orchestra.run.smithi160.stdout:Need to get 113 MB of archives. 2024-09-17T16:14:43.604 INFO:teuthology.orchestra.run.smithi160.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T16:14:43.604 INFO:teuthology.orchestra.run.smithi160.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T16:14:43.783 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T16:14:43.871 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T16:14:44.046 INFO:teuthology.orchestra.run.smithi160.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T16:14:44.144 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T16:14:44.673 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T16:14:44.673 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T16:14:44.673 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T16:14:44.793 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T16:14:44.813 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T16:14:44.951 INFO:teuthology.orchestra.run.smithi160.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T16:14:44.951 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T16:14:44.951 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T16:14:45.073 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T16:14:45.094 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T16:14:45.192 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 113 MB in 1s (75.5 MB/s) 2024-09-17T16:14:45.367 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T16:14:45.480 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 113 MB in 2s (73.8 MB/s) 2024-09-17T16:14:45.649 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T16:14:47.422 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T16:14:47.426 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:47.587 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T16:14:47.591 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:47.605 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:47.813 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:49.955 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T16:14:49.976 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:50.023 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T16:14:50.044 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:50.062 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:50.134 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:50.524 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T16:14:50.545 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:50.591 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:50.610 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T16:14:50.630 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:14:50.660 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:14:57.022 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:14:57.141 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:14:57.448 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:14:57.558 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:14:57.561 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:14:57.710 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:14:57.889 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T16:14:57.911 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T16:14:57.956 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T16:14:58.051 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:14:58.172 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:14:58.467 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T16:14:58.489 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T16:14:58.525 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T16:15:03.161 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T16:15:03.198 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:15:03.236 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:04.501 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T16:15:04.537 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T16:15:04.592 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:06.175 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:15:06.297 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:15:06.640 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T16:15:06.761 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:06.887 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T16:15:07.016 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:07.306 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T16:15:07.429 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T16:15:07.762 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T16:15:07.892 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:08.009 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T16:15:08.139 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:08.332 INFO:teuthology.orchestra.run.smithi160.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:08.333 INFO:teuthology.orchestra.run.smithi160.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:08.333 INFO:teuthology.orchestra.run.smithi160.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:08.333 INFO:teuthology.orchestra.run.smithi160.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:08.558 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:09.463 INFO:teuthology.orchestra.run.smithi105.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:09.463 INFO:teuthology.orchestra.run.smithi105.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:09.463 INFO:teuthology.orchestra.run.smithi105.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:09.463 INFO:teuthology.orchestra.run.smithi105.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:09.623 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:10.018 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T16:15:10.152 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T16:15:11.179 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:11.600 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T16:15:11.701 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T16:15:11.852 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:12.580 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:12.647 INFO:teuthology.orchestra.run.smithi160.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T16:15:12.648 INFO:teuthology.orchestra.run.smithi160.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:13.262 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T16:15:13.331 INFO:teuthology.orchestra.run.smithi105.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T16:15:13.331 INFO:teuthology.orchestra.run.smithi105.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:27.266 INFO:teuthology.orchestra.run.smithi160.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T16:15:27.266 INFO:teuthology.orchestra.run.smithi160.stdout:Sourcing file `/etc/default/grub' 2024-09-17T16:15:27.285 INFO:teuthology.orchestra.run.smithi160.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T16:15:27.312 INFO:teuthology.orchestra.run.smithi160.stdout:Generating grub configuration file ... 2024-09-17T16:15:27.603 INFO:teuthology.orchestra.run.smithi160.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:27.623 INFO:teuthology.orchestra.run.smithi160.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:27.846 INFO:teuthology.orchestra.run.smithi105.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T16:15:27.846 INFO:teuthology.orchestra.run.smithi105.stdout:Sourcing file `/etc/default/grub' 2024-09-17T16:15:27.872 INFO:teuthology.orchestra.run.smithi105.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T16:15:27.891 INFO:teuthology.orchestra.run.smithi105.stdout:Generating grub configuration file ... 2024-09-17T16:15:27.915 INFO:teuthology.orchestra.run.smithi160.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:27.921 INFO:teuthology.orchestra.run.smithi160.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:27.990 INFO:teuthology.orchestra.run.smithi160.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T16:15:27.995 INFO:teuthology.orchestra.run.smithi160.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T16:15:28.148 INFO:teuthology.orchestra.run.smithi160.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T16:15:28.148 INFO:teuthology.orchestra.run.smithi160.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T16:15:28.148 INFO:teuthology.orchestra.run.smithi160.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T16:15:28.191 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:28.196 INFO:teuthology.orchestra.run.smithi160.stdout:done 2024-09-17T16:15:28.205 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:28.523 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:28.529 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:28.598 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T16:15:28.604 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T16:15:28.707 DEBUG:teuthology.orchestra.run.smithi160:> dpkg -s linux-image-generic 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Package: linux-image-generic 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Status: install ok installed 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Priority: optional 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Section: kernel 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Installed-Size: 21 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T16:15:28.733 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: amd64 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Source: linux-meta 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Version: 5.15.0.122.122 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Recommends: thermald 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout:Description: Generic Linux kernel image 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout: This package will always depend on the latest generic kernel image 2024-09-17T16:15:28.734 INFO:teuthology.orchestra.run.smithi160.stdout: available. 2024-09-17T16:15:28.734 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T16:15:28.734 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T16:15:28.735 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T16:15:28.735 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-09-17T16:15:28.735 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T16:15:28.735 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get clean 2024-09-17T16:15:28.777 INFO:teuthology.orchestra.run.smithi105.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T16:15:28.777 INFO:teuthology.orchestra.run.smithi105.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T16:15:28.777 INFO:teuthology.orchestra.run.smithi105.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T16:15:28.822 INFO:teuthology.orchestra.run.smithi105.stdout:done 2024-09-17T16:15:28.907 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get update 2024-09-17T16:15:29.033 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:15:29.040 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T16:15:29.058 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T16:15:29.151 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T16:15:29.289 DEBUG:teuthology.orchestra.run.smithi105:> dpkg -s linux-image-generic 2024-09-17T16:15:29.308 INFO:teuthology.orchestra.run.smithi105.stdout:Package: linux-image-generic 2024-09-17T16:15:29.308 INFO:teuthology.orchestra.run.smithi105.stdout:Status: install ok installed 2024-09-17T16:15:29.308 INFO:teuthology.orchestra.run.smithi105.stdout:Priority: optional 2024-09-17T16:15:29.308 INFO:teuthology.orchestra.run.smithi105.stdout:Section: kernel 2024-09-17T16:15:29.308 INFO:teuthology.orchestra.run.smithi105.stdout:Installed-Size: 21 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: amd64 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Source: linux-meta 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Version: 5.15.0.122.122 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Recommends: thermald 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout:Description: Generic Linux kernel image 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout: This package will always depend on the latest generic kernel image 2024-09-17T16:15:29.309 INFO:teuthology.orchestra.run.smithi105.stdout: available. 2024-09-17T16:15:29.309 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T16:15:29.309 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T16:15:29.309 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T16:15:29.310 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T16:15:29.310 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T16:15:29.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get clean 2024-09-17T16:15:29.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-09-17T16:15:29.639 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:15:29.649 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T16:15:29.666 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T16:15:29.761 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T16:15:30.468 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:15:30.486 DEBUG:teuthology.orchestra.run.smithi160:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T16:15:30.546 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:15:30.758 INFO:teuthology.orchestra.run.smithi160.stdout:Building dependency tree... 2024-09-17T16:15:30.758 INFO:teuthology.orchestra.run.smithi160.stdout:Reading state information... 2024-09-17T16:15:30.948 INFO:teuthology.orchestra.run.smithi160.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T16:15:30.948 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:15:30.949 INFO:teuthology.orchestra.run.smithi160.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:15:30.949 INFO:teuthology.orchestra.run.smithi160.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:15:30.949 INFO:teuthology.orchestra.run.smithi160.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:15:30.949 INFO:teuthology.orchestra.run.smithi160.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T16:15:30.949 INFO:teuthology.orchestra.run.smithi160.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T16:15:30.950 INFO:teuthology.orchestra.run.smithi160.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:15:30.989 INFO:teuthology.orchestra.run.smithi160.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T16:15:30.991 DEBUG:teuthology.orchestra.run.smithi160:> dpkg -s linux-image-generic 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Package: linux-image-generic 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Status: install ok installed 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Priority: optional 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Section: kernel 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Installed-Size: 21 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: amd64 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Source: linux-meta 2024-09-17T16:15:31.007 INFO:teuthology.orchestra.run.smithi160.stdout:Version: 5.15.0.122.122 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout:Recommends: thermald 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout:Description: Generic Linux kernel image 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout: This package will always depend on the latest generic kernel image 2024-09-17T16:15:31.008 INFO:teuthology.orchestra.run.smithi160.stdout: available. 2024-09-17T16:15:31.008 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-17T16:15:31.027 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:15:31.045 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T16:15:31.057 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.ZT0sdVRsf3 2024-09-17T16:15:31.058 DEBUG:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ZT0sdVRsf3 2024-09-17T16:15:31.105 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:15:31.117 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.ZT0sdVRsf3 2024-09-17T16:15:31.231 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.ZT0sdVRsf3 is 10KB 2024-09-17T16:15:31.243 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.ZT0sdVRsf3 2024-09-17T16:15:31.249 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.305 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:15:31.305 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.305 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.315 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-09-17T16:15:31.316 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-09-17T16:15:31.373 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T16:15:31.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo update-grub 2024-09-17T16:15:31.498 INFO:teuthology.orchestra.run.smithi105.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T16:15:31.499 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:15:31.499 INFO:teuthology.orchestra.run.smithi105.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:15:31.499 INFO:teuthology.orchestra.run.smithi105.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:15:31.499 INFO:teuthology.orchestra.run.smithi105.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:15:31.499 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T16:15:31.500 INFO:teuthology.orchestra.run.smithi105.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T16:15:31.500 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:15:31.538 INFO:teuthology.orchestra.run.smithi105.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T16:15:31.539 DEBUG:teuthology.orchestra.run.smithi105:> dpkg -s linux-image-generic 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Package: linux-image-generic 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Status: install ok installed 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Priority: optional 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Section: kernel 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Installed-Size: 21 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T16:15:31.554 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: amd64 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Source: linux-meta 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Version: 5.15.0.122.122 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Recommends: thermald 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:Description: Generic Linux kernel image 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout: This package will always depend on the latest generic kernel image 2024-09-17T16:15:31.555 INFO:teuthology.orchestra.run.smithi105.stdout: available. 2024-09-17T16:15:31.555 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:15:31.601 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.5QwSKuzqZ3 2024-09-17T16:15:31.602 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5QwSKuzqZ3 2024-09-17T16:15:31.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.5QwSKuzqZ3 2024-09-17T16:15:31.782 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.5QwSKuzqZ3 is 10KB 2024-09-17T16:15:31.795 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.5QwSKuzqZ3 2024-09-17T16:15:31.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.856 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:15:31.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T16:15:31.921 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T16:15:31.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo update-grub 2024-09-17T16:15:32.566 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub' 2024-09-17T16:15:32.567 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T16:15:32.569 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2024-09-17T16:15:32.751 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:32.757 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:32.991 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:32.996 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:33.039 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T16:15:33.044 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T16:15:33.174 INFO:teuthology.orchestra.run.smithi160.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T16:15:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T16:15:33.175 INFO:teuthology.orchestra.run.smithi160.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T16:15:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub' 2024-09-17T16:15:33.187 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T16:15:33.189 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-09-17T16:15:33.191 INFO:teuthology.orchestra.run.smithi160.stderr:done 2024-09-17T16:15:33.207 DEBUG:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2024-09-17T16:15:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T16:15:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T16:15:33.607 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T16:15:33.612 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T16:15:33.653 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T16:15:33.659 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T16:15:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T16:15:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T16:15:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T16:15:33.814 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-09-17T16:15:33.829 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-09-17T16:16:03.237 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-09-17T16:16:03.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:16:03.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:16:03.833 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T16:16:03.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:16:03.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:16:21.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-17T16:16:22.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-17T16:16:30.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:16:30.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:16:31.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:16:31.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:16:33.633 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-17T16:16:34.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-17T16:16:45.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:16:45.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:16:46.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:16:46.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:17:04.829 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-17T16:17:19.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:17:19.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:17:20.185 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-09-17T16:17:20.297 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-17T16:17:21.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:17:21.142 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-17T16:17:21.143 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-17T16:17:21.189 INFO:teuthology.orchestra.run.smithi105.stdout:5.15.0-122-generic 2024-09-17T16:17:21.189 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T16:17:21.189 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T16:17:21.189 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T16:17:22.190 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T16:17:22.191 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T16:17:22.306 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-09-17T16:17:22.330 DEBUG:teuthology.parallel:result is None 2024-09-17T16:17:35.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:17:35.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:17:35.533 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-09-17T16:17:36.177 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:17:36.177 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.15.0-122-generic"... 2024-09-17T16:17:36.178 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-09-17T16:17:36.224 INFO:teuthology.orchestra.run.smithi160.stdout:5.15.0-122-generic 2024-09-17T16:17:36.224 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T16:17:36.224 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T16:17:36.224 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T16:17:37.225 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-09-17T16:17:37.225 DEBUG:teuthology.orchestra.run.smithi160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T16:17:37.612 INFO:teuthology.orchestra.run.smithi160.stdout:ttyS1 2024-09-17T16:17:37.638 DEBUG:teuthology.parallel:result is None 2024-09-17T16:17:37.638 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T16:17:37.646 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T16:17:37.646 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T16:17:37.649 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T16:17:37.654 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T16:17:37.757 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T16:17:37.824 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T16:17:37.825 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T16:17:37.827 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T16:17:37.850 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T16:17:37.857 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T16:17:37.857 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T16:17:37.881 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T16:17:37.902 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T16:17:37.910 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T16:17:37.912 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T16:17:37.920 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T16:17:37.921 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T16:17:38.029 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T16:17:38.030 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T16:17:38.032 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T16:17:38.046 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T16:17:38.167 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T16:17:38.168 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T16:17:38.170 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T16:17:38.176 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T16:17:38.221 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T16:17:38.268 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:17:38.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T16:17:38.335 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T16:17:38.340 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T16:17:38.388 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:17:38.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T16:17:38.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-09-17T16:17:38.456 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-09-17T16:17:38.532 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T16:17:38.542 INFO:teuthology.task.internal:Starting timer... 2024-09-17T16:17:38.542 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T16:17:38.610 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T16:17:38.644 DEBUG:teuthology.task.selinux:Excluding smithi105: OS 'ubuntu' does not support SELinux 2024-09-17T16:17:38.645 DEBUG:teuthology.task.selinux:Excluding smithi160: OS 'ubuntu' does not support SELinux 2024-09-17T16:17:38.645 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T16:17:38.645 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T16:17:38.645 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T16:17:38.645 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T16:17:38.803 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T16:17:38.822 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T16:17:38.823 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi160.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T16:25:05.479 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com')] 2024-09-17T16:25:05.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:25:05.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:25:05.565 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-17T16:25:05.641 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-17T16:25:05.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:25:05.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T16:25:05.721 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-09-17T16:25:05.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-09-17T16:25:05.801 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T16:25:05.811 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T16:25:05.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T16:25:05.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T16:25:05.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T16:25:05.814 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T16:25:05.842 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T16:25:05.842 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Command line: ntpd -gq 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: ---------------------------------------------------- 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: corporation. Support and training for ntp-4 are 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: available at https://www.nwtime.org/support 2024-09-17T16:25:05.843 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: ---------------------------------------------------- 2024-09-17T16:25:05.844 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: proto: precision = 0.063 usec (-24) 2024-09-17T16:25:05.844 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: basedate set to 2022-02-04 2024-09-17T16:25:05.844 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: gps base set to 2022-02-06 (week 2196) 2024-09-17T16:25:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T16:25:05.845 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T16:25:05.845 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T16:25:05.845 INFO:teuthology.orchestra.run.smithi105.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T16:25:05.846 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen normally on 3 enp3s0f1 172.21.15.105:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen normally on 4 lo [::1]:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d429%5]:123 2024-09-17T16:25:05.847 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:05 ntpd[18460]: Listening on routing socket on fd #22 for interface updates 2024-09-17T16:25:05.869 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Command line: ntpd -gq 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: ---------------------------------------------------- 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: corporation. Support and training for ntp-4 are 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: available at https://www.nwtime.org/support 2024-09-17T16:25:05.870 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: ---------------------------------------------------- 2024-09-17T16:25:05.871 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: proto: precision = 0.057 usec (-24) 2024-09-17T16:25:05.871 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: basedate set to 2022-02-04 2024-09-17T16:25:05.871 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: gps base set to 2022-02-06 (week 2196) 2024-09-17T16:25:05.871 INFO:teuthology.orchestra.run.smithi160.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T16:25:05.871 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T16:25:05.872 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T16:25:05.872 INFO:teuthology.orchestra.run.smithi160.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen normally on 3 enp3s0f1 172.21.15.160:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen normally on 4 lo [::1]:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5b%5]:123 2024-09-17T16:25:05.873 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:05 ntpd[18485]: Listening on routing socket on fd #22 for interface updates 2024-09-17T16:25:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:06 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:06.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:06 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:07.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:07 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:07.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:07 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:08.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:08 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:08.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:08.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:08.870 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:08.871 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:08 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:09 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:10 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:10.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:10.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:10 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:11 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.846 INFO:teuthology.orchestra.run.smithi105.stdout:17 Sep 16:25:12 ntpd[18460]: ntpd: time slew +0.000082 s 2024-09-17T16:25:12.846 INFO:teuthology.orchestra.run.smithi105.stdout:ntpd: time slew +0.000082s 2024-09-17T16:25:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T16:25:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:17 Sep 16:25:12 ntpd[18460]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T16:25:12.871 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.871 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.871 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.872 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:12.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T16:25:12.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T16:25:12.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T16:25:12.873 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T16:25:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:17 Sep 16:25:12 ntpd[18485]: ntpd: time slew -0.001499 s 2024-09-17T16:25:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:ntpd: time slew -0.001499s 2024-09-17T16:25:12.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T16:25:12.874 INFO:teuthology.orchestra.run.smithi160.stderr:17 Sep 16:25:12 ntpd[18485]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.888 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout:============================================================================== 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.918 INFO:teuthology.orchestra.run.smithi160.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T16:25:12.919 INFO:teuthology.run_tasks:Running task install... 2024-09-17T16:25:12.928 DEBUG:teuthology.task.install:project ceph 2024-09-17T16:25:12.928 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}} 2024-09-17T16:25:12.928 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'} 2024-09-17T16:25:12.928 INFO:teuthology.task.install:Using flavor: default 2024-09-17T16:25:12.934 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T16:25:12.934 INFO:teuthology.task.install:extra packages: [] 2024-09-17T16:25:12.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-key list | grep Ceph 2024-09-17T16:25:12.935 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-key list | grep Ceph 2024-09-17T16:25:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T16:25:13.013 INFO:teuthology.orchestra.run.smithi160.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T16:25:13.024 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T16:25:13.024 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T16:25:13.025 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-17T16:25:13.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:25:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T16:25:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T16:25:13.040 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-17T16:25:13.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:25:13.176 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default/ 2024-09-17T16:25:13.176 INFO:teuthology.task.install.deb:Package version is 19.3.0-4975-g1889a165-1jammy 2024-09-17T16:25:13.183 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default/ 2024-09-17T16:25:13.183 INFO:teuthology.task.install.deb:Package version is 19.3.0-4975-g1889a165-1jammy 2024-09-17T16:25:13.300 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:25:13.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T16:25:13.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-09-17T16:25:13.330 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:25:13.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T16:25:13.342 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get update 2024-09-17T16:25:13.484 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T16:25:13.505 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:25:13.506 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T16:25:13.517 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T16:25:13.533 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T16:25:13.570 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T16:25:13.581 INFO:teuthology.orchestra.run.smithi160.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T16:25:13.592 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-17T16:25:13.602 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-17T16:25:13.606 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T16:25:13.618 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T16:25:13.624 INFO:teuthology.orchestra.run.smithi160.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T16:25:13.639 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T16:25:13.645 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T16:25:13.666 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2024-09-17T16:25:13.672 INFO:teuthology.orchestra.run.smithi160.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2024-09-17T16:25:13.688 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T16:25:13.770 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T16:25:13.836 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 28.6 kB in 0s (89.9 kB/s) 2024-09-17T16:25:13.905 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 28.6 kB in 0s (69.8 kB/s) 2024-09-17T16:25:14.881 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:25:14.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4975-g1889a165-1jammy cephadm=19.3.0-4975-g1889a165-1jammy ceph-mds=19.3.0-4975-g1889a165-1jammy ceph-mgr=19.3.0-4975-g1889a165-1jammy ceph-common=19.3.0-4975-g1889a165-1jammy ceph-fuse=19.3.0-4975-g1889a165-1jammy ceph-test=19.3.0-4975-g1889a165-1jammy ceph-volume=19.3.0-4975-g1889a165-1jammy radosgw=19.3.0-4975-g1889a165-1jammy python3-rados=19.3.0-4975-g1889a165-1jammy python3-rgw=19.3.0-4975-g1889a165-1jammy python3-cephfs=19.3.0-4975-g1889a165-1jammy python3-rbd=19.3.0-4975-g1889a165-1jammy libcephfs2=19.3.0-4975-g1889a165-1jammy libcephfs-dev=19.3.0-4975-g1889a165-1jammy librados2=19.3.0-4975-g1889a165-1jammy librbd1=19.3.0-4975-g1889a165-1jammy rbd-fuse=19.3.0-4975-g1889a165-1jammy 2024-09-17T16:25:14.901 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:25:14.918 DEBUG:teuthology.orchestra.run.smithi160:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4975-g1889a165-1jammy cephadm=19.3.0-4975-g1889a165-1jammy ceph-mds=19.3.0-4975-g1889a165-1jammy ceph-mgr=19.3.0-4975-g1889a165-1jammy ceph-common=19.3.0-4975-g1889a165-1jammy ceph-fuse=19.3.0-4975-g1889a165-1jammy ceph-test=19.3.0-4975-g1889a165-1jammy ceph-volume=19.3.0-4975-g1889a165-1jammy radosgw=19.3.0-4975-g1889a165-1jammy python3-rados=19.3.0-4975-g1889a165-1jammy python3-rgw=19.3.0-4975-g1889a165-1jammy python3-cephfs=19.3.0-4975-g1889a165-1jammy python3-rbd=19.3.0-4975-g1889a165-1jammy libcephfs2=19.3.0-4975-g1889a165-1jammy libcephfs-dev=19.3.0-4975-g1889a165-1jammy librados2=19.3.0-4975-g1889a165-1jammy librbd1=19.3.0-4975-g1889a165-1jammy rbd-fuse=19.3.0-4975-g1889a165-1jammy 2024-09-17T16:25:14.966 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-09-17T16:25:14.983 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2024-09-17T16:25:15.173 INFO:teuthology.orchestra.run.smithi160.stdout:Building dependency tree... 2024-09-17T16:25:15.174 INFO:teuthology.orchestra.run.smithi160.stdout:Reading state information... 2024-09-17T16:25:15.175 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-09-17T16:25:15.176 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-09-17T16:25:15.361 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T16:25:15.362 INFO:teuthology.orchestra.run.smithi160.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout:The following additional packages will be installed: 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T16:25:15.363 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-17T16:25:15.364 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-17T16:25:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet zip 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout:Suggested packages: 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T16:25:15.366 INFO:teuthology.orchestra.run.smithi160.stdout:Recommended packages: 2024-09-17T16:25:15.367 INFO:teuthology.orchestra.run.smithi160.stdout: btrfs-tools 2024-09-17T16:25:15.384 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T16:25:15.385 INFO:teuthology.orchestra.run.smithi105.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T16:25:15.385 INFO:teuthology.orchestra.run.smithi105.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T16:25:15.385 INFO:teuthology.orchestra.run.smithi105.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T16:25:15.385 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T16:25:15.385 INFO:teuthology.orchestra.run.smithi105.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T16:25:15.386 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T16:25:15.386 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2024-09-17T16:25:15.386 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T16:25:15.387 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T16:25:15.387 INFO:teuthology.orchestra.run.smithi105.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T16:25:15.387 INFO:teuthology.orchestra.run.smithi105.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T16:25:15.387 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-17T16:25:15.388 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-17T16:25:15.389 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet zip 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout:Recommended packages: 2024-09-17T16:25:15.390 INFO:teuthology.orchestra.run.smithi105.stdout: btrfs-tools 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout:The following NEW packages will be installed: 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-17T16:25:15.469 INFO:teuthology.orchestra.run.smithi160.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-17T16:25:15.470 INFO:teuthology.orchestra.run.smithi160.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-17T16:25:15.470 INFO:teuthology.orchestra.run.smithi160.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-17T16:25:15.470 INFO:teuthology.orchestra.run.smithi160.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-17T16:25:15.471 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-17T16:25:15.472 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-17T16:25:15.472 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-17T16:25:15.472 INFO:teuthology.orchestra.run.smithi160.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-17T16:25:15.493 INFO:teuthology.orchestra.run.smithi105.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-17T16:25:15.494 INFO:teuthology.orchestra.run.smithi105.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-17T16:25:15.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-17T16:25:15.496 INFO:teuthology.orchestra.run.smithi105.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-17T16:25:15.613 INFO:teuthology.orchestra.run.smithi160.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T16:25:15.614 INFO:teuthology.orchestra.run.smithi160.stdout:Need to get 244 MB of archives. 2024-09-17T16:25:15.614 INFO:teuthology.orchestra.run.smithi160.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-17T16:25:15.614 INFO:teuthology.orchestra.run.smithi160.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4975-g1889a165-1jammy [3,624 kB] 2024-09-17T16:25:15.637 INFO:teuthology.orchestra.run.smithi105.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T16:25:15.637 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 244 MB of archives. 2024-09-17T16:25:15.637 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-17T16:25:15.637 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4975-g1889a165-1jammy [3,624 kB] 2024-09-17T16:25:15.658 INFO:teuthology.orchestra.run.smithi160.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T16:25:15.683 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T16:25:15.828 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4975-g1889a165-1jammy [3,332 kB] 2024-09-17T16:25:15.935 INFO:teuthology.orchestra.run.smithi160.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4975-g1889a165-1jammy [722 kB] 2024-09-17T16:25:15.949 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4975-g1889a165-1jammy [356 kB] 2024-09-17T16:25:15.957 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4975-g1889a165-1jammy [32.8 kB] 2024-09-17T16:25:15.957 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4975-g1889a165-1jammy [180 kB] 2024-09-17T16:25:15.961 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4975-g1889a165-1jammy [71.3 kB] 2024-09-17T16:25:15.962 INFO:teuthology.orchestra.run.smithi160.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4975-g1889a165-1jammy [338 kB] 2024-09-17T16:25:15.978 INFO:teuthology.orchestra.run.smithi160.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4975-g1889a165-1jammy [7,966 kB] 2024-09-17T16:25:16.008 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4975-g1889a165-1jammy [3,332 kB] 2024-09-17T16:25:16.079 INFO:teuthology.orchestra.run.smithi160.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T16:25:16.092 INFO:teuthology.orchestra.run.smithi160.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T16:25:16.106 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T16:25:16.118 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T16:25:16.173 INFO:teuthology.orchestra.run.smithi160.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T16:25:16.199 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T16:25:16.216 INFO:teuthology.orchestra.run.smithi160.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4975-g1889a165-1jammy [112 kB] 2024-09-17T16:25:16.219 INFO:teuthology.orchestra.run.smithi160.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4975-g1889a165-1jammy [473 kB] 2024-09-17T16:25:16.228 INFO:teuthology.orchestra.run.smithi160.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4975-g1889a165-1jammy [28.2 MB] 2024-09-17T16:25:16.255 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4975-g1889a165-1jammy [722 kB] 2024-09-17T16:25:16.317 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4975-g1889a165-1jammy [356 kB] 2024-09-17T16:25:16.325 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4975-g1889a165-1jammy [32.8 kB] 2024-09-17T16:25:16.326 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4975-g1889a165-1jammy [180 kB] 2024-09-17T16:25:16.331 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4975-g1889a165-1jammy [71.3 kB] 2024-09-17T16:25:16.379 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4975-g1889a165-1jammy [338 kB] 2024-09-17T16:25:16.388 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4975-g1889a165-1jammy [7,966 kB] 2024-09-17T16:25:16.403 INFO:teuthology.orchestra.run.smithi160.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T16:25:16.413 INFO:teuthology.orchestra.run.smithi160.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T16:25:16.428 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T16:25:16.439 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T16:25:16.443 INFO:teuthology.orchestra.run.smithi160.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T16:25:16.452 INFO:teuthology.orchestra.run.smithi160.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T16:25:16.454 INFO:teuthology.orchestra.run.smithi160.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T16:25:16.455 INFO:teuthology.orchestra.run.smithi160.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T16:25:16.457 INFO:teuthology.orchestra.run.smithi160.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T16:25:16.469 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T16:25:16.474 INFO:teuthology.orchestra.run.smithi160.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T16:25:16.477 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T16:25:16.478 INFO:teuthology.orchestra.run.smithi160.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T16:25:16.480 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T16:25:16.481 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T16:25:16.482 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T16:25:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T16:25:16.499 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T16:25:16.503 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T16:25:16.510 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T16:25:16.565 INFO:teuthology.orchestra.run.smithi160.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T16:25:16.565 INFO:teuthology.orchestra.run.smithi160.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T16:25:16.572 INFO:teuthology.orchestra.run.smithi160.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T16:25:16.574 INFO:teuthology.orchestra.run.smithi160.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T16:25:16.578 INFO:teuthology.orchestra.run.smithi160.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T16:25:16.579 INFO:teuthology.orchestra.run.smithi160.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T16:25:16.580 INFO:teuthology.orchestra.run.smithi160.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T16:25:16.582 INFO:teuthology.orchestra.run.smithi160.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T16:25:16.582 INFO:teuthology.orchestra.run.smithi160.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T16:25:16.589 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T16:25:16.590 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T16:25:16.594 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T16:25:16.598 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T16:25:16.601 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T16:25:16.603 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T16:25:16.603 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T16:25:16.605 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T16:25:16.605 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T16:25:16.649 INFO:teuthology.orchestra.run.smithi160.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T16:25:16.665 INFO:teuthology.orchestra.run.smithi160.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T16:25:16.666 INFO:teuthology.orchestra.run.smithi160.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T16:25:16.667 INFO:teuthology.orchestra.run.smithi160.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T16:25:16.668 INFO:teuthology.orchestra.run.smithi160.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T16:25:16.673 INFO:teuthology.orchestra.run.smithi160.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T16:25:16.673 INFO:teuthology.orchestra.run.smithi105.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T16:25:16.689 INFO:teuthology.orchestra.run.smithi105.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T16:25:16.689 INFO:teuthology.orchestra.run.smithi105.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T16:25:16.690 INFO:teuthology.orchestra.run.smithi105.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T16:25:16.690 INFO:teuthology.orchestra.run.smithi105.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T16:25:16.695 INFO:teuthology.orchestra.run.smithi105.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T16:25:16.731 INFO:teuthology.orchestra.run.smithi160.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T16:25:16.732 INFO:teuthology.orchestra.run.smithi160.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T16:25:16.733 INFO:teuthology.orchestra.run.smithi160.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T16:25:16.734 INFO:teuthology.orchestra.run.smithi160.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T16:25:16.755 INFO:teuthology.orchestra.run.smithi105.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T16:25:16.755 INFO:teuthology.orchestra.run.smithi105.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T16:25:16.756 INFO:teuthology.orchestra.run.smithi105.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T16:25:16.758 INFO:teuthology.orchestra.run.smithi105.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T16:25:16.812 INFO:teuthology.orchestra.run.smithi160.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T16:25:16.813 INFO:teuthology.orchestra.run.smithi160.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T16:25:16.816 INFO:teuthology.orchestra.run.smithi160.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T16:25:16.817 INFO:teuthology.orchestra.run.smithi160.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T16:25:16.818 INFO:teuthology.orchestra.run.smithi160.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T16:25:16.836 INFO:teuthology.orchestra.run.smithi105.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T16:25:16.836 INFO:teuthology.orchestra.run.smithi105.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T16:25:16.839 INFO:teuthology.orchestra.run.smithi105.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T16:25:16.840 INFO:teuthology.orchestra.run.smithi105.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T16:25:16.841 INFO:teuthology.orchestra.run.smithi105.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T16:25:16.882 INFO:teuthology.orchestra.run.smithi105.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4975-g1889a165-1jammy [112 kB] 2024-09-17T16:25:16.885 INFO:teuthology.orchestra.run.smithi105.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4975-g1889a165-1jammy [473 kB] 2024-09-17T16:25:16.894 INFO:teuthology.orchestra.run.smithi160.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T16:25:16.894 INFO:teuthology.orchestra.run.smithi160.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T16:25:16.895 INFO:teuthology.orchestra.run.smithi160.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T16:25:16.897 INFO:teuthology.orchestra.run.smithi160.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T16:25:16.899 INFO:teuthology.orchestra.run.smithi160.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T16:25:16.913 INFO:teuthology.orchestra.run.smithi105.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4975-g1889a165-1jammy [28.2 MB] 2024-09-17T16:25:16.917 INFO:teuthology.orchestra.run.smithi105.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T16:25:16.917 INFO:teuthology.orchestra.run.smithi105.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T16:25:16.919 INFO:teuthology.orchestra.run.smithi105.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T16:25:16.920 INFO:teuthology.orchestra.run.smithi105.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T16:25:16.922 INFO:teuthology.orchestra.run.smithi105.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T16:25:16.975 INFO:teuthology.orchestra.run.smithi160.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T16:25:16.977 INFO:teuthology.orchestra.run.smithi160.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T16:25:16.980 INFO:teuthology.orchestra.run.smithi160.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T16:25:16.982 INFO:teuthology.orchestra.run.smithi160.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T16:25:16.986 INFO:teuthology.orchestra.run.smithi160.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T16:25:16.998 INFO:teuthology.orchestra.run.smithi105.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T16:25:17.000 INFO:teuthology.orchestra.run.smithi105.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T16:25:17.002 INFO:teuthology.orchestra.run.smithi105.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T16:25:17.004 INFO:teuthology.orchestra.run.smithi105.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T16:25:17.009 INFO:teuthology.orchestra.run.smithi105.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T16:25:17.057 INFO:teuthology.orchestra.run.smithi160.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T16:25:17.058 INFO:teuthology.orchestra.run.smithi160.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T16:25:17.063 INFO:teuthology.orchestra.run.smithi160.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T16:25:17.080 INFO:teuthology.orchestra.run.smithi105.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T16:25:17.081 INFO:teuthology.orchestra.run.smithi105.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T16:25:17.085 INFO:teuthology.orchestra.run.smithi105.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T16:25:17.207 INFO:teuthology.orchestra.run.smithi160.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4975-g1889a165-1jammy [5,209 kB] 2024-09-17T16:25:17.227 INFO:teuthology.orchestra.run.smithi160.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T16:25:17.229 INFO:teuthology.orchestra.run.smithi160.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T16:25:17.231 INFO:teuthology.orchestra.run.smithi160.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T16:25:17.232 INFO:teuthology.orchestra.run.smithi160.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T16:25:17.234 INFO:teuthology.orchestra.run.smithi160.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T16:25:17.248 INFO:teuthology.orchestra.run.smithi105.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T16:25:17.250 INFO:teuthology.orchestra.run.smithi105.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T16:25:17.252 INFO:teuthology.orchestra.run.smithi105.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T16:25:17.253 INFO:teuthology.orchestra.run.smithi105.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T16:25:17.255 INFO:teuthology.orchestra.run.smithi105.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T16:25:17.268 INFO:teuthology.orchestra.run.smithi160.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T16:25:17.290 INFO:teuthology.orchestra.run.smithi105.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T16:25:17.293 INFO:teuthology.orchestra.run.smithi105.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T16:25:17.293 INFO:teuthology.orchestra.run.smithi105.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T16:25:17.303 INFO:teuthology.orchestra.run.smithi160.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T16:25:17.304 INFO:teuthology.orchestra.run.smithi160.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T16:25:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4975-g1889a165-1jammy [251 kB] 2024-09-17T16:25:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T16:25:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4975-g1889a165-1jammy [124 kB] 2024-09-17T16:25:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T16:25:17.337 INFO:teuthology.orchestra.run.smithi160.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4975-g1889a165-1jammy [1,561 kB] 2024-09-17T16:25:17.354 INFO:teuthology.orchestra.run.smithi105.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T16:25:17.354 INFO:teuthology.orchestra.run.smithi105.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T16:25:17.393 INFO:teuthology.orchestra.run.smithi160.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T16:25:17.394 INFO:teuthology.orchestra.run.smithi160.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T16:25:17.395 INFO:teuthology.orchestra.run.smithi160.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T16:25:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T16:25:17.397 INFO:teuthology.orchestra.run.smithi160.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T16:25:17.399 INFO:teuthology.orchestra.run.smithi160.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T16:25:17.402 INFO:teuthology.orchestra.run.smithi160.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4975-g1889a165-1jammy [6,310 kB] 2024-09-17T16:25:17.405 INFO:teuthology.orchestra.run.smithi160.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T16:25:17.414 INFO:teuthology.orchestra.run.smithi105.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T16:25:17.416 INFO:teuthology.orchestra.run.smithi105.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T16:25:17.416 INFO:teuthology.orchestra.run.smithi105.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T16:25:17.417 INFO:teuthology.orchestra.run.smithi105.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T16:25:17.418 INFO:teuthology.orchestra.run.smithi105.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T16:25:17.419 INFO:teuthology.orchestra.run.smithi105.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T16:25:17.426 INFO:teuthology.orchestra.run.smithi105.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T16:25:17.475 INFO:teuthology.orchestra.run.smithi160.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T16:25:17.478 INFO:teuthology.orchestra.run.smithi160.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T16:25:17.480 INFO:teuthology.orchestra.run.smithi160.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T16:25:17.496 INFO:teuthology.orchestra.run.smithi105.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T16:25:17.500 INFO:teuthology.orchestra.run.smithi105.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T16:25:17.501 INFO:teuthology.orchestra.run.smithi105.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T16:25:17.557 INFO:teuthology.orchestra.run.smithi160.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T16:25:17.563 INFO:teuthology.orchestra.run.smithi160.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T16:25:17.565 INFO:teuthology.orchestra.run.smithi160.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T16:25:17.566 INFO:teuthology.orchestra.run.smithi160.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T16:25:17.576 INFO:teuthology.orchestra.run.smithi160.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T16:25:17.577 INFO:teuthology.orchestra.run.smithi160.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T16:25:17.578 INFO:teuthology.orchestra.run.smithi105.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T16:25:17.584 INFO:teuthology.orchestra.run.smithi105.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T16:25:17.584 INFO:teuthology.orchestra.run.smithi160.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T16:25:17.586 INFO:teuthology.orchestra.run.smithi105.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T16:25:17.587 INFO:teuthology.orchestra.run.smithi105.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T16:25:17.598 INFO:teuthology.orchestra.run.smithi105.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T16:25:17.599 INFO:teuthology.orchestra.run.smithi105.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T16:25:17.607 INFO:teuthology.orchestra.run.smithi105.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T16:25:17.638 INFO:teuthology.orchestra.run.smithi160.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T16:25:17.640 INFO:teuthology.orchestra.run.smithi160.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T16:25:17.641 INFO:teuthology.orchestra.run.smithi160.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T16:25:17.659 INFO:teuthology.orchestra.run.smithi105.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T16:25:17.660 INFO:teuthology.orchestra.run.smithi105.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T16:25:17.661 INFO:teuthology.orchestra.run.smithi105.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T16:25:17.688 INFO:teuthology.orchestra.run.smithi160.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4975-g1889a165-1jammy [23.1 MB] 2024-09-17T16:25:17.720 INFO:teuthology.orchestra.run.smithi160.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T16:25:17.722 INFO:teuthology.orchestra.run.smithi160.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T16:25:17.739 INFO:teuthology.orchestra.run.smithi160.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T16:25:17.740 INFO:teuthology.orchestra.run.smithi105.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T16:25:17.740 INFO:teuthology.orchestra.run.smithi160.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T16:25:17.741 INFO:teuthology.orchestra.run.smithi160.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T16:25:17.742 INFO:teuthology.orchestra.run.smithi105.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T16:25:17.746 INFO:teuthology.orchestra.run.smithi160.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T16:25:17.748 INFO:teuthology.orchestra.run.smithi160.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T16:25:17.759 INFO:teuthology.orchestra.run.smithi105.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T16:25:17.760 INFO:teuthology.orchestra.run.smithi105.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T16:25:17.760 INFO:teuthology.orchestra.run.smithi105.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T16:25:17.765 INFO:teuthology.orchestra.run.smithi105.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T16:25:17.767 INFO:teuthology.orchestra.run.smithi105.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T16:25:18.120 INFO:teuthology.orchestra.run.smithi105.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4975-g1889a165-1jammy [5,209 kB] 2024-09-17T16:25:18.224 INFO:teuthology.orchestra.run.smithi105.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4975-g1889a165-1jammy [251 kB] 2024-09-17T16:25:18.230 INFO:teuthology.orchestra.run.smithi105.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4975-g1889a165-1jammy [124 kB] 2024-09-17T16:25:18.233 INFO:teuthology.orchestra.run.smithi105.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4975-g1889a165-1jammy [1,561 kB] 2024-09-17T16:25:18.296 INFO:teuthology.orchestra.run.smithi105.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4975-g1889a165-1jammy [6,310 kB] 2024-09-17T16:25:18.525 INFO:teuthology.orchestra.run.smithi160.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4975-g1889a165-1jammy [14.0 kB] 2024-09-17T16:25:18.526 INFO:teuthology.orchestra.run.smithi160.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4975-g1889a165-1jammy [880 kB] 2024-09-17T16:25:18.534 INFO:teuthology.orchestra.run.smithi105.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4975-g1889a165-1jammy [23.1 MB] 2024-09-17T16:25:18.543 INFO:teuthology.orchestra.run.smithi160.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4975-g1889a165-1jammy [2,452 kB] 2024-09-17T16:25:18.584 INFO:teuthology.orchestra.run.smithi160.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4975-g1889a165-1jammy [813 kB] 2024-09-17T16:25:18.607 INFO:teuthology.orchestra.run.smithi160.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4975-g1889a165-1jammy [160 kB] 2024-09-17T16:25:18.611 INFO:teuthology.orchestra.run.smithi160.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4975-g1889a165-1jammy [61.5 MB] 2024-09-17T16:25:19.519 INFO:teuthology.orchestra.run.smithi105.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4975-g1889a165-1jammy [14.0 kB] 2024-09-17T16:25:19.520 INFO:teuthology.orchestra.run.smithi105.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4975-g1889a165-1jammy [880 kB] 2024-09-17T16:25:19.538 INFO:teuthology.orchestra.run.smithi105.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4975-g1889a165-1jammy [2,452 kB] 2024-09-17T16:25:19.580 INFO:teuthology.orchestra.run.smithi105.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4975-g1889a165-1jammy [813 kB] 2024-09-17T16:25:19.610 INFO:teuthology.orchestra.run.smithi105.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4975-g1889a165-1jammy [160 kB] 2024-09-17T16:25:19.613 INFO:teuthology.orchestra.run.smithi105.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4975-g1889a165-1jammy [61.5 MB] 2024-09-17T16:25:20.990 INFO:teuthology.orchestra.run.smithi160.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4975-g1889a165-1jammy [8,625 kB] 2024-09-17T16:25:21.199 INFO:teuthology.orchestra.run.smithi160.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4975-g1889a165-1jammy [14.2 kB] 2024-09-17T16:25:21.200 INFO:teuthology.orchestra.run.smithi160.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4975-g1889a165-1jammy [48.5 MB] 2024-09-17T16:25:22.208 INFO:teuthology.orchestra.run.smithi105.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4975-g1889a165-1jammy [8,625 kB] 2024-09-17T16:25:22.432 INFO:teuthology.orchestra.run.smithi105.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4975-g1889a165-1jammy [14.2 kB] 2024-09-17T16:25:22.432 INFO:teuthology.orchestra.run.smithi105.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4975-g1889a165-1jammy [48.5 MB] 2024-09-17T16:25:23.014 INFO:teuthology.orchestra.run.smithi160.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4975-g1889a165-1jammy [130 kB] 2024-09-17T16:25:23.017 INFO:teuthology.orchestra.run.smithi160.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4975-g1889a165-1jammy [39.0 kB] 2024-09-17T16:25:23.018 INFO:teuthology.orchestra.run.smithi160.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4975-g1889a165-1jammy [15.8 MB] 2024-09-17T16:25:23.543 INFO:teuthology.orchestra.run.smithi160.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4975-g1889a165-1jammy [91.3 kB] 2024-09-17T16:25:24.124 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 244 MB in 8s (30.2 MB/s) 2024-09-17T16:25:24.137 INFO:teuthology.orchestra.run.smithi105.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4975-g1889a165-1jammy [130 kB] 2024-09-17T16:25:24.205 INFO:teuthology.orchestra.run.smithi105.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4975-g1889a165-1jammy [39.0 kB] 2024-09-17T16:25:24.207 INFO:teuthology.orchestra.run.smithi105.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4975-g1889a165-1jammy [15.8 MB] 2024-09-17T16:25:24.291 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T16:25:24.391 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T16:25:24.396 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T16:25:24.481 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T16:25:24.561 INFO:teuthology.orchestra.run.smithi105.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-bharath10-testing-2024-09-17-1348/1889a165b5128787ec693450a3d1c5df5d6dc203/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4975-g1889a165-1jammy [91.3 kB] 2024-09-17T16:25:24.809 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T16:25:24.835 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T16:25:24.876 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T16:25:25.135 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 244 MB in 9s (26.9 MB/s) 2024-09-17T16:25:25.187 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T16:25:25.213 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T16:25:25.253 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T16:25:25.290 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T16:25:25.390 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T16:25:25.396 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T16:25:25.472 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T16:25:25.698 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T16:25:25.725 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:25.751 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T16:25:25.777 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T16:25:25.781 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:25.809 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T16:25:26.154 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T16:25:26.160 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T16:25:26.180 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T16:25:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:26.220 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T16:25:26.248 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:26.571 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T16:25:26.597 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:26.637 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:26.649 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T16:25:26.675 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:26.990 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T16:25:27.016 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T16:25:27.065 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T16:25:27.069 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T16:25:27.095 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:27.135 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:27.385 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librados2. 2024-09-17T16:25:27.412 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../007-librados2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:27.451 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librados2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:27.472 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T16:25:27.498 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T16:25:27.538 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:25:27.871 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libnbd0. 2024-09-17T16:25:27.898 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T16:25:27.916 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T16:25:27.929 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T16:25:27.943 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T16:25:27.983 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T16:25:28.223 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librbd1. 2024-09-17T16:25:28.250 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../009-librbd1_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:28.278 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librados2. 2024-09-17T16:25:28.304 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../007-librados2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:28.306 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librbd1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:28.336 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librados2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:28.751 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T16:25:28.777 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:28.814 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libnbd0. 2024-09-17T16:25:28.817 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libcephfs2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:28.840 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T16:25:28.872 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T16:25:29.095 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rados. 2024-09-17T16:25:29.121 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rados (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:29.192 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librbd1. 2024-09-17T16:25:29.218 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../009-librbd1_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:29.258 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librbd1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:29.439 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T16:25:29.454 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:29.489 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-ceph-argparse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:29.678 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T16:25:29.704 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:29.725 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T16:25:29.736 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:29.741 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:29.783 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cephfs (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:30.014 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rados. 2024-09-17T16:25:30.030 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:30.063 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rados (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:30.069 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T16:25:30.096 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:30.135 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:30.400 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T16:25:30.426 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:30.466 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-argparse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:30.471 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T16:25:30.498 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T16:25:30.538 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T16:25:30.718 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T16:25:30.734 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:30.777 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cephfs (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:30.841 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T16:25:30.868 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T16:25:30.907 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T16:25:31.088 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T16:25:31.114 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:31.143 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T16:25:31.155 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:31.159 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:31.201 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rbd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:31.500 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T16:25:31.526 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T16:25:31.537 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T16:25:31.564 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T16:25:31.566 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T16:25:31.604 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T16:25:31.873 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T16:25:31.889 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T16:25:31.894 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T16:25:31.920 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T16:25:31.923 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T16:25:31.960 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T16:25:32.201 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T16:25:32.213 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T16:25:32.216 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T16:25:32.229 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:32.259 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T16:25:32.263 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rbd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:32.596 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package lua5.1. 2024-09-17T16:25:32.599 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T16:25:32.622 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T16:25:32.625 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T16:25:32.662 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T16:25:32.666 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T16:25:32.943 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T16:25:32.959 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T16:25:32.981 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package lua-any. 2024-09-17T16:25:32.997 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-17T16:25:33.002 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T16:25:33.040 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T16:25:33.267 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package zip. 2024-09-17T16:25:33.283 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-17T16:25:33.313 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T16:25:33.317 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T16:25:33.340 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T16:25:33.380 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T16:25:33.629 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package unzip. 2024-09-17T16:25:33.655 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T16:25:33.675 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua5.1. 2024-09-17T16:25:33.695 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T16:25:33.701 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T16:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T16:25:34.082 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package luarocks. 2024-09-17T16:25:34.108 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T16:25:34.149 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T16:25:34.311 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-any. 2024-09-17T16:25:34.326 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-17T16:25:34.369 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T16:25:34.564 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librgw2. 2024-09-17T16:25:34.592 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../026-librgw2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:34.622 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package zip. 2024-09-17T16:25:34.631 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librgw2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:34.637 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-17T16:25:34.680 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T16:25:35.109 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package unzip. 2024-09-17T16:25:35.135 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T16:25:35.184 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T16:25:35.217 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T16:25:35.232 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:35.275 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:35.586 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T16:25:35.596 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package luarocks. 2024-09-17T16:25:35.613 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T16:25:35.622 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T16:25:35.652 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T16:25:35.662 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T16:25:35.963 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T16:25:35.990 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:36.030 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libradosstriper1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:36.045 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librgw2. 2024-09-17T16:25:36.071 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../026-librgw2_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:36.128 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librgw2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:36.483 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-common. 2024-09-17T16:25:36.510 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:36.550 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:36.756 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T16:25:36.772 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:36.814 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:37.151 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T16:25:37.177 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T16:25:37.217 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T16:25:37.520 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T16:25:37.547 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:37.587 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libradosstriper1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:37.921 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-base. 2024-09-17T16:25:37.947 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:37.997 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-base (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:38.015 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-common. 2024-09-17T16:25:38.042 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:38.081 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:38.592 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T16:25:38.619 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T16:25:38.659 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T16:25:38.961 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T16:25:38.988 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T16:25:39.028 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T16:25:39.495 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-base. 2024-09-17T16:25:39.522 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:39.571 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-base (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:39.597 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T16:25:39.624 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T16:25:39.664 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T16:25:39.967 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T16:25:39.993 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T16:25:40.033 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T16:25:40.117 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T16:25:40.143 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T16:25:40.183 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T16:25:40.336 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T16:25:40.362 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T16:25:40.402 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T16:25:40.469 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T16:25:40.496 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T16:25:40.536 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T16:25:40.705 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T16:25:40.731 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T16:25:40.763 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T16:25:40.849 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T16:25:40.876 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T16:25:40.915 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T16:25:41.041 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-portend. 2024-09-17T16:25:41.067 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-17T16:25:41.108 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T16:25:41.168 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T16:25:41.195 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T16:25:41.235 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T16:25:41.410 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T16:25:41.437 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T16:25:41.477 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T16:25:41.521 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T16:25:41.548 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T16:25:41.587 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T16:25:41.779 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T16:25:41.806 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T16:25:41.846 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T16:25:41.891 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T16:25:41.917 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T16:25:41.957 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T16:25:42.183 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T16:25:42.209 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T16:25:42.252 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-portend. 2024-09-17T16:25:42.257 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T16:25:42.279 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-17T16:25:42.318 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T16:25:42.527 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T16:25:42.543 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T16:25:42.571 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T16:25:42.577 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T16:25:42.598 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T16:25:42.637 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T16:25:42.905 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T16:25:42.931 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T16:25:42.941 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T16:25:42.967 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T16:25:42.971 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T16:25:43.007 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T16:25:43.274 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-mako. 2024-09-17T16:25:43.300 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T16:25:43.319 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T16:25:43.340 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T16:25:43.346 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T16:25:43.385 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T16:25:43.677 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T16:25:43.703 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T16:25:43.705 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T16:25:43.732 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T16:25:43.743 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T16:25:43.772 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T16:25:44.013 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T16:25:44.039 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T16:25:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T16:25:44.083 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T16:25:44.110 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T16:25:44.149 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T16:25:44.357 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-webob. 2024-09-17T16:25:44.383 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T16:25:44.423 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T16:25:44.602 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-mako. 2024-09-17T16:25:44.629 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T16:25:44.669 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T16:25:44.927 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T16:25:44.953 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T16:25:44.963 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T16:25:44.979 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T16:25:44.996 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T16:25:45.013 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T16:25:45.308 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T16:25:45.321 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T16:25:45.335 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T16:25:45.348 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T16:25:45.375 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T16:25:45.387 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T16:25:45.690 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-paste. 2024-09-17T16:25:45.694 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webob. 2024-09-17T16:25:45.717 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T16:25:45.721 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T16:25:45.757 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T16:25:45.760 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T16:25:46.102 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T16:25:46.122 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T16:25:46.128 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T16:25:46.149 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T16:25:46.168 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T16:25:46.192 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T16:25:46.471 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T16:25:46.492 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T16:25:46.497 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T16:25:46.519 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T16:25:46.537 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T16:25:46.558 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T16:25:46.849 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T16:25:46.869 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-paste. 2024-09-17T16:25:46.875 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T16:25:46.896 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T16:25:46.907 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T16:25:46.936 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T16:25:47.226 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T16:25:47.253 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T16:25:47.272 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T16:25:47.288 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T16:25:47.292 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T16:25:47.330 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T16:25:47.630 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T16:25:47.633 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T16:25:47.649 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T16:25:47.657 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T16:25:47.683 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T16:25:47.697 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T16:25:47.945 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T16:25:47.971 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T16:25:48.003 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T16:25:48.016 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-idna. 2024-09-17T16:25:48.043 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T16:25:48.083 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T16:25:48.306 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T16:25:48.332 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T16:25:48.372 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T16:25:48.394 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T16:25:48.421 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T16:25:48.461 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T16:25:49.206 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-requests. 2024-09-17T16:25:49.233 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T16:25:49.589 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T16:25:49.637 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T16:25:49.663 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T16:25:49.703 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T16:25:49.875 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T16:25:49.902 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T16:25:49.950 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T16:25:50.031 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-idna. 2024-09-17T16:25:50.057 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T16:25:50.097 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T16:25:50.236 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T16:25:50.252 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:50.294 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:50.417 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T16:25:50.443 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T16:25:50.484 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T16:25:50.691 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T16:25:50.717 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:50.757 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:50.803 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-requests. 2024-09-17T16:25:50.830 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T16:25:50.870 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T16:25:51.077 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T16:25:51.103 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T16:25:51.143 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T16:25:51.181 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T16:25:51.207 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T16:25:51.247 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T16:25:51.412 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T16:25:51.439 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T16:25:51.471 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T16:25:51.559 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T16:25:51.586 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:51.617 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:51.782 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T16:25:51.808 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:51.848 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:52.006 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T16:25:52.032 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:52.072 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:52.243 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T16:25:52.270 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:52.309 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mon (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:52.367 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T16:25:52.393 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T16:25:52.433 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T16:25:52.732 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T16:25:52.758 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T16:25:52.802 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T16:25:52.879 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T16:25:52.906 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:52.946 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-osd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:53.164 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T16:25:53.190 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:53.230 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:53.642 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T16:25:53.669 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:53.708 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mon (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:53.990 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph. 2024-09-17T16:25:54.017 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../067-ceph_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:54.057 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:54.262 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T16:25:54.289 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:54.320 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-osd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:54.378 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T16:25:54.405 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:54.445 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:54.823 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T16:25:54.850 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:54.889 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mds (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:55.268 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package cephadm. 2024-09-17T16:25:55.294 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../070-cephadm_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:55.333 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:55.490 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph. 2024-09-17T16:25:55.516 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../067-ceph_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:55.556 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:55.661 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T16:25:55.688 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T16:25:55.728 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T16:25:55.893 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T16:25:55.919 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:55.959 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:56.038 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T16:25:56.054 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T16:25:56.097 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T16:25:56.329 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T16:25:56.344 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:56.387 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mds (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:56.740 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephadm. 2024-09-17T16:25:56.755 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../070-cephadm_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:25:56.789 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:57.068 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-babel. 2024-09-17T16:25:57.095 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T16:25:57.109 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T16:25:57.134 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T16:25:57.136 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T16:25:57.176 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T16:25:57.429 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T16:25:57.456 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T16:25:57.487 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T16:25:57.496 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T16:25:57.513 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T16:25:57.553 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T16:25:57.815 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T16:25:57.842 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:57.882 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:58.227 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T16:25:58.253 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T16:25:58.293 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T16:25:58.563 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-routes. 2024-09-17T16:25:58.578 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T16:25:58.608 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-babel. 2024-09-17T16:25:58.612 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T16:25:58.635 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T16:25:58.675 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T16:25:58.915 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T16:25:58.937 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:58.965 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:59.011 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T16:25:59.038 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T16:25:59.077 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T16:25:59.422 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T16:25:59.449 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:25:59.489 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:25:59.809 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T16:25:59.836 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T16:25:59.875 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T16:26:00.178 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-routes. 2024-09-17T16:26:00.205 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T16:26:00.244 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T16:26:00.573 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T16:26:00.600 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:00.639 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:01.928 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T16:26:01.945 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T16:26:01.978 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T16:26:02.431 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T16:26:02.459 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T16:26:02.498 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T16:26:02.868 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T16:26:02.895 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T16:26:02.934 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T16:26:03.271 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T16:26:03.298 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T16:26:03.439 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T16:26:03.461 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T16:26:03.478 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T16:26:03.527 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T16:26:03.990 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T16:26:04.017 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T16:26:04.056 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T16:26:04.115 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T16:26:04.143 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:04.181 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:04.435 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T16:26:04.462 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T16:26:04.501 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T16:26:04.770 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T16:26:04.798 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T16:26:04.914 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T16:26:05.092 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T16:26:05.120 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T16:26:05.159 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T16:26:05.470 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T16:26:05.498 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T16:26:05.537 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T16:26:05.682 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T16:26:05.710 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:05.748 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:05.831 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T16:26:05.847 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T16:26:05.881 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T16:26:06.176 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T16:26:06.192 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-17T16:26:06.226 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T16:26:06.513 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T16:26:06.541 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T16:26:06.579 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T16:26:06.710 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T16:26:06.738 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T16:26:06.776 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T16:26:06.899 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T16:26:06.926 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T16:26:06.965 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T16:26:07.096 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T16:26:07.123 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T16:26:07.162 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T16:26:07.294 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T16:26:07.321 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T16:26:07.360 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T16:26:07.474 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T16:26:07.502 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T16:26:07.532 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T16:26:07.646 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T16:26:07.663 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T16:26:07.696 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T16:26:07.810 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T16:26:07.838 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-17T16:26:07.877 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T16:26:08.164 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T16:26:08.191 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T16:26:08.230 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T16:26:08.369 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T16:26:08.397 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:08.435 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:08.550 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T16:26:08.578 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T16:26:08.617 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T16:26:08.729 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T16:26:08.757 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T16:26:08.788 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T16:26:08.928 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T16:26:08.956 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T16:26:08.996 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T16:26:09.124 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T16:26:09.152 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T16:26:09.190 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T16:26:09.315 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T16:26:09.342 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T16:26:09.381 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T16:26:09.437 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package jq. 2024-09-17T16:26:09.451 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T16:26:09.493 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T16:26:09.746 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package socat. 2024-09-17T16:26:09.762 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T16:26:09.803 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T16:26:10.028 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T16:26:10.044 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:10.087 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:10.090 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T16:26:10.106 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T16:26:10.140 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T16:26:10.348 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T16:26:10.376 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T16:26:10.502 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-test. 2024-09-17T16:26:10.530 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:10.531 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T16:26:10.569 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-test (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:11.142 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T16:26:11.171 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T16:26:11.209 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T16:26:11.462 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package jq. 2024-09-17T16:26:11.489 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T16:26:11.528 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T16:26:11.781 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package socat. 2024-09-17T16:26:11.808 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T16:26:11.847 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T16:26:12.142 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T16:26:12.170 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T16:26:12.200 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T16:26:12.580 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-test. 2024-09-17T16:26:12.608 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:12.646 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-test (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:12.823 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T16:26:12.852 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-volume (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:13.168 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T16:26:13.196 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:13.235 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libcephfs-dev (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:13.521 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T16:26:13.549 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T16:26:13.587 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T16:26:13.909 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T16:26:13.937 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T16:26:13.975 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T16:26:14.354 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T16:26:14.382 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T16:26:14.420 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T16:26:14.792 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package pkg-config. 2024-09-17T16:26:14.820 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T16:26:14.858 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T16:26:14.942 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T16:26:14.970 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4975-g1889a165-1jammy_all.deb ... 2024-09-17T16:26:15.017 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-volume (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:15.094 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T16:26:15.110 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T16:26:15.161 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T16:26:15.329 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T16:26:15.357 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:15.395 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs-dev (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:15.556 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T16:26:15.584 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T16:26:15.622 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T16:26:15.715 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T16:26:15.743 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T16:26:15.781 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T16:26:15.925 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T16:26:15.953 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T16:26:15.991 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T16:26:16.112 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T16:26:16.139 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T16:26:16.178 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T16:26:16.320 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T16:26:16.348 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T16:26:16.386 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T16:26:16.549 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T16:26:16.576 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T16:26:16.615 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T16:26:16.697 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T16:26:16.725 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T16:26:16.764 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T16:26:17.054 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package pkg-config. 2024-09-17T16:26:17.082 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T16:26:17.108 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-py. 2024-09-17T16:26:17.120 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T16:26:17.136 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-17T16:26:17.183 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T16:26:17.373 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T16:26:17.401 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T16:26:17.439 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T16:26:17.503 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T16:26:17.531 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T16:26:17.578 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T16:26:17.893 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T16:26:17.921 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T16:26:17.959 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T16:26:18.040 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T16:26:18.068 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T16:26:18.106 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T16:26:18.254 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T16:26:18.282 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T16:26:18.320 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T16:26:18.417 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-toml. 2024-09-17T16:26:18.445 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-17T16:26:18.484 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T16:26:18.649 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T16:26:18.676 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T16:26:18.715 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T16:26:18.795 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T16:26:18.824 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T16:26:18.862 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T16:26:19.035 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T16:26:19.063 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T16:26:19.101 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T16:26:19.215 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T16:26:19.243 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T16:26:19.281 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T16:26:19.437 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-py. 2024-09-17T16:26:19.465 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-17T16:26:19.504 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T16:26:19.576 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T16:26:19.604 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T16:26:19.644 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T16:26:19.815 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T16:26:19.844 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T16:26:19.882 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T16:26:20.345 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T16:26:20.373 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T16:26:20.396 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package radosgw. 2024-09-17T16:26:20.411 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T16:26:20.425 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../117-radosgw_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:20.471 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking radosgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:20.705 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-toml. 2024-09-17T16:26:20.734 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-17T16:26:20.772 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T16:26:21.091 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T16:26:21.120 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T16:26:21.157 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T16:26:21.315 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T16:26:21.332 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:21.373 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking rbd-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:21.513 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T16:26:21.542 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T16:26:21.580 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T16:26:21.661 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T16:26:21.849 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T16:26:21.866 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T16:26:21.907 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T16:26:21.929 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T16:26:22.054 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T16:26:22.317 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T16:26:22.653 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package radosgw. 2024-09-17T16:26:22.671 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T16:26:22.681 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../117-radosgw_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:22.719 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking radosgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:23.124 INFO:teuthology.orchestra.run.smithi160.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T16:26:23.189 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:23.631 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T16:26:23.659 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4975-g1889a165-1jammy_amd64.deb ... 2024-09-17T16:26:23.697 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking rbd-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:23.731 INFO:teuthology.orchestra.run.smithi160.stdout:Adding system user cephadm....done 2024-09-17T16:26:23.895 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T16:26:24.010 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T16:26:24.171 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T16:26:24.268 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T16:26:24.403 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T16:26:24.422 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T16:26:24.549 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T16:26:24.673 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T16:26:24.808 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T16:26:24.989 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T16:26:25.068 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T16:26:25.185 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T16:26:25.399 INFO:teuthology.orchestra.run.smithi105.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T16:26:25.471 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:25.504 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T16:26:25.872 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T16:26:25.971 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user cephadm....done 2024-09-17T16:26:26.127 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T16:26:26.131 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T16:26:26.408 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T16:26:26.496 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T16:26:26.701 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T16:26:26.738 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T16:26:26.848 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T16:26:26.935 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T16:26:27.090 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T16:26:27.195 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T16:26:27.325 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T16:26:27.446 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-ceph-argparse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:27.451 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T16:26:27.722 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T16:26:27.761 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T16:26:27.831 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T16:26:27.940 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T16:26:28.066 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T16:26:28.121 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T16:26:28.192 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T16:26:28.364 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T16:26:28.451 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T16:26:28.624 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T16:26:28.877 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T16:26:28.892 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T16:26:28.908 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T16:26:28.971 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T16:26:29.097 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T16:26:29.143 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T16:26:29.402 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T16:26:29.448 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T16:26:29.574 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T16:26:29.662 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-argparse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:29.842 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T16:26:29.996 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T16:26:30.097 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T16:26:30.110 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T16:26:30.223 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T16:26:30.349 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T16:26:30.436 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T16:26:30.475 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T16:26:30.562 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T16:26:30.726 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T16:26:31.124 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T16:26:31.158 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T16:26:31.221 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T16:26:31.230 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T16:26:31.322 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T16:26:31.482 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T16:26:31.607 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T16:26:31.665 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T16:26:31.725 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T16:26:31.766 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T16:26:32.033 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T16:26:32.143 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T16:26:32.293 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T16:26:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T16:26:32.729 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T16:26:32.769 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T16:26:33.036 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T16:26:33.405 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T16:26:33.405 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T16:26:33.648 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T16:26:33.698 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T16:26:33.774 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T16:26:33.917 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T16:26:34.041 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T16:26:34.218 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T16:26:34.292 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T16:26:34.417 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T16:26:34.735 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T16:26:34.853 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T16:26:35.061 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T16:26:35.205 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T16:26:35.481 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T16:26:35.816 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T16:26:36.058 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T16:26:36.184 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:36.192 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T16:26:36.427 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:36.477 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T16:26:36.570 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T16:26:36.837 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T16:26:36.947 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T16:26:37.045 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T16:26:37.047 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T16:26:37.341 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T16:26:37.616 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T16:26:37.868 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:37.985 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T16:26:38.218 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T16:26:38.288 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T16:26:38.461 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:38.531 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T16:26:38.604 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T16:26:38.791 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T16:26:38.863 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T16:26:38.981 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T16:26:39.042 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T16:26:39.107 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T16:26:39.284 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T16:26:39.391 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T16:26:39.594 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T16:26:39.642 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T16:26:39.720 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T16:26:39.893 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:40.004 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T16:26:40.019 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T16:26:40.139 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T16:26:40.321 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T16:26:40.431 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T16:26:40.547 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T16:26:40.732 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T16:26:40.798 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T16:26:40.983 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T16:26:41.041 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T16:26:41.084 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T16:26:41.202 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T16:26:41.291 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T16:26:41.291 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T16:26:41.545 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T16:26:41.552 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T16:26:41.671 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T16:26:41.677 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T16:26:41.962 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T16:26:42.047 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T16:26:42.088 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T16:26:42.307 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T16:26:42.372 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T16:26:42.433 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T16:26:42.665 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T16:26:42.684 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:42.933 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T16:26:42.943 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T16:26:43.059 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T16:26:43.185 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T16:26:43.252 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T16:26:43.274 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T16:26:43.528 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T16:26:43.553 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T16:26:43.654 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T16:26:43.830 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T16:26:43.956 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T16:26:44.030 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T16:26:44.232 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T16:26:44.281 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T16:26:44.391 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T16:26:44.593 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T16:26:44.642 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T16:26:44.768 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T16:26:44.861 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T16:26:45.085 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T16:26:45.246 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T16:26:45.401 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T16:26:45.526 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T16:26:45.746 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T16:26:45.848 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T16:26:45.858 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T16:26:45.984 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librados2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:46.110 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:46.115 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T16:26:46.236 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T16:26:46.460 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T16:26:46.539 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libcephfs2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:46.656 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libradosstriper1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:46.769 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T16:26:46.782 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T16:26:47.059 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librbd1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:47.171 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T16:26:47.185 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-modules-core (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:47.328 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:47.447 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T16:26:47.537 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T16:26:47.537 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T16:26:47.766 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T16:26:47.893 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librados2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:47.987 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libcephfs-dev (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.043 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.113 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rados (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.169 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T16:26:48.231 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librgw2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.357 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T16:26:48.456 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.582 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libradosstriper1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:48.708 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T16:26:48.984 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librbd1 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.110 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-modules-core (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.236 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.448 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T16:26:49.448 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T16:26:49.569 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rbd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.694 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up rbd-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.820 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.929 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs-dev (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:49.955 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cephfs (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:50.030 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rados (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:50.081 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:50.156 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librgw2 (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:50.274 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T16:26:50.590 INFO:teuthology.orchestra.run.smithi160.stdout:Adding group ceph....done 2024-09-17T16:26:50.950 INFO:teuthology.orchestra.run.smithi160.stdout:Adding system user ceph....done 2024-09-17T16:26:51.110 INFO:teuthology.orchestra.run.smithi160.stdout:Setting system user ceph properties....done 2024-09-17T16:26:51.122 INFO:teuthology.orchestra.run.smithi160.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T16:26:51.259 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T16:26:51.469 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rbd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:51.581 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T16:26:51.587 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up rbd-fuse (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:51.688 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:51.789 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cephfs (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:51.907 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-common (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:52.057 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-test (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:52.183 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up radosgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:52.458 INFO:teuthology.orchestra.run.smithi105.stdout:Adding group ceph....done 2024-09-17T16:26:52.668 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T16:26:52.668 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T16:26:52.836 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user ceph....done 2024-09-17T16:26:53.003 INFO:teuthology.orchestra.run.smithi105.stdout:Setting system user ceph properties....done 2024-09-17T16:26:53.018 INFO:teuthology.orchestra.run.smithi105.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T16:26:53.118 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-base (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:53.152 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T16:26:53.463 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T16:26:53.469 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T16:26:53.903 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mds (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:53.941 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-test (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:54.092 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up radosgw (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:54.134 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T16:26:54.134 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T16:26:54.573 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T16:26:54.574 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T16:26:54.663 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:54.905 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T16:26:54.905 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T16:26:55.120 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-base (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:55.457 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T16:26:55.515 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-osd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:55.801 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T16:26:55.802 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T16:26:55.914 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mds (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:56.136 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T16:26:56.137 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T16:26:56.275 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:56.418 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:56.552 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mon (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:56.590 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:56.764 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T16:26:56.764 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T16:26:56.804 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T16:26:56.804 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T16:26:57.270 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:57.287 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-osd (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:57.421 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:57.547 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-dashboard (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:57.690 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-volume (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:57.955 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T16:26:57.955 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T16:26:58.043 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T16:26:58.169 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T16:26:58.313 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T16:26:58.437 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:58.555 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:58.681 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mon (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:58.891 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T16:26:58.891 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T16:26:59.341 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-cephadm (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:59.447 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T16:26:59.483 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:59.609 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-dashboard (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:26:59.760 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-volume (19.3.0-4975-g1889a165-1jammy) ... 2024-09-17T16:27:00.084 INFO:teuthology.orchestra.run.smithi160.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T16:27:00.089 DEBUG:teuthology.parallel:result is None 2024-09-17T16:27:00.180 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T16:27:00.272 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T16:27:00.417 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T16:27:01.692 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T16:27:02.358 INFO:teuthology.orchestra.run.smithi105.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T16:27:02.362 DEBUG:teuthology.parallel:result is None 2024-09-17T16:27:02.363 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:27:02.516 DEBUG:teuthology.orchestra.run.smithi105:> dpkg-query -W -f '${Version}' ceph 2024-09-17T16:27:02.534 INFO:teuthology.orchestra.run.smithi105.stdout:19.3.0-4975-g1889a165-1jammy 2024-09-17T16:27:02.535 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975-g1889a165-1jammy 2024-09-17T16:27:02.535 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975-g1889a165-1jammy is installed. 2024-09-17T16:27:02.537 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:27:02.680 DEBUG:teuthology.orchestra.run.smithi160:> dpkg-query -W -f '${Version}' ceph 2024-09-17T16:27:02.700 INFO:teuthology.orchestra.run.smithi160.stdout:19.3.0-4975-g1889a165-1jammy 2024-09-17T16:27:02.700 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4975-g1889a165-1jammy 2024-09-17T16:27:02.700 INFO:teuthology.task.install:The correct ceph version 19.3.0-4975-g1889a165-1jammy is installed. 2024-09-17T16:27:02.702 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T16:27:02.703 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:02.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T16:27:02.717 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:02.718 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T16:27:02.755 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T16:27:02.755 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:02.755 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T16:27:02.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T16:27:02.828 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:02.829 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T16:27:02.842 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T16:27:02.898 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T16:27:02.898 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:02.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T16:27:02.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T16:27:02.965 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:02.966 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T16:27:02.979 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T16:27:03.034 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T16:27:03.034 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:03.034 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T16:27:03.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T16:27:03.100 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:03.101 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T16:27:03.113 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T16:27:03.170 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T16:27:03.256 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T16:27:03.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-09-17T16:27:03.260 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /var/log/ceph 2024-09-17T16:27:03.274 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T16:27:03.274 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T16:27:03.315 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T16:27:03.330 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T16:27:03.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T16:27:03.371 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T16:27:03.395 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T16:27:03.395 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_FULL\\)', '\\(SLOW_OPS\\)', '\\(MON_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T16:27:03.395 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060', 'branch': 'wip-bharath10-testing-2024-09-17-1348', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-hybrid} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7909060', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath10-testing-2024-09-17-1348'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_FULL\\)', '\\(SLOW_OPS\\)', '\\(MON_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'slow request'], 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath10-testing-2024-09-17-1348', 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 3421, 'sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath10-testing-2024-09-17-1348', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_1889a165b5128787ec693450a3d1c5df5d6dc203/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '1889a165b5128787ec693450a3d1c5df5d6dc203', 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPY1BFaEPASCia2fh3TnVU5B8nPB8spKzkfQsiOerydGXcBsEjpi2e6o6xdty0K1u0kv51u6uAvJ/ixdbhiFVI8=', 'smithi160.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDLD/HWlkC4pe/aVgYesztpUNYj13mT4gSGojjieF8/JL0teng+0g0v5ptgsPH4m4d3ZrzikpaSCUevJWW2pHk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'store_thrash': True, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['rados/test.sh unit_test_scan']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_15:01:33', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T16:27:03.396 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T16:27:03.427 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T16:27:03.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T16:27:03.475 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T16:27:03.490 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:03.490 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T16:27:03.534 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T16:27:03.534 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-09-17T16:27:03.582 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T16:27:03.582 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:03.582 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-17T16:27:03.582 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:03.583 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-17 16:21:36.258709744 +0000 2024-09-17T16:27:03.583 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-17 16:21:36.014711920 +0000 2024-09-17T16:27:03.583 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-17 16:21:36.014711920 +0000 2024-09-17T16:27:03.583 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-09-17T16:27:03.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T16:27:03.636 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-17T16:27:03.637 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-17T16:27:03.637 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000432757 s, 1.2 MB/s 2024-09-17T16:27:03.638 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T16:27:03.688 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-17 16:21:36.738705462 +0000 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-17 16:21:36.478707782 +0000 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-17 16:21:36.478707782 +0000 2024-09-17T16:27:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-09-17T16:27:03.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T16:27:03.788 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-17T16:27:03.789 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-17T16:27:03.789 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000337926 s, 1.5 MB/s 2024-09-17T16:27:03.791 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T16:27:03.839 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-17 16:21:37.234701038 +0000 2024-09-17T16:27:03.890 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-17 16:21:36.982703286 +0000 2024-09-17T16:27:03.891 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-17 16:21:36.982703286 +0000 2024-09-17T16:27:03.891 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-09-17T16:27:03.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T16:27:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-17T16:27:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-17T16:27:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000358064 s, 1.4 MB/s 2024-09-17T16:27:03.947 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T16:27:03.995 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-17 16:21:37.734696579 +0000 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-17 16:21:37.482698826 +0000 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-17 16:21:37.482698826 +0000 2024-09-17T16:27:04.042 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-09-17T16:27:04.043 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T16:27:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-17T16:27:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-17T16:27:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00038916 s, 1.3 MB/s 2024-09-17T16:27:04.097 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T16:27:04.147 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:04.147 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:04.147 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T16:27:04.154 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T16:27:04.154 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-09-17T16:27:04.203 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T16:27:04.203 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:04.203 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-17T16:27:04.203 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:04.203 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-17 16:21:36.007256065 +0000 2024-09-17T16:27:04.204 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-17 16:21:36.003256101 +0000 2024-09-17T16:27:04.204 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-17 16:21:36.003256101 +0000 2024-09-17T16:27:04.204 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-09-17T16:27:04.204 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T16:27:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-17T16:27:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-17T16:27:04.258 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000432129 s, 1.2 MB/s 2024-09-17T16:27:04.259 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T16:27:04.308 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-09-17T16:27:04.354 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-17 16:21:36.755249383 +0000 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-17 16:21:36.479251849 +0000 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-17 16:21:36.479251849 +0000 2024-09-17T16:27:04.355 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-09-17T16:27:04.355 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T16:27:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-17T16:27:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-17T16:27:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.00042272 s, 1.2 MB/s 2024-09-17T16:27:04.410 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T16:27:04.460 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-09-17T16:27:04.506 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-17 16:21:37.279244703 +0000 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-17 16:21:37.019247026 +0000 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-17 16:21:37.019247026 +0000 2024-09-17T16:27:04.507 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-09-17T16:27:04.507 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T16:27:04.561 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-17T16:27:04.561 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-17T16:27:04.561 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000376824 s, 1.4 MB/s 2024-09-17T16:27:04.562 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T16:27:04.612 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-09-17T16:27:04.658 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-09-17 16:21:37.771240309 +0000 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-09-17 16:21:37.515242596 +0000 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-09-17 16:21:37.515242596 +0000 2024-09-17T16:27:04.659 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-09-17T16:27:04.659 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T16:27:04.713 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-09-17T16:27:04.713 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-09-17T16:27:04.713 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000380913 s, 1.3 MB/s 2024-09-17T16:27:04.714 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T16:27:04.764 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:04.765 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T16:27:04.765 INFO:tasks.ceph:Generating config... 2024-09-17T16:27:04.767 INFO:tasks.ceph:[client] debug monc = 20 2024-09-17T16:27:04.767 INFO:tasks.ceph:[client] debug ms = 1 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] debug ms = 1 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] debug objecter = 20 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] debug rados = 20 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T16:27:04.767 INFO:tasks.ceph:[global] ms type = async 2024-09-17T16:27:04.767 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-17T16:27:04.768 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-17T16:27:04.768 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T16:27:04.769 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd class default list = * 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd class load list = * 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T16:27:04.770 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-17T16:27:04.771 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-17T16:27:04.771 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T16:27:04.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:04.806 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T16:27:04.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T16:27:04.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T16:27:04.945 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]'), ('mon.c', '[v2:172.21.15.105:3302,v1:172.21.15.105:6791]'), ('mon.d', '[v2:172.21.15.105:3303,v1:172.21.15.105:6792]'), ('mon.e', '[v2:172.21.15.105:3304,v1:172.21.15.105:6793]'), ('mon.f', '172.21.15.160'), ('mon.g', '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]'), ('mon.h', '[v2:172.21.15.160:3302,v1:172.21.15.160:6791]'), ('mon.i', '[v2:172.21.15.160:3303,v1:172.21.15.160:6792]')] 2024-09-17T16:27:04.945 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790],[v2:172.21.15.105:3302,v1:172.21.15.105:6791],[v2:172.21.15.105:3303,v1:172.21.15.105:6792],[v2:172.21.15.105:3304,v1:172.21.15.105:6793],172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790],[v2:172.21.15.160:3302,v1:172.21.15.160:6791],[v2:172.21.15.160:3303,v1:172.21.15.160:6792]', 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-09-17T16:27:04.946 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:04.946 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T16:27:04.990 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --addv b '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]' --addv c '[v2:172.21.15.105:3302,v1:172.21.15.105:6791]' --addv d '[v2:172.21.15.105:3303,v1:172.21.15.105:6792]' --addv e '[v2:172.21.15.105:3304,v1:172.21.15.105:6793]' --add f 172.21.15.160 --addv g '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]' --addv h '[v2:172.21.15.160:3302,v1:172.21.15.160:6791]' --addv i '[v2:172.21.15.160:3303,v1:172.21.15.160:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:05.096 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:05.096 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:05.096 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-09-17T16:27:05.096 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 17 (quincy) 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 3 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:3: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:4: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:5: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:6: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:7: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:05.097 INFO:teuthology.orchestra.run.smithi105.stdout:8: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:05.098 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-09-17T16:27:05.099 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T16:27:05.104 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b... 2024-09-17T16:27:05.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T16:27:05.147 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-09-17T16:27:05.172 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-09-17T16:27:05.173 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-09-17T16:27:05.174 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790],[v2:172.21.15.105:3302,v1:172.21.15.105:6791],[v2:172.21.15.105:3303,v1:172.21.15.105:6792],[v2:172.21.15.105:3304,v1:172.21.15.105:6793],172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790],[v2:172.21.15.160:3302,v1:172.21.15.160:6791],[v2:172.21.15.160:3303,v1:172.21.15.160:6792]" 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: debug objecter = 20 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: debug rados = 20 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: mon client directed command retry = 5 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: mon election default strategy = 3 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject socket failures = 5000 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: ms type = async 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.175 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.176 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: bluefs allocator = hybrid 2024-09-17T16:27:05.177 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = hybrid 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore write v2 = True 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore zero block detection = True 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: osd class default list = * 2024-09-17T16:27:05.178 INFO:teuthology.orchestra.run.smithi105.stdout: osd class load list = * 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify cached snaps = True 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify missing on start = True 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock override recovery settings = True 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd op queue = debug_random 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: osd op queue cut off = debug_random 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-09-17T16:27:05.179 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-09-17T16:27:05.180 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon min osdmap epochs = 25 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune interval = 2 2024-09-17T16:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune min = 15 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune txsize = 2 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: mon scrub inject crc mismatch = 0.01 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: mon scrub inject missing keys = 0.05 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: mon scrub interval = 300 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on pool no app = False 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: paxos service trim min = 5 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: debug monc = 20 2024-09-17T16:27:05.182 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.d] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.e] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.f] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.g] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.h] 2024-09-17T16:27:05.183 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.i] 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2024-09-17T16:27:05.184 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2024-09-17T16:27:05.185 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon_allow_pool_size_one = true 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.186 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: mon health detail to clog = false 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = "172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790],[v2:172.21.15.105:3302,v1:172.21.15.105:6791],[v2:172.21.15.105:3303,v1:172.21.15.105:6792],[v2:172.21.15.105:3304,v1:172.21.15.105:6793],172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790],[v2:172.21.15.160:3302,v1:172.21.15.160:6791],[v2:172.21.15.160:3303,v1:172.21.15.160:6792]" 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: debug objecter = 20 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: debug rados = 20 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: mon client directed command retry = 5 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: mon election default strategy = 3 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: ms inject socket failures = 5000 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: ms type = async 2024-09-17T16:27:05.187 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd mclock profile = high_recovery_ops 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2024-09-17T16:27:05.188 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug trim objects = true 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.189 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluefs allocator = hybrid 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore allocator = hybrid 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore block size = 96636764160 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore fsck on mount = True 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore write v2 = True 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore zero block detection = True 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluefs = 20 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluestore = 20 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 20 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: debug rocksdb = 10 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T16:27:05.190 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd full ratio = 0.9 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd nearfull ratio = 0.8 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd class default list = * 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd class load list = * 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify cached snaps = True 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify missing on start = True 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd failsafe full ratio = 0.95 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd mclock override recovery settings = True 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd objectstore = bluestore 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd op queue = debug_random 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: osd op queue cut off = debug_random 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2024-09-17T16:27:05.191 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mgr/telemetry/nag = false 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2024-09-17T16:27:05.192 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: # 1m isn't quite enough 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon_down_mkfs_grace = 2m 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.193 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_filestore_osds = false 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon min osdmap epochs = 25 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon osdmap full prune interval = 2 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon osdmap full prune min = 15 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon osdmap full prune txsize = 2 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon scrub inject crc mismatch = 0.01 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon scrub inject missing keys = 0.05 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon scrub interval = 300 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on pool no app = False 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: paxos service trim min = 5 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2024-09-17T16:27:05.194 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout: debug monc = 20 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.b] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.c] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.d] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.e] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.f] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.g] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.h] 2024-09-17T16:27:05.195 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.i] 2024-09-17T16:27:05.197 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T16:27:05.197 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T16:27:05.286 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T16:27:05.287 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:05.287 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T16:27:05.291 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:05.292 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T16:27:05.337 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:27:05.338 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:05.338 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T16:27:05.338 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T16:27:05.401 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:05.401 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:05.446 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:27:05.446 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:05.446 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T16:27:05.446 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T16:27:05.469 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:05.469 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:05.515 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T16:27:05.515 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T16:27:05.515 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T16:27:05.601 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T16:27:05.614 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T16:27:05.614 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T16:27:05.614 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T16:27:05.692 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T16:27:05.703 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T16:27:05.703 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T16:27:05.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T16:27:05.717 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:05.717 INFO:tasks.ceph:role: osd.0 2024-09-17T16:27:05.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:27:05.717 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T16:27:05.781 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:05.781 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:05.782 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:05.782 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:05.782 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:06.238 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-17T16:27:06.240 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-17T16:27:06.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T16:27:06.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T16:27:06.341 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:06.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:06.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T16:27:06.397 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:06.397 INFO:tasks.ceph:role: osd.1 2024-09-17T16:27:06.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:27:06.397 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:06.463 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:06.936 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-17T16:27:06.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-17T16:27:06.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T16:27:07.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T16:27:07.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:07.109 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:07.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T16:27:07.165 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:07.165 INFO:tasks.ceph:role: osd.2 2024-09-17T16:27:07.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:27:07.166 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:07.230 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:07.706 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-17T16:27:07.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-17T16:27:07.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T16:27:07.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T16:27:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:07.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:07.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:07.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:07.965+0000 7fe13681b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T16:27:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:07.965+0000 7fe13681b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T16:27:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:07.965+0000 7fe13681b7c0 -1 bdev(0x557e739b5800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:07.965+0000 7fe13681b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T16:27:11.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T16:27:11.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:11.221+0000 7f7a5e31b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T16:27:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:11.221+0000 7f7a5e31b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T16:27:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:11.221+0000 7f7a5e31b7c0 -1 bdev(0x56339aadf800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:11.225+0000 7f7a5e31b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T16:27:14.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T16:27:14.442 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:14.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:14.517+0000 7fc5a563e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T16:27:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:14.517+0000 7fc5a563e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T16:27:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:14.517+0000 7fc5a563e7c0 -1 bdev(0x55565fa3b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:14.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:14.517+0000 7fc5a563e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T16:27:17.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T16:27:17.711 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T16:27:17.726 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:17.726 INFO:tasks.ceph:role: osd.3 2024-09-17T16:27:17.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:27:17.726 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:17.791 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:18.276 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-09-17T16:27:18.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-09-17T16:27:18.278 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-17T16:27:18.422 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T16:27:18.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:18.434 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:18.435 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T16:27:18.491 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:18.491 INFO:tasks.ceph:role: osd.4 2024-09-17T16:27:18.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:27:18.491 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T16:27:18.554 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:18.554 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:18.554 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:18.554 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:18.555 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:18.999 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-09-17T16:27:19.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-09-17T16:27:19.002 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-17T16:27:19.083 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T16:27:19.136 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:19.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:19.137 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T16:27:19.190 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T16:27:19.191 INFO:tasks.ceph:role: osd.5 2024-09-17T16:27:19.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:27:19.191 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-09-17T16:27:19.253 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T16:27:19.254 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T16:27:19.254 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T16:27:19.254 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T16:27:19.706 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-09-17T16:27:19.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-09-17T16:27:19.708 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-17T16:27:19.855 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T16:27:19.866 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T16:27:19.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:27:19.867 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:19.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:19.970+0000 7f04c6a077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T16:27:19.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:19.970+0000 7f04c6a077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T16:27:19.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:19.970+0000 7f04c6a077c0 -1 bdev(0x55bd67e13800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:19.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:19.970+0000 7f04c6a077c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T16:27:23.110 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T16:27:23.124 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:23.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:23.198+0000 7f4d92c7a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T16:27:23.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:23.198+0000 7f4d92c7a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T16:27:23.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:23.198+0000 7f4d92c7a7c0 -1 bdev(0x563ea656b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:23.205 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:23.198+0000 7f4d92c7a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T16:27:26.355 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T16:27:26.411 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:26.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:26.490+0000 7f900a63a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T16:27:26.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:26.490+0000 7f900a63a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T16:27:26.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:26.490+0000 7f900a63a7c0 -1 bdev(0x55a9d01ef800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T16:27:26.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:26.490+0000 7f900a63a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T16:27:29.692 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T16:27:29.750 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T16:27:29.750 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:29.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T16:27:29.765 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:29.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T16:27:29.822 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:29.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T16:27:29.877 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:29.877 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T16:27:29.891 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:29.891 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T16:27:29.946 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:29.946 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T16:27:30.002 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:30.002 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T16:27:30.058 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:30.059 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T16:27:30.107 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T16:27:30.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T16:27:30.110 DEBUG:teuthology.orchestra.run.smithi160:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T16:27:30.152 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbrelmkgbEORAAkxsHz7cH1IRIxlhXJEuOhQ== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDfrelmIIGHDRAAkPzqigNObu1N3WreIy/ibA== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDirelmMOslHxAAtngVzFdrTRz9oYTqrVWTtw== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDZrelmJXDIIxAABEqjvVoNU9IopVkHUnPP9Q== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDnrelmBysUOhAAYG0+xAhwfefopK4gD+QJDA== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDrrelmf68tDBAANXmflGMMuA6zsl6Lu5n9gg== 2024-09-17T16:27:30.153 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-09-17T16:27:30.154 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDurelmur+THRAA+A4RkIF//8FzNWnl+pKCkg== 2024-09-17T16:27:30.154 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-09-17T16:27:30.154 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDZrelm3nZCKRAAVcmpn0WdnVir/S8PlTC1xQ== 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDbrelmkgbEORAAkxsHz7cH1IRIxlhXJEuOhQ== 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDfrelmIIGHDRAAkPzqigNObu1N3WreIy/ibA== 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDirelmMOslHxAAtngVzFdrTRz9oYTqrVWTtw== 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.x] 2024-09-17T16:27:30.161 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDZrelmJXDIIxAABEqjvVoNU9IopVkHUnPP9Q== 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.3] 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDnrelmBysUOhAAYG0+xAhwfefopK4gD+QJDA== 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.4] 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDrrelmf68tDBAANXmflGMMuA6zsl6Lu5n9gg== 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.5] 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDurelmur+THRAA+A4RkIF//8FzNWnl+pKCkg== 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2024-09-17T16:27:30.162 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDZrelm3nZCKRAAVcmpn0WdnVir/S8PlTC1xQ== 2024-09-17T16:27:30.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.199 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.250 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.291 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.375 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T16:27:30.455 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T16:27:30.506 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.539 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.623 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.671 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T16:27:30.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T16:27:30.760 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T16:27:30.850 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T16:27:30.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T16:27:30.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:30.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T16:27:30.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T16:27:31.015 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T16:27:31.117 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T16:27:31.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T16:27:31.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-09-17T16:27:31.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-09-17T16:27:31.431 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-09-17T16:27:31.483 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.576 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-09-17T16:27:31.585 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-09-17T16:27:31.599 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.694 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-09-17T16:27:31.705 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-09-17T16:27:31.756 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.848 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-09-17T16:27:31.860 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-09-17T16:27:31.912 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:31.998 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-09-17T16:27:32.054 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-09-17T16:27:32.110 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T16:27:32.206 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-09-17T16:27:32.216 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:32.219 DEBUG:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T16:27:32.263 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T16:27:32.263 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T16:27:32.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T16:27:32.267 INFO:tasks.ceph.mon.a:Started 2024-09-17T16:27:32.268 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T16:27:32.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T16:27:32.271 INFO:tasks.ceph.mon.b:Started 2024-09-17T16:27:32.271 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T16:27:32.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T16:27:32.274 INFO:tasks.ceph.mon.c:Started 2024-09-17T16:27:32.275 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-17T16:27:32.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-17T16:27:32.278 INFO:tasks.ceph.mon.d:Started 2024-09-17T16:27:32.278 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-17T16:27:32.278 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-17T16:27:32.280 INFO:tasks.ceph.mon.e:Started 2024-09-17T16:27:32.280 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-17T16:27:32.281 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-17T16:27:32.307 INFO:tasks.ceph.mon.f:Started 2024-09-17T16:27:32.307 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-17T16:27:32.307 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-17T16:27:32.309 INFO:tasks.ceph.mon.g:Started 2024-09-17T16:27:32.309 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-17T16:27:32.309 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-17T16:27:32.311 INFO:tasks.ceph.mon.h:Started 2024-09-17T16:27:32.311 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-17T16:27:32.311 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-17T16:27:32.313 INFO:tasks.ceph.mon.i:Started 2024-09-17T16:27:32.313 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T16:27:32.314 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T16:27:32.314 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T16:27:32.316 INFO:tasks.ceph.mgr.x:Started 2024-09-17T16:27:32.316 DEBUG:tasks.ceph:set 0 configs 2024-09-17T16:27:32.316 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T16:27:32.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T16:27:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: get_monmap_and_config 2024-09-17T16:27:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: build_initial_monmap 2024-09-17T16:27:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: monmap: 2024-09-17T16:27:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:32.436564+0000 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:32.436564+0000 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 Processor -- start 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 -- start start 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: init 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient: _add_conns ranks=[8,9,3,5,2,7,1,6,10,4,0] 2024-09-17T16:27:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-e con 0x7f12bc0a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-a con 0x7f12bc0a5070 addr v2:172.21.15.105:3300/0 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-g con 0x7f12bc137a50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f12bc06cf10 con 0x7f12bc0a5070 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc06c590 con 0x7f12bc137a50 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f12bc0a62d0 con 0x7f12bc0a4750 2024-09-17T16:27:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c3dc7640 10 monclient(hunting): authenticate will time out at 935.701936s 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:55716/0 (socket says 172.21.15.105:55716) 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12bc0a5070 auth_method 0 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 learned_addr learned my addr 172.21.15.105:0/3583392716 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12bc0a4750 auth_method 0 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12bc137a50 auth_method 0 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12bc0a4750 auth_method 0 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12bc0a5070 auth_method 0 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12bc137a50 auth_method 0 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:32.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12bc0a5070 auth_method 0 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12bc0a4750 auth_method 0 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12bc137a50 auth_method 0 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12bc0a4750 auth_method 0 2024-09-17T16:27:33.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12bc0a5070 auth_method 0 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:33.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12bc137a50 auth_method 0 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:33.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:33.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:33.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:35.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): tick 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 msgr2=0x7f12bc0a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f12bc0a4750 0x7f12bc0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 msgr2=0x7f12bc137ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12bc137a50 0x7f12bc137ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 msgr2=0x7f12bc137510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.105:3300/0 conn(0x7f12bc0a5070 0x7f12bc137510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:35.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient: _add_conns ranks=[6,5,10,2,7,1,0,4] 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-d con 0x7f12a8003840 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-h con 0x7f12a8006270 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-f con 0x7f12a8008dd0 addr v2:172.21.15.160:3300/0 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f12bc06c590 con 0x7f12a8008dd0 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f12bc0a62d0 con 0x7f12a8006270 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:35.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f12a800b9a0 con 0x7f12a8003840 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12b37fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003840 auth_method 0 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006270 auth_method 0 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8008dd0 auth_method 0 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:35.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:35.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.432+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003840 auth_method 0 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8008dd0 auth_method 0 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006270 auth_method 0 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:35.636+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8008dd0 auth_method 0 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006270 auth_method 0 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003840 auth_method 0 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.040+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8008dd0 auth_method 0 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006270 auth_method 0 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003840 auth_method 0 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:36.844+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): tick 2024-09-17T16:27:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:27:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:27:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:27:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 msgr2=0x7f12a8005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f12a8003840 0x7f12a8005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 msgr2=0x7f12a8008700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f12a8006270 0x7f12a8008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> v2:172.21.15.160:3300/0 conn(0x7f12a8008dd0 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 reap_dead start 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient: _add_conns ranks=[4,7,2,0,1] 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 reap_dead start 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-c con 0x7f12a8003c60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-i con 0x7f12a8006c30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-b con 0x7f12a8007010 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc06c590 con 0x7f12a8007010 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f12bc0a62d0 con 0x7f12a8003c60 2024-09-17T16:27:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f12a800b9a0 con 0x7f12a8006c30 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12b37fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8007010 auth_method 0 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.436+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:27:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8007010 auth_method 0 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:38.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:38.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:38.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:38.640+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:27:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8007010 auth_method 0 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.044+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:27:39.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:39.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8007010 auth_method 0 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:39.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:27:39.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:39.848+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:27:41.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): tick 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a800b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a800b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a80066f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 msgr2=0x7f12a8007490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:27:41.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a8007010 0x7f12a8007490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient: _add_conns ranks=[1,0] 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f12a8007d90 addr v1:172.21.15.160:6789/0 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f12a8010fe0 addr v1:172.21.15.105:6789/0 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12bc0a09d0 con 0x7f12a8010fe0 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 1 -- 172.21.15.105:0/3583392716 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a8015b00 con 0x7f12a8007d90 2024-09-17T16:27:41.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:41.436+0000 7f12b37fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:42.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3105557536 0 0) 0x7f12a8015b00 con 0x7f12a8007d90 2024-09-17T16:27:42.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): _negotiate 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): my global_id is 4105 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12a0003530 con 0x7f12a8007d90 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3556310813 0 0) 0x7f12a0003530 con 0x7f12a8007d90 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 >> v1:172.21.15.105:6789/0 conn(0x7f12a8010fe0 legacy=0x7f12a80133d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _finish_hunting 0 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 monclient: found mon.noname-f-legacy 2024-09-17T16:27:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.160:6789/0 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a8016ce0 con 0x7f12a8007d90 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _finish_auth 0 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _check_auth_tickets 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f12ac00f040 con 0x7f12a8007d90 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: dump: 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.734 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] but i'm connected to v1:172.21.15.160:6789/0, reconnecting 2024-09-17T16:27:42.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:42.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 >> v1:172.21.15.160:6789/0 conn(0x7f12a8007d90 legacy=0x7f12a80081a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient: _add_conns ranks=[3,4,7,2,6,1,0,8,5] 2024-09-17T16:27:42.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0004260 0x7f12a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): picked mon.g con 0x7f12a0004260 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:42.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a0004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): picked mon.c con 0x7f12a8003c60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): picked mon.i con 0x7f12a8006c30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f12a000c010 con 0x7f12a0004260 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f12a00105a0 con 0x7f12a8003c60 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f12a0010720 con 0x7f12a8006c30 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c0b61640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:27:42.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 1 -- 172.21.15.105:0/3583392716 reap_dead start 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a0004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c2364640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1362640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:27:42.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1b63640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0004260 0x7f12a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a0004260 auth_method 0 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.724+0000 7f12c1b63640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c2364640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c2364640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c2364640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 10 monclient(hunting): handle_auth_done global_id 4105 payload 293 2024-09-17T16:27:42.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 msgr2=0x7f12a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0004260 msgr2=0x7f12a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0004260 0x7f12a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 10 monclient: _finish_hunting 0 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 monclient: found mon.c 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 1 -- 172.21.15.105:0/3583392716 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a0003530 con 0x7f12a8003c60 2024-09-17T16:27:42.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 10 monclient: _finish_auth 0 2024-09-17T16:27:42.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 10 monclient: _check_auth_tickets 2024-09-17T16:27:42.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c1362640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:42.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c3dc7640 5 monclient: authenticate success, global_id 4105 2024-09-17T16:27:42.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.728+0000 7f12c3dc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c1362640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a0004f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b801cb30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=823333f2d6646cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f12b8003d90 con 0x7f12a8003c60 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: dump: 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.746 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:42.747 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b801b0d0 con 0x7f12a8003c60 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c3dc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 1 -- 172.21.15.105:0/3583392716 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f12b801b280 con 0x7f12a8003c60 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c3dc7640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c3dc7640 4 set_mon_vals no callback set 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.732+0000 7f12c0b61640 10 monclient: dump: 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:42.748 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:42.749 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 10 monclient: shutdown 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 -- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 msgr2=0x7f12a0004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a0004f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b801cb30 comp rx=0 tx=0).stop 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 -- 172.21.15.105:0/3583392716 shutdown_connections 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f12a8006c30 0x7f12a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f12a8003c60 0x7f12a0004f90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/3583392716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0004260 0x7f12a0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 -- 172.21.15.105:0/3583392716 >> 172.21.15.105:0/3583392716 conn(0x7f12bc0a94d0 msgr2=0x7f12bc0a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 -- 172.21.15.105:0/3583392716 shutdown_connections 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 1 -- 172.21.15.105:0/3583392716 wait complete. 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 10 monclient: build_initial_monmap 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.736+0000 7f12c3dc7640 10 monclient: monmap: 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:42.751 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:42.752 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 librados: starting msgr at 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 librados: starting objecter 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 Processor -- start 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- start start 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 librados: setting wanted keys 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 librados: calling monclient init 2024-09-17T16:27:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: init 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _add_conns ranks=[2,1,3,5,4,6,0,8,7] 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-b con 0x7f12a0004260 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a8003c60 0x7f12bc121fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-f con 0x7f12a8003c60 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a8006c30 0x7f12bc122500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): picked mon.noname-g con 0x7f12a8006c30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc06b5b0 con 0x7f12a8003c60 2024-09-17T16:27:42.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0a05e0 con 0x7f12a0004260 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc06c810 con 0x7f12a8006c30 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient(hunting): authenticate will time out at 946.009186s 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43958/0 (socket says 172.21.15.105:43958) 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 -- 172.21.15.105:0/1459585848 learned_addr learned my addr 172.21.15.105:0/1459585848 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): get_auth_request con 0x7f12a0004260 auth_method 0 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c2364640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a8006c30 0x7f12bc122500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1362640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a8003c60 0x7f12bc121fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c2364640 10 monclient(hunting): get_auth_request con 0x7f12a8006c30 auth_method 0 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c2364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c2364640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c2364640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1362640 10 monclient(hunting): get_auth_request con 0x7f12a8003c60 auth_method 0 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1362640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1362640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1362640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient(hunting): handle_auth_done global_id 4106 payload 824 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 -- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a8006c30 msgr2=0x7f12bc122500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a8006c30 0x7f12bc122500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 -- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a8003c60 msgr2=0x7f12bc121fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a8003c60 0x7f12bc121fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient: _finish_hunting 0 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 monclient: found mon.noname-b 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 -- 172.21.15.105:0/1459585848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc124970 con 0x7f12a0004260 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c1b63640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006e10 tx=0x7f12ac018f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 client.?.objecter ms_handle_connect 0x7f12a0004260 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f12ac02a070 con 0x7f12a0004260 2024-09-17T16:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: dump: 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: _finish_auth 0 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 5 monclient: authenticate success, global_id 4106 2024-09-17T16:27:42.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac000e50 con 0x7f12a0004260 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _renew_subs 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc123480 con 0x7f12a0004260 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f12ac0044c0 con 0x7f12a0004260 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 client.4106.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b3fff640 4 set_mon_vals no callback set 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 monclient: dump: 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.763 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _renew_subs 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc124270 con 0x7f12a0004260 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 librados: init done 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12ac016cf0 con 0x7f12a0004260 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f12ac00f070 con 0x7f12a0004260 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 10 client.4106.objecter ms_dispatch 0x7f12bc000ba0 osd_map(1..1 src has 1..1) 2024-09-17T16:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 3 client.4106.objecter handle_osd_map got epochs [1,1] > 0 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 3 client.4106.objecter handle_osd_map decoding full epoch 1 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12b27fc640 20 client.4106.objecter dump_active .. 0 homeless 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.740+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc137e60 con 0x7f12a0004260 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.744+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f12ac033020 con 0x7f12a0004260 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.744+0000 7f12b27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.744+0000 7f12b27fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:42.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.808+0000 7f12c3dc7640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-17T16:27:42.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.808+0000 7f12c3dc7640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-17T16:27:42.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.808+0000 7f12c3dc7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:42.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.808+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f12bc073820 con 0x7f12a0004260 2024-09-17T16:27:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12b27fc640 1 -- 172.21.15.105:0/1459585848 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f12ac01f040 con 0x7f12a0004260 2024-09-17T16:27:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12b27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-17T16:27:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12b27fc640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-17T16:27:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 10 librados: watch_flush enter 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 10 librados: watch_flush exit 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 20 client.4106.objecter shutdown clearing up homeless session... 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 10 client.4106.objecter successfully canceled tick 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 10 monclient: shutdown 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 msgr2=0x7f12bc121a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006e10 tx=0x7f12ac018f10 comp rx=0 tx=0).stop 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 shutdown_connections 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a8006c30 0x7f12bc122500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a8003c60 0x7f12bc121fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 --2- 172.21.15.105:0/1459585848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f12a0004260 0x7f12bc121a80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 >> 172.21.15.105:0/1459585848 conn(0x7f12bc0a94d0 msgr2=0x7f12bc1beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 shutdown_connections 2024-09-17T16:27:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 -- 172.21.15.105:0/1459585848 wait complete. 2024-09-17T16:27:42.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:42.820+0000 7f12c3dc7640 1 librados: shutdown 2024-09-17T16:27:42.847 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T16:27:42.847 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T16:27:42.847 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:42.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T16:27:42.857 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:42.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T16:27:42.911 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-17T16:27:42.911 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T16:27:42.965 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:42.965 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T16:27:42.981 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:42.981 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T16:27:43.034 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:27:43.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T16:27:43.089 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new da6872ee-3d6d-43b3-9bf3-57913747db7a 0 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: get_monmap_and_config 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: monmap: 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.256 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:43.255607+0000 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:43.255607+0000 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.257 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:43.258 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:43.258 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 Processor -- start 2024-09-17T16:27:43.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 -- start start 2024-09-17T16:27:43.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: init 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient: _add_conns ranks=[4,7,2,5,3,6,1,8,0,9,10] 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc780a5860 0x7fdc780a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-c con 0x7fdc780a5860 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-i con 0x7fdc780a18f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 0x7fdc78137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-b con 0x7fdc780a4070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780a66c0 con 0x7fdc780a4070 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdc780a1d70 con 0x7fdc780a5860 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdc780a33d0 con 0x7fdc780a18f0 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7fca1640 10 monclient(hunting): authenticate will time out at 941.523587s 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.160:33888/0 (socket says 172.21.15.160:33888) 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 1 -- 172.21.15.160:0/715629215 learned_addr learned my addr 172.21.15.160:0/715629215 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): get_auth_request con 0x7fdc780a18f0 auth_method 0 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7da3d640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc780a5860 0x7fdc780a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.250+0000 7fdc7e23e640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 0x7fdc78137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_done global_id 4120 payload 274 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 -- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc780a5860 msgr2=0x7fdc780a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc780a5860 0x7fdc780a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 -- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 msgr2=0x7fdc78137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 0x7fdc78137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 monclient: found mon.noname-i 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 -- 172.21.15.160:0/715629215 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc780a36b0 con 0x7fdc780a18f0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7e23e640 10 monclient: get_auth_request con 0x7fdc780a4070 auth_method 0 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7e23e640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 0x7fdc78137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:43.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c01c210 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a806e7451a59e27d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 1 -- 172.21.15.160:0/715629215 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc6c0b0070 con 0x7fdc780a18f0 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: dump: 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.264 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 1 -- 172.21.15.160:0/715629215 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c0aa810 con 0x7fdc780a18f0 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 5 monclient: authenticate success, global_id 4120 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 1 -- 172.21.15.160:0/715629215 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc6c0a9300 con 0x7fdc780a18f0 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 4 set_mon_vals no callback set 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7ca3b640 10 monclient: dump: 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.266 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.267 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 10 monclient: shutdown 2024-09-17T16:27:43.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 -- 172.21.15.160:0/715629215 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 msgr2=0x7fdc780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c01c210 comp rx=0 tx=0).stop 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 -- 172.21.15.160:0/715629215 shutdown_connections 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc780a4070 0x7fdc78137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc780a6180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/715629215 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc780a5860 0x7fdc780a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 -- 172.21.15.160:0/715629215 >> 172.21.15.160:0/715629215 conn(0x7fdc780aa450 msgr2=0x7fdc780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 -- 172.21.15.160:0/715629215 shutdown_connections 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 1 -- 172.21.15.160:0/715629215 wait complete. 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.254+0000 7fdc7fca1640 10 monclient: monmap: 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.273 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.274 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 librados: starting msgr at 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 librados: starting objecter 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 Processor -- start 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- start start 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 librados: setting wanted keys 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 librados: calling monclient init 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: init 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _add_conns ranks=[7,1,6,5,3,2,0,4,8] 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc78133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-i con 0x7fdc780a18f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-f con 0x7fdc780a4070 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:27:43.275 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdc780a5860 0x7fdc78071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): picked mon.noname-d con 0x7fdc780a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78141480 con 0x7fdc780a4070 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdc78141300 con 0x7fdc780a5860 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdc78141600 con 0x7fdc780a18f0 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient(hunting): authenticate will time out at 941.528769s 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc78133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc78133080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.160:33890/0 (socket says 172.21.15.160:33890) 2024-09-17T16:27:43.276 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58384/0 (socket says 172.21.15.160:58384) 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 1 -- 172.21.15.160:0/3853073360 learned_addr learned my addr 172.21.15.160:0/3853073360 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): get_auth_request con 0x7fdc780a18f0 auth_method 0 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): get_auth_request con 0x7fdc780a4070 auth_method 0 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7da3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7e23e640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdc780a5860 0x7fdc78071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7e23e640 10 monclient(hunting): get_auth_request con 0x7fdc780a5860 auth_method 0 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7e23e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7e23e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7e23e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient(hunting): handle_auth_done global_id 4123 payload 824 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 -- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 msgr2=0x7fdc78133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc78133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 -- 172.21.15.160:0/3853073360 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdc780a5860 msgr2=0x7fdc78071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdc780a5860 0x7fdc78071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 monclient: found mon.noname-f 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 -- 172.21.15.160:0/3853073360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7813dcc0 con 0x7fdc780a4070 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7d23c640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009870 tx=0x7fdc6c018fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 client.?.objecter ms_handle_connect 0x7fdc780a4070 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc6c0b0040 con 0x7fdc780a4070 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: dump: 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.279 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.280 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c003620 con 0x7fdc780a4070 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc6c003c40 con 0x7fdc780a4070 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 monclient: dump: 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.281 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.282 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc77fff640 4 set_mon_vals no callback set 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 5 monclient: authenticate success, global_id 4123 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _renew_subs 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc78120540 con 0x7fdc780a4070 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 client.4123.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _renew_subs 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc78133b00 con 0x7fdc780a4070 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 librados: init done 2024-09-17T16:27:43.283 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc6c004180 con 0x7fdc780a4070 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdc6c0b4ca0 con 0x7fdc780a4070 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 10 client.4123.objecter ms_dispatch 0x7fdc78000ba0 osd_map(2..2 src has 1..2) 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 3 client.4123.objecter handle_osd_map got epochs [2,2] > 0 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 3 client.4123.objecter handle_osd_map decoding full epoch 2 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc767fc640 20 client.4123.objecter dump_active .. 0 homeless 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.258+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc78137610 con 0x7fdc780a4070 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.262+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdc6c0b9060 con 0x7fdc780a4070 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.262+0000 7fdc767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.262+0000 7fdc767fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:43.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.334+0000 7fdc7fca1640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "da6872ee-3d6d-43b3-9bf3-57913747db7a", "id": 0}] 2024-09-17T16:27:43.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.334+0000 7fdc7fca1640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "da6872ee-3d6d-43b3-9bf3-57913747db7a", "id": 0}] 2024-09-17T16:27:43.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.334+0000 7fdc7fca1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.334+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da6872ee-3d6d-43b3-9bf3-57913747db7a", "id": 0} v 0) -- 0x7fdc78069fa0 con 0x7fdc780a4070 2024-09-17T16:27:43.345 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.338+0000 7fdc767fc640 1 -- 172.21.15.160:0/3853073360 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da6872ee-3d6d-43b3-9bf3-57913747db7a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdc6c016150 con 0x7fdc780a4070 2024-09-17T16:27:43.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.338+0000 7fdc767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "da6872ee-3d6d-43b3-9bf3-57913747db7a", "id": 0}] 2024-09-17T16:27:43.346 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.338+0000 7fdc767fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:43.346 INFO:teuthology.orchestra.run.smithi160.stdout:0 2024-09-17T16:27:43.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 10 librados: watch_flush enter 2024-09-17T16:27:43.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 10 librados: watch_flush exit 2024-09-17T16:27:43.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 20 client.4123.objecter shutdown clearing up homeless session... 2024-09-17T16:27:43.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 10 client.4123.objecter successfully canceled tick 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 10 monclient: shutdown 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 msgr2=0x7fdc781335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009870 tx=0x7fdc6c018fb0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 shutdown_connections 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdc780a5860 0x7fdc78071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc780a4070 0x7fdc781335c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 --2- 172.21.15.160:0/3853073360 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc780a18f0 0x7fdc78133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 >> 172.21.15.160:0/3853073360 conn(0x7fdc780aa450 msgr2=0x7fdc7813d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 shutdown_connections 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 -- 172.21.15.160:0/3853073360 wait complete. 2024-09-17T16:27:43.348 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.342+0000 7fdc7fca1640 1 librados: shutdown 2024-09-17T16:27:43.364 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new ed40322f-b150-4b91-94f8-556fad6e845b 1 2024-09-17T16:27:43.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: get_monmap_and_config 2024-09-17T16:27:43.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: monmap: 2024-09-17T16:27:43.487 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:43.486739+0000 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:43.486739+0000 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.489 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:43.489 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:43.489 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 Processor -- start 2024-09-17T16:27:43.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 -- start start 2024-09-17T16:27:43.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: init 2024-09-17T16:27:43.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient: _add_conns ranks=[3,0,2,7,8,1,6,9,5,10,4] 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-g con 0x7f3f1c09b410 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3f1c0a2690 addr v1:172.21.15.105:6789/0 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c136020 0x7f3f1c13dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-b con 0x7f3f1c136020 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1c0a4cd0 con 0x7f3f1c0a2690 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c0a0e40 con 0x7f3f1c136020 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c0a1120 con 0x7f3f1c09b410 2024-09-17T16:27:43.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f22875640 10 monclient(hunting): authenticate will time out at 941.754726s 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:49448/0 (socket says 172.21.15.160:49448) 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 -- 172.21.15.160:0/443016737 learned_addr learned my addr 172.21.15.160:0/443016737 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): get_auth_request con 0x7f3f1c09b410 auth_method 0 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f20e12640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c136020 0x7f3f1c13dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f20e12640 10 monclient(hunting): get_auth_request con 0x7f3f1c136020 auth_method 0 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f20e12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f20e12640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f20e12640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient(hunting): handle_auth_done global_id 4116 payload 274 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 -- 172.21.15.160:0/443016737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c136020 msgr2=0x7f3f1c13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c136020 0x7f3f1c13dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 -- 172.21.15.160:0/443016737 >> v1:172.21.15.105:6789/0 conn(0x7f3f1c0a2690 legacy=0x7f3f1c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 monclient: found mon.noname-g 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.482+0000 7f3f1bfff640 1 -- 172.21.15.160:0/443016737 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c0a1400 con 0x7f3f1c09b410 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1bfff640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400bb40 tx=0x7f3f1401eb10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=cc34613fd4eb3e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.494 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 1 -- 172.21.15.160:0/443016737 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f140b4070 con 0x7f3f1c09b410 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: dump: 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.495 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 1 -- 172.21.15.160:0/443016737 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140a7c00 con 0x7f3f1c09b410 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 1 -- 172.21.15.160:0/443016737 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f140a7df0 con 0x7f3f1c09b410 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 5 monclient: authenticate success, global_id 4116 2024-09-17T16:27:43.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 4 set_mon_vals no callback set 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1affd640 10 monclient: dump: 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.497 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.498 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: shutdown 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- 172.21.15.160:0/443016737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 msgr2=0x7f3f1c09b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400bb40 tx=0x7f3f1401eb10 comp rx=0 tx=0).stop 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- 172.21.15.160:0/443016737 shutdown_connections 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c136020 0x7f3f1c13dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- 172.21.15.160:0/443016737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c09b410 0x7f3f1c09b7f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- 172.21.15.160:0/443016737 >> 172.21.15.160:0/443016737 conn(0x7f3f1c0a83e0 msgr2=0x7f3f1c0a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- 172.21.15.160:0/443016737 shutdown_connections 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- 172.21.15.160:0/443016737 wait complete. 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: monmap: 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.503 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.504 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 librados: starting msgr at 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 librados: starting objecter 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 Processor -- start 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- start start 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 librados: setting wanted keys 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 librados: calling monclient init 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: init 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient: _add_conns ranks=[2,3,8,1,0,5,4,7,6] 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c09b410 0x7f3f1c12d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-b con 0x7f3f1c09b410 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-g con 0x7f3f1c136020 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3f1c0746a0 0x7f3f1c12df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): picked mon.noname-e con 0x7f3f1c0746a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c1401e0 con 0x7f3f1c09b410 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c140360 con 0x7f3f1c136020 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3f1c1404e0 con 0x7f3f1c0746a0 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f22875640 10 monclient(hunting): authenticate will time out at 941.759768s 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:49460/0 (socket says 172.21.15.160:49460) 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 1 -- 172.21.15.160:0/2071457288 learned_addr learned my addr 172.21.15.160:0/2071457288 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 10 monclient(hunting): get_auth_request con 0x7f3f1c136020 auth_method 0 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.486+0000 7f3f1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1bfff640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c09b410 0x7f3f1c12d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f20e12640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3f1c0746a0 0x7f3f1c12df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1bfff640 10 monclient(hunting): get_auth_request con 0x7f3f1c09b410 auth_method 0 2024-09-17T16:27:43.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f20e12640 10 monclient(hunting): get_auth_request con 0x7f3f1c0746a0 auth_method 0 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f20e12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f20e12640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f20e12640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient(hunting): handle_auth_done global_id 4125 payload 824 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 -- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3f1c0746a0 msgr2=0x7f3f1c12df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3f1c0746a0 0x7f3f1c12df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 -- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c09b410 msgr2=0x7f3f1c12d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c09b410 0x7f3f1c12d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 monclient: found mon.noname-g 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 -- 172.21.15.160:0/2071457288 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c133120 con 0x7f3f1c136020 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1b7fe640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c440 tx=0x7f3f1000c900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 client.?.objecter ms_handle_connect 0x7f3f1c136020 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f10016030 con 0x7f3f1c136020 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: dump: 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.509 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f10005490 con 0x7f3f1c136020 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f10012aa0 con 0x7f3f1c136020 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 monclient: dump: 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.511 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f1a7fc640 4 set_mon_vals no callback set 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 5 monclient: authenticate success, global_id 4125 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _renew_subs 2024-09-17T16:27:43.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1307c0 con 0x7f3f1c136020 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 client.4125.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _renew_subs 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c12f510 con 0x7f3f1c136020 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 1 librados: init done 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f10012cc0 con 0x7f3f1c136020 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3f10017070 con 0x7f3f1c136020 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 10 client.4125.objecter ms_dispatch 0x7f3f1c000ba0 osd_map(3..3 src has 1..3) 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 3 client.4125.objecter handle_osd_map got epochs [3,3] > 0 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 3 client.4125.objecter handle_osd_map decoding full epoch 3 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f18ff9640 20 client.4125.objecter dump_active .. 0 homeless 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.490+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c13da90 con 0x7f3f1c136020 2024-09-17T16:27:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.494+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3f1001f020 con 0x7f3f1c136020 2024-09-17T16:27:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.494+0000 7f3f18ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.494+0000 7f3f18ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:43.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.562+0000 7f3f22875640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ed40322f-b150-4b91-94f8-556fad6e845b", "id": 1}] 2024-09-17T16:27:43.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.562+0000 7f3f22875640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ed40322f-b150-4b91-94f8-556fad6e845b", "id": 1}] 2024-09-17T16:27:43.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.562+0000 7f3f22875640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.569 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.562+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed40322f-b150-4b91-94f8-556fad6e845b", "id": 1} v 0) -- 0x7f3f1c06d7a0 con 0x7f3f1c136020 2024-09-17T16:27:43.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f18ff9640 1 -- 172.21.15.160:0/2071457288 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed40322f-b150-4b91-94f8-556fad6e845b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3f1c1307c0 con 0x7f3f1c136020 2024-09-17T16:27:43.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f18ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ed40322f-b150-4b91-94f8-556fad6e845b", "id": 1}] 2024-09-17T16:27:43.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f18ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:43.584 INFO:teuthology.orchestra.run.smithi160.stdout:1 2024-09-17T16:27:43.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 10 librados: watch_flush enter 2024-09-17T16:27:43.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 10 librados: watch_flush exit 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 20 client.4125.objecter shutdown clearing up homeless session... 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 10 client.4125.objecter successfully canceled tick 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 10 monclient: shutdown 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 msgr2=0x7f3f1c12d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c440 tx=0x7f3f1000c900 comp rx=0 tx=0).stop 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 shutdown_connections 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3f1c0746a0 0x7f3f1c12df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f1c136020 0x7f3f1c12d9e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 --2- 172.21.15.160:0/2071457288 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c09b410 0x7f3f1c12d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 >> 172.21.15.160:0/2071457288 conn(0x7f3f1c0a83e0 msgr2=0x7f3f1c0a62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 shutdown_connections 2024-09-17T16:27:43.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 -- 172.21.15.160:0/2071457288 wait complete. 2024-09-17T16:27:43.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.578+0000 7f3f22875640 1 librados: shutdown 2024-09-17T16:27:43.600 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new a4157a35-9183-4b1a-9afc-4e08def6d0b1 2 2024-09-17T16:27:43.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: get_monmap_and_config 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: monmap: 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:43.715225+0000 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:43.715225+0000 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.716 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:43.717 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 Processor -- start 2024-09-17T16:27:43.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 -- start start 2024-09-17T16:27:43.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: init 2024-09-17T16:27:43.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient: _add_conns ranks=[10,0,7,9,6,3,1,4,2,5,8] 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8580a5860 0x7fd8580a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-f con 0x7fd8580a5860 addr v2:172.21.15.160:3300/0 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8580a18f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8580a4070 0x7fd8580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-i con 0x7fd8580a4070 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8580a1d70 con 0x7fd8580a18f0 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fd8580a66c0 con 0x7fd8580a5860 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fd8580a33d0 con 0x7fd8580a4070 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85cb98640 10 monclient(hunting): authenticate will time out at 941.982994s 2024-09-17T16:27:43.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8580a5860 0x7fd8580a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8580a5860 0x7fd8580a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58386/0 (socket says 172.21.15.160:58386) 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 -- 172.21.15.160:0/4206248469 learned_addr learned my addr 172.21.15.160:0/4206248469 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): get_auth_request con 0x7fd8580a5860 auth_method 0 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8580a4070 0x7fd8580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): get_auth_request con 0x7fd8580a4070 auth_method 0 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd856d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient(hunting): handle_auth_done global_id 4132 payload 274 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 -- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8580a4070 msgr2=0x7fd8580a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8580a4070 0x7fd8580a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 -- 172.21.15.160:0/4206248469 >> v1:172.21.15.105:6789/0 conn(0x7fd8580a18f0 legacy=0x7fd858137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 monclient: found mon.noname-f 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.710+0000 7fd85659c640 1 -- 172.21.15.160:0/4206248469 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8580a36b0 con 0x7fd8580a5860 2024-09-17T16:27:43.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85659c640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8580a5860 0x7fd8580a5c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd850009e80 tx=0x7fd85001c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fd83fecbd60915 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 1 -- 172.21.15.160:0/4206248469 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8500ac020 con 0x7fd8580a5860 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: dump: 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.728 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 1 -- 172.21.15.160:0/4206248469 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850009110 con 0x7fd8580a5860 2024-09-17T16:27:43.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 5 monclient: authenticate success, global_id 4132 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 1 -- 172.21.15.160:0/4206248469 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8500a6d60 con 0x7fd8580a5860 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 4 set_mon_vals no callback set 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85559a640 10 monclient: dump: 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.730 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 10 monclient: shutdown 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8580a5860 msgr2=0x7fd8580a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8580a5860 0x7fd8580a5c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd850009e80 tx=0x7fd85001c6f0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- 172.21.15.160:0/4206248469 shutdown_connections 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8580a4070 0x7fd8580a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 --2- 172.21.15.160:0/4206248469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8580a5860 0x7fd8580a5c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- 172.21.15.160:0/4206248469 >> 172.21.15.160:0/4206248469 conn(0x7fd8580aa450 msgr2=0x7fd8580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- 172.21.15.160:0/4206248469 shutdown_connections 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- 172.21.15.160:0/4206248469 wait complete. 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 10 monclient: monmap: 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.732 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 librados: starting msgr at 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 librados: starting objecter 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 Processor -- start 2024-09-17T16:27:43.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 -- start start 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 librados: setting wanted keys 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 1 librados: calling monclient init 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.714+0000 7fd85cb98640 10 monclient: init 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _add_conns ranks=[4,3,6,8,7,5,2,0,1] 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 0x7fd8581d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-c con 0x7fd8580a18f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-g con 0x7fd8580a4070 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 0x7fd85806d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): picked mon.noname-d con 0x7fd8580a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd858142490 con 0x7fd8580a4070 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fd858142310 con 0x7fd8580a18f0 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd858142610 con 0x7fd8580a5860 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient(hunting): authenticate will time out at 941.988602s 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:49470/0 (socket says 172.21.15.160:49470) 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 -- 172.21.15.160:0/3497582878 learned_addr learned my addr 172.21.15.160:0/3497582878 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): get_auth_request con 0x7fd8580a4070 auth_method 0 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd856d9d640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 0x7fd85806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85659c640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 0x7fd8581d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 -- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 msgr2=0x7fd85806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 0x7fd85806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 -- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 msgr2=0x7fd8581d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 0x7fd8581d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 monclient: found mon.noname-g 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.736 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 -- 172.21.15.160:0/3497582878 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8580a0420 con 0x7fd8580a4070 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd856d9d640 10 monclient: get_auth_request con 0x7fd8580a5860 auth_method 0 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85659c640 10 monclient: get_auth_request con 0x7fd8580a18f0 auth_method 0 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd856d9d640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 0x7fd85806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85659c640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 0x7fd8581d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd855d9b640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c490 tx=0x7fd84c001e80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 client.?.objecter ms_handle_connect 0x7fd8580a4070 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd84c017070 con 0x7fd8580a4070 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: dump: 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.737 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c0052e0 con 0x7fd8580a4070 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 5 monclient: authenticate success, global_id 4134 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _renew_subs 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 -- 172.21.15.160:0/3497582878 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85813c770 con 0x7fd8580a4070 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd84c010300 con 0x7fd8580a4070 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 monclient: dump: 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.739 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.740 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd854d99640 4 set_mon_vals no callback set 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 client.4134.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _renew_subs 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 -- 172.21.15.160:0/3497582878 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85806ebd0 con 0x7fd8580a4070 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd85cb98640 1 librados: init done 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd84c003600 con 0x7fd8580a4070 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fd84c012400 con 0x7fd8580a4070 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 10 client.4134.objecter ms_dispatch 0x7fd858000ba0 osd_map(4..4 src has 1..4) 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 3 client.4134.objecter handle_osd_map got epochs [4,4] > 0 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 3 client.4134.objecter handle_osd_map decoding full epoch 4 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd846ffd640 20 client.4134.objecter dump_active .. 0 homeless 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd844ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd844ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd844ff9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.718+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd830005090 con 0x7fd8580a4070 2024-09-17T16:27:43.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.722+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd84c020070 con 0x7fd8580a4070 2024-09-17T16:27:43.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.722+0000 7fd846ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.722+0000 7fd846ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:43.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.794+0000 7fd844ff9640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a4157a35-9183-4b1a-9afc-4e08def6d0b1", "id": 2}] 2024-09-17T16:27:43.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.794+0000 7fd844ff9640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a4157a35-9183-4b1a-9afc-4e08def6d0b1", "id": 2}] 2024-09-17T16:27:43.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.794+0000 7fd844ff9640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:43.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.794+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4157a35-9183-4b1a-9afc-4e08def6d0b1", "id": 2} v 0) -- 0x7fd830005680 con 0x7fd8580a4070 2024-09-17T16:27:43.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.798+0000 7fd846ffd640 1 -- 172.21.15.160:0/3497582878 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4157a35-9183-4b1a-9afc-4e08def6d0b1", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fd8580a0420 con 0x7fd8580a4070 2024-09-17T16:27:43.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.798+0000 7fd846ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a4157a35-9183-4b1a-9afc-4e08def6d0b1", "id": 2}] 2024-09-17T16:27:43.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.798+0000 7fd846ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:43.806 INFO:teuthology.orchestra.run.smithi160.stdout:2 2024-09-17T16:27:43.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 10 librados: watch_flush enter 2024-09-17T16:27:43.807 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 10 librados: watch_flush exit 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 20 client.4134.objecter shutdown clearing up homeless session... 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 10 client.4134.objecter successfully canceled tick 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 10 monclient: shutdown 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 msgr2=0x7fd8581d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c490 tx=0x7fd84c001e80 comp rx=0 tx=0).stop 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 shutdown_connections 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8580a5860 0x7fd85806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8580a4070 0x7fd8581d1ce0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 --2- 172.21.15.160:0/3497582878 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd8580a18f0 0x7fd8581d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 >> 172.21.15.160:0/3497582878 conn(0x7fd8580aa450 msgr2=0x7fd8580a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.808 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 shutdown_connections 2024-09-17T16:27:43.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 -- 172.21.15.160:0/3497582878 wait complete. 2024-09-17T16:27:43.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.802+0000 7fd844ff9640 1 librados: shutdown 2024-09-17T16:27:43.822 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 994d2203-e554-437e-b09e-3b2861c4415e 3 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: get_monmap_and_config 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: monmap: 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:43.934616+0000 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:43.934616+0000 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.935 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:43.936 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 Processor -- start 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 -- start start 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: init 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient: _add_conns ranks=[0,10,1,4,7,2,9,3,6,5,8] 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44e009b510 addr v1:172.21.15.105:6789/0 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44e00a0540 0x7f44e013c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-f con 0x7f44e00a0540 addr v2:172.21.15.160:3300/0 2024-09-17T16:27:43.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f44e013cc50 addr v1:172.21.15.160:6789/0 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e00696c0 con 0x7f44e009b510 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e00694c0 con 0x7f44e013cc50 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f44e009fa80 con 0x7f44e00a0540 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44e55b4640 10 monclient(hunting): authenticate will time out at 942.202319s 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44e00a0540 0x7f44e013c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44e00a0540 0x7f44e013c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58402/0 (socket says 172.21.15.160:58402) 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44df7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f44e013cc50 0x7f44e013f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39066/0 (socket says 172.21.15.160:39066) 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 -- 172.21.15.160:0/1808246768 learned_addr learned my addr 172.21.15.160:0/1808246768 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): get_auth_request con 0x7f44e00a0540 auth_method 0 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.939 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient(hunting): handle_auth_done global_id 4141 payload 274 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 -- 172.21.15.160:0/1808246768 >> v1:172.21.15.160:6789/0 conn(0x7f44e013cc50 legacy=0x7f44e013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 -- 172.21.15.160:0/1808246768 >> v1:172.21.15.105:6789/0 conn(0x7f44e009b510 legacy=0x7f44e009b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 monclient: found mon.noname-f 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:27:43.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.930+0000 7f44de7fc640 1 -- 172.21.15.160:0/1808246768 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e009c000 con 0x7f44e00a0540 2024-09-17T16:27:43.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 1 --2- 172.21.15.160:0/1808246768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44e00a0540 0x7f44e013c710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b261f6a3b189fba server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 1 -- 172.21.15.160:0/1808246768 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44d40b0070 con 0x7f44e00a0540 2024-09-17T16:27:43.945 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: dump: 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.946 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 1 -- 172.21.15.160:0/1808246768 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d40aa810 con 0x7f44e00a0540 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 1 -- 172.21.15.160:0/1808246768 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44d40a9300 con 0x7f44e00a0540 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:27:43.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44ddffb640 10 monclient: dump: 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.948 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 5 monclient: authenticate success, global_id 4141 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 4 set_mon_vals no callback set 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: shutdown 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- 172.21.15.160:0/1808246768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44e00a0540 msgr2=0x7f44e013c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 --2- 172.21.15.160:0/1808246768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44e00a0540 0x7f44e013c710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d401c420 comp rx=0 tx=0).stop 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- 172.21.15.160:0/1808246768 shutdown_connections 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 --2- 172.21.15.160:0/1808246768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44e00a0540 0x7f44e013c710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- 172.21.15.160:0/1808246768 >> 172.21.15.160:0/1808246768 conn(0x7f44e00a4e70 msgr2=0x7f44e00a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- 172.21.15.160:0/1808246768 shutdown_connections 2024-09-17T16:27:43.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- 172.21.15.160:0/1808246768 wait complete. 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: build_initial_monmap 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: monmap: 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:43.950 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 librados: starting msgr at 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 librados: starting objecter 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 Processor -- start 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- start start 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 librados: setting wanted keys 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 librados: calling monclient init 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: init 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:43.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient: _add_conns ranks=[5,7,2,8,6,1,0,3,4] 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-h con 0x7f44e009b510 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44e00a0540 0x7f44e013bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-i con 0x7f44e00a0540 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 0x7f44e0073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): picked mon.noname-b con 0x7f44e013cc50 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f44e0143100 con 0x7f44e013cc50 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f44e0142f80 con 0x7f44e009b510 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f44e0143280 con 0x7f44e00a0540 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44e55b4640 10 monclient(hunting): authenticate will time out at 942.207540s 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.160:54038/0 (socket says 172.21.15.160:54038) 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 1 -- 172.21.15.160:0/925799436 learned_addr learned my addr 172.21.15.160:0/925799436 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): get_auth_request con 0x7f44e009b510 auth_method 0 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44e00a0540 0x7f44e013bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): get_auth_request con 0x7f44e00a0540 auth_method 0 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.934+0000 7f44df7fe640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 0x7f44e0073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 10 monclient(hunting): handle_auth_done global_id 4127 payload 824 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 -- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44e00a0540 msgr2=0x7f44e013bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44e00a0540 0x7f44e013bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 -- 172.21.15.160:0/925799436 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 msgr2=0x7f44e0073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 0x7f44e0073600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 10 monclient: _finish_hunting 0 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 monclient: found mon.noname-h 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 -- 172.21.15.160:0/925799436 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e00911b0 con 0x7f44e009b510 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44df7fe640 10 monclient: get_auth_request con 0x7f44e013cc50 auth_method 0 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44df7fe640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 0x7f44e0073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44deffd640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c8d0 tx=0x7f44d800cd90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 client.?.objecter ms_handle_connect 0x7f44e009b510 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44d8001900 con 0x7f44e009b510 2024-09-17T16:27:43.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: dump: 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.956 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: _finish_auth 0 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8003df0 con 0x7f44e009b510 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 5 monclient: authenticate success, global_id 4127 2024-09-17T16:27:43.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _renew_subs 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e0090910 con 0x7f44e009b510 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44d800acc0 con 0x7f44e009b510 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 monclient: dump: 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:43.958 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44dd7fa640 4 set_mon_vals no callback set 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 client.4127.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _renew_subs 2024-09-17T16:27:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e0090ed0 con 0x7f44e009b510 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 1 librados: init done 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44d8003980 con 0x7f44e009b510 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f44d80057b0 con 0x7f44e009b510 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 10 client.4127.objecter ms_dispatch 0x7f44e0000ba0 osd_map(5..5 src has 1..5) 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 3 client.4127.objecter handle_osd_map got epochs [5,5] > 0 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 3 client.4127.objecter handle_osd_map decoding full epoch 5 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44c77fe640 20 client.4127.objecter dump_active .. 0 homeless 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:27:43.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.938+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e009b8f0 con 0x7f44e009b510 2024-09-17T16:27:43.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.942+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f44d8020020 con 0x7f44e009b510 2024-09-17T16:27:43.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.942+0000 7f44c77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:43.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:43.942+0000 7f44c77fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:44.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.010+0000 7f44e55b4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "994d2203-e554-437e-b09e-3b2861c4415e", "id": 3}] 2024-09-17T16:27:44.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.010+0000 7f44e55b4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "994d2203-e554-437e-b09e-3b2861c4415e", "id": 3}] 2024-09-17T16:27:44.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.010+0000 7f44e55b4640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:27:44.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.010+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_command({"prefix": "osd new", "uuid": "994d2203-e554-437e-b09e-3b2861c4415e", "id": 3} v 0) -- 0x7f44e006d710 con 0x7f44e009b510 2024-09-17T16:27:44.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44c77fe640 1 -- 172.21.15.160:0/925799436 <== mon.5 v2:172.21.15.160:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "994d2203-e554-437e-b09e-3b2861c4415e", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f44d8006b60 con 0x7f44e009b510 2024-09-17T16:27:44.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44c77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "994d2203-e554-437e-b09e-3b2861c4415e", "id": 3}] 2024-09-17T16:27:44.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44c77fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:44.024 INFO:teuthology.orchestra.run.smithi160.stdout:3 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 10 librados: watch_flush enter 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 10 librados: watch_flush exit 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 20 client.4127.objecter shutdown clearing up homeless session... 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 10 client.4127.objecter successfully canceled tick 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 10 monclient: shutdown 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 msgr2=0x7f44e013b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c8d0 tx=0x7f44d800cd90 comp rx=0 tx=0).stop 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 shutdown_connections 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44e013cc50 0x7f44e0073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44e00a0540 0x7f44e013bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 --2- 172.21.15.160:0/925799436 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44e009b510 0x7f44e013b7a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 >> 172.21.15.160:0/925799436 conn(0x7f44e00a4e70 msgr2=0x7f44e00a4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 shutdown_connections 2024-09-17T16:27:44.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 -- 172.21.15.160:0/925799436 wait complete. 2024-09-17T16:27:44.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.018+0000 7f44e55b4640 1 librados: shutdown 2024-09-17T16:27:44.040 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new ce840dc5-8cda-4070-9106-591c85c3f28b 4 2024-09-17T16:27:44.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: get_monmap_and_config 2024-09-17T16:27:44.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: monmap: 2024-09-17T16:27:44.156 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:44.155857+0000 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:44.155857+0000 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.157 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.158 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.158 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:44.158 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:44.158 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 Processor -- start 2024-09-17T16:27:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 -- start start 2024-09-17T16:27:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: init 2024-09-17T16:27:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient: _add_conns ranks=[2,5,7,3,8,10,4,0,9,1,6] 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 0x7f2b4c09b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-b con 0x7f2b4c09b510 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 0x7f2b4c0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-h con 0x7f2b4c0a25a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-i con 0x7f2b4c0a42c0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c09f3b0 con 0x7f2b4c09b510 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f2b4c0a18e0 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2b4c0a1bc0 con 0x7f2b4c0a42c0 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b546c8640 10 monclient(hunting): authenticate will time out at 942.423740s 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b51c63640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 0x7f2b4c0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.160:33910/0 (socket says 172.21.15.160:33910) 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 1 -- 172.21.15.160:0/641266782 learned_addr learned my addr 172.21.15.160:0/641266782 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 10 monclient(hunting): get_auth_request con 0x7f2b4c0a42c0 auth_method 0 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b52c65640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b51c63640 10 monclient(hunting): get_auth_request con 0x7f2b4c0a25a0 auth_method 0 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b51c63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b51c63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.150+0000 7f2b51c63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52c65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52c65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 0x7f2b4c09b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52c65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 10 monclient(hunting): get_auth_request con 0x7f2b4c09b510 auth_method 0 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient(hunting): handle_auth_done global_id 4136 payload 274 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 -- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 msgr2=0x7f2b4c0a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 -- 172.21.15.160:0/641266782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 msgr2=0x7f2b4c09b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 0x7f2b4c09b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 monclient: found mon.noname-h 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 -- 172.21.15.160:0/641266782 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c0a5c20 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52c65640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b52464640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 0x7f2b4c09b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51c63640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 0x7f2b4c0a3d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0097f0 tx=0x7f2b3c01c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5287306b88dc5cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 1 -- 172.21.15.160:0/641266782 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b3c0b0070 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:27:44.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: dump: 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.167 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 1 -- 172.21.15.160:0/641266782 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0aa810 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 5 monclient: authenticate success, global_id 4136 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 4 set_mon_vals no callback set 2024-09-17T16:27:44.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 1 -- 172.21.15.160:0/641266782 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b3c0a9300 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b51462640 10 monclient: dump: 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.169 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 10 monclient: shutdown 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 -- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 msgr2=0x7f2b4c0a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 0x7f2b4c0a3d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0097f0 tx=0x7f2b3c01c3a0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 -- 172.21.15.160:0/641266782 shutdown_connections 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a42c0 0x7f2b4c0a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b4c0a25a0 0x7f2b4c0a3d80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 --2- 172.21.15.160:0/641266782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c09b510 0x7f2b4c09b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 -- 172.21.15.160:0/641266782 >> 172.21.15.160:0/641266782 conn(0x7f2b4c0a4e70 msgr2=0x7f2b4c0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 -- 172.21.15.160:0/641266782 shutdown_connections 2024-09-17T16:27:44.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 1 -- 172.21.15.160:0/641266782 wait complete. 2024-09-17T16:27:44.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.154+0000 7f2b546c8640 10 monclient: monmap: 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.175 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 librados: starting msgr at 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 librados: starting objecter 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 Processor -- start 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- start start 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 librados: setting wanted keys 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 librados: calling monclient init 2024-09-17T16:27:44.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: init 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _add_conns ranks=[8,7,2,1,3,0,6,4,5] 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2b4c09b510 0x7f2b4c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-e con 0x7f2b4c09b510 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-i con 0x7f2b4c0a25a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c0a42c0 0x7f2b4c070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): picked mon.noname-b con 0x7f2b4c0a42c0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c140140 con 0x7f2b4c0a42c0 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2b4c09c040 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f2b4c1402c0 con 0x7f2b4c09b510 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient(hunting): authenticate will time out at 942.428713s 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.160:33918/0 (socket says 172.21.15.160:33918) 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 -- 172.21.15.160:0/4015856515 learned_addr learned my addr 172.21.15.160:0/4015856515 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): get_auth_request con 0x7f2b4c0a25a0 auth_method 0 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b52c65640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c0a42c0 0x7f2b4c070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient(hunting): handle_auth_done global_id 4165 payload 824 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b52464640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2b4c09b510 0x7f2b4c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 -- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2b4c09b510 msgr2=0x7f2b4c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2b4c09b510 0x7f2b4c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 -- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c0a42c0 msgr2=0x7f2b4c070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c0a42c0 0x7f2b4c070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 monclient: found mon.noname-i 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 -- 172.21.15.160:0/4015856515 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c072140 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b51c63640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c01c1d0 tx=0x7f2b3c019140 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 client.?.objecter ms_handle_connect 0x7f2b4c0a25a0 2024-09-17T16:27:44.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b3c0b0040 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: dump: 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.180 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0b9070 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 5 monclient: authenticate success, global_id 4165 2024-09-17T16:27:44.181 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _renew_subs 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c072420 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b3c0044b0 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b50c61640 4 set_mon_vals no callback set 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 monclient: dump: 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.182 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 client.4165.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _renew_subs 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c09ae60 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 librados: init done 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b3c0aac90 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f2b3c0be460 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 10 client.4165.objecter ms_dispatch 0x7f2b4c000ba0 osd_map(6..6 src has 1..6) 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 3 client.4165.objecter handle_osd_map got epochs [6,6] > 0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 3 client.4165.objecter handle_osd_map decoding full epoch 6 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b42ffd640 20 client.4165.objecter dump_active .. 0 homeless 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.158+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c0a3d80 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.162+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2b3c0160b0 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.162+0000 7f2b42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.162+0000 7f2b42ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:44.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.230+0000 7f2b546c8640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ce840dc5-8cda-4070-9106-591c85c3f28b", "id": 4}] 2024-09-17T16:27:44.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.230+0000 7f2b546c8640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ce840dc5-8cda-4070-9106-591c85c3f28b", "id": 4}] 2024-09-17T16:27:44.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.230+0000 7f2b546c8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.230+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "osd new", "uuid": "ce840dc5-8cda-4070-9106-591c85c3f28b", "id": 4} v 0) -- 0x7f2b4c098040 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b42ffd640 1 -- 172.21.15.160:0/4015856515 <== mon.7 v2:172.21.15.160:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce840dc5-8cda-4070-9106-591c85c3f28b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f2b3c0aa7e0 con 0x7f2b4c0a25a0 2024-09-17T16:27:44.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ce840dc5-8cda-4070-9106-591c85c3f28b", "id": 4}] 2024-09-17T16:27:44.244 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b42ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:44.245 INFO:teuthology.orchestra.run.smithi160.stdout:4 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 10 librados: watch_flush enter 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 10 librados: watch_flush exit 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 20 client.4165.objecter shutdown clearing up homeless session... 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 10 client.4165.objecter successfully canceled tick 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 10 monclient: shutdown 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.246 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 msgr2=0x7f2b4c070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c01c1d0 tx=0x7f2b3c019140 comp rx=0 tx=0).stop 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 shutdown_connections 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2b4c0a42c0 0x7f2b4c070a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2b4c0a25a0 0x7f2b4c070520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 --2- 172.21.15.160:0/4015856515 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2b4c09b510 0x7f2b4c06ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 >> 172.21.15.160:0/4015856515 conn(0x7f2b4c0a4e70 msgr2=0x7f2b4c09f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 shutdown_connections 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 -- 172.21.15.160:0/4015856515 wait complete. 2024-09-17T16:27:44.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.238+0000 7f2b546c8640 1 librados: shutdown 2024-09-17T16:27:44.261 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 4a264646-336c-4e1e-97e0-bbc5e144e545 5 2024-09-17T16:27:44.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: get_monmap_and_config 2024-09-17T16:27:44.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: monmap: 2024-09-17T16:27:44.376 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:44.376 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:44.375794+0000 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:44.375794+0000 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.377 INFO:teuthology.orchestra.run.smithi160.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:44.378 INFO:teuthology.orchestra.run.smithi160.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:44.378 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 Processor -- start 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 -- start start 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: init 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient: _add_conns ranks=[6,8,10,1,7,5,9,2,0,3,4] 2024-09-17T16:27:44.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 0x7ffb840a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-d con 0x7ffb840a2e50 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb840a3770 0x7ffb840a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-e con 0x7ffb840a3770 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffb8409b050 0x7ffb8409b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-f con 0x7ffb8409b050 addr v2:172.21.15.160:3300/0 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ffb840a28e0 con 0x7ffb8409b050 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ffb840a40d0 con 0x7ffb840a2e50 2024-09-17T16:27:44.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ffb8409c4f0 con 0x7ffb840a3770 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb8b174640 10 monclient(hunting): authenticate will time out at 942.643701s 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffb8409b050 0x7ffb8409b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffb8409b050 0x7ffb8409b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58404/0 (socket says 172.21.15.160:58404) 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 1 -- 172.21.15.160:0/1133244089 learned_addr learned my addr 172.21.15.160:0/1133244089 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 10 monclient(hunting): get_auth_request con 0x7ffb8409b050 auth_method 0 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.370+0000 7ffb89711640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb840a3770 0x7ffb840a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): get_auth_request con 0x7ffb840a3770 auth_method 0 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 0x7ffb840a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 10 monclient(hunting): get_auth_request con 0x7ffb840a2e50 auth_method 0 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient(hunting): handle_auth_done global_id 4159 payload 274 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 -- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb840a3770 msgr2=0x7ffb840a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb840a3770 0x7ffb840a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 -- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 msgr2=0x7ffb840a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 0x7ffb840a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 monclient: found mon.noname-f 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 -- 172.21.15.160:0/1133244089 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8409c7d0 con 0x7ffb8409b050 2024-09-17T16:27:44.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb88f10640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 0x7ffb840a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb89711640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8409b050 0x7ffb8409b4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009e50 tx=0x7ffb7c01d320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb8976320d7233c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 1 -- 172.21.15.160:0/1133244089 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb7c0090d0 con 0x7ffb8409b050 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: dump: 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.390 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 1 -- 172.21.15.160:0/1133244089 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0a5b90 con 0x7ffb8409b050 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 1 -- 172.21.15.160:0/1133244089 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb7c0a5d80 con 0x7ffb8409b050 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb8b174640 5 monclient: authenticate success, global_id 4159 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb8b174640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb8b174640 4 set_mon_vals no callback set 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.374+0000 7ffb7b7fe640 10 monclient: dump: 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.392 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.393 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: shutdown 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/1133244089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8409b050 msgr2=0x7ffb8409b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8409b050 0x7ffb8409b4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009e50 tx=0x7ffb7c01d320 comp rx=0 tx=0).stop 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/1133244089 shutdown_connections 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8409b050 0x7ffb8409b4b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb840a3770 0x7ffb840a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- 172.21.15.160:0/1133244089 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb840a2e50 0x7ffb840a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/1133244089 >> 172.21.15.160:0/1133244089 conn(0x7ffb840a8620 msgr2=0x7ffb840a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/1133244089 shutdown_connections 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/1133244089 wait complete. 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: monmap: 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 0 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 0.000000 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:created 0.000000 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 1 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.395 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 librados: starting msgr at 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 librados: starting objecter 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 Processor -- start 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- start start 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 librados: setting wanted keys 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 librados: calling monclient init 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: init 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _add_conns ranks=[3,0,2,8,1,6,5,7,4] 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-g con 0x7ffb8409b050 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb840a2e50 0x7ffb8409a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-a con 0x7ffb840a2e50 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb840a3770 0x7ffb84072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): picked mon.noname-b con 0x7ffb840a3770 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb840a44a0 con 0x7ffb840a2e50 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ffb840a4320 con 0x7ffb840a3770 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ffb84141230 con 0x7ffb8409b050 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient(hunting): authenticate will time out at 942.650085s 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:49474/0 (socket says 172.21.15.160:49474) 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 -- 172.21.15.160:0/888192806 learned_addr learned my addr 172.21.15.160:0/888192806 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): get_auth_request con 0x7ffb8409b050 auth_method 0 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb840a2e50 0x7ffb8409a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): get_auth_request con 0x7ffb840a2e50 auth_method 0 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb89711640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb840a3770 0x7ffb84072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb89711640 10 monclient(hunting): get_auth_request con 0x7ffb840a3770 auth_method 0 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb89711640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb89711640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb89711640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient(hunting): handle_auth_done global_id 4143 payload 824 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 -- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb840a3770 msgr2=0x7ffb84072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb840a3770 0x7ffb84072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 -- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb840a2e50 msgr2=0x7ffb8409a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb840a2e50 0x7ffb8409a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 monclient: found mon.noname-g 2024-09-17T16:27:44.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 -- 172.21.15.160:0/888192806 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8412eb50 con 0x7ffb8409b050 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb88f10640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000e840 tx=0x7ffb8000ed00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 client.?.objecter ms_handle_connect 0x7ffb8409b050 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb8000f040 con 0x7ffb8409b050 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: dump: 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.400 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb800044a0 con 0x7ffb8409b050 2024-09-17T16:27:44.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 5 monclient: authenticate success, global_id 4143 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _renew_subs 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8412ee30 con 0x7ffb8409b050 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb8000ee30 con 0x7ffb8409b050 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb7affd640 4 set_mon_vals no callback set 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 client.4143.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 monclient: dump: 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:epoch 1 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.402 INFO:teuthology.orchestra.run.smithi160.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:election_strategy: 3 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _renew_subs 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8412e870 con 0x7ffb8409b050 2024-09-17T16:27:44.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb8b174640 1 librados: init done 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb8000c6d0 con 0x7ffb8409b050 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ffb80003bc0 con 0x7ffb8409b050 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 10 client.4143.objecter ms_dispatch 0x7ffb84000ba0 osd_map(7..7 src has 1..7) 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 3 client.4143.objecter handle_osd_map got epochs [7,7] > 0 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 3 client.4143.objecter handle_osd_map decoding full epoch 7 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.378+0000 7ffb797fa640 20 client.4143.objecter dump_active .. 0 homeless 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb8b174640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb8b174640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb8b174640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb840a3230 con 0x7ffb8409b050 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffb80017020 con 0x7ffb8409b050 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.382+0000 7ffb797fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:44.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.454+0000 7ffb8b174640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "4a264646-336c-4e1e-97e0-bbc5e144e545", "id": 5}] 2024-09-17T16:27:44.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.454+0000 7ffb8b174640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "4a264646-336c-4e1e-97e0-bbc5e144e545", "id": 5}] 2024-09-17T16:27:44.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.454+0000 7ffb8b174640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:27:44.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.454+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4a264646-336c-4e1e-97e0-bbc5e144e545", "id": 5} v 0) -- 0x7ffb8406d7a0 con 0x7ffb8409b050 2024-09-17T16:27:44.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.462+0000 7ffb797fa640 1 -- 172.21.15.160:0/888192806 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a264646-336c-4e1e-97e0-bbc5e144e545", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ffb80003620 con 0x7ffb8409b050 2024-09-17T16:27:44.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.462+0000 7ffb797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "4a264646-336c-4e1e-97e0-bbc5e144e545", "id": 5}] 2024-09-17T16:27:44.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.462+0000 7ffb797fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:44.470 INFO:teuthology.orchestra.run.smithi160.stdout:5 2024-09-17T16:27:44.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 10 librados: watch_flush enter 2024-09-17T16:27:44.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 10 librados: watch_flush exit 2024-09-17T16:27:44.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 20 client.4143.objecter shutdown clearing up homeless session... 2024-09-17T16:27:44.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 10 client.4143.objecter successfully canceled tick 2024-09-17T16:27:44.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 10 monclient: shutdown 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 msgr2=0x7ffb8409a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000e840 tx=0x7ffb8000ed00 comp rx=0 tx=0).stop 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 shutdown_connections 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb840a3770 0x7ffb84072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb840a2e50 0x7ffb8409a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 --2- 172.21.15.160:0/888192806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8409b050 0x7ffb8409a2a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 >> 172.21.15.160:0/888192806 conn(0x7ffb840a8620 msgr2=0x7ffb840a6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 shutdown_connections 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 -- 172.21.15.160:0/888192806 wait complete. 2024-09-17T16:27:44.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-09-17T16:27:44.466+0000 7ffb8b174640 1 librados: shutdown 2024-09-17T16:27:44.486 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T16:27:44.486 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T16:27:44.490 INFO:tasks.ceph.osd.0:Started 2024-09-17T16:27:44.491 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T16:27:44.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T16:27:44.493 INFO:tasks.ceph.osd.1:Started 2024-09-17T16:27:44.494 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T16:27:44.494 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T16:27:44.497 INFO:tasks.ceph.osd.2:Started 2024-09-17T16:27:44.497 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T16:27:44.497 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T16:27:44.499 INFO:tasks.ceph.osd.3:Started 2024-09-17T16:27:44.499 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T16:27:44.499 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T16:27:44.501 INFO:tasks.ceph.osd.4:Started 2024-09-17T16:27:44.501 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T16:27:44.501 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T16:27:44.503 INFO:tasks.ceph.osd.5:Started 2024-09-17T16:27:44.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: get_monmap_and_config 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: monmap: 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.624 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:44.621570+0000 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:44.621570+0000 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.625 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:44.626 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 Processor -- start 2024-09-17T16:27:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 -- start start 2024-09-17T16:27:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: init 2024-09-17T16:27:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient: _add_conns ranks=[3,7,4,6,0,10,1,5,9,2,8] 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f338c09b510 0x7f338c09b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-g con 0x7f338c09b510 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a25a0 0x7f338c0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-i con 0x7f338c0a25a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-c con 0x7f338c0a42c0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f338c09f3b0 con 0x7f338c09b510 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f338c0a18e0 con 0x7f338c0a42c0 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f338c0a1bc0 con 0x7f338c0a25a0 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.616+0000 7f3393ae3640 10 monclient(hunting): authenticate will time out at 947.887977s 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:58926/0 (socket says 172.21.15.105:58926) 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 -- 172.21.15.105:0/2932150275 learned_addr learned my addr 172.21.15.105:0/2932150275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): get_auth_request con 0x7f338c0a42c0 auth_method 0 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a25a0 0x7f338c0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): get_auth_request con 0x7f338c0a25a0 auth_method 0 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339187f640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f338c09b510 0x7f338c09b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339187f640 10 monclient(hunting): get_auth_request con 0x7f338c09b510 auth_method 0 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339187f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339187f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339187f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient(hunting): handle_auth_done global_id 4126 payload 274 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 -- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a25a0 msgr2=0x7f338c0a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a25a0 0x7f338c0a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 -- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f338c09b510 msgr2=0x7f338c09b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f338c09b510 0x7f338c09b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 monclient: found mon.noname-c 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 -- 172.21.15.105:0/2932150275 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c0a5c20 con 0x7f338c0a42c0 2024-09-17T16:27:44.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3392080640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3384009e80 tx=0x7f338401c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9119c8bf23302497 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 1 -- 172.21.15.105:0/2932150275 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33840090d0 con 0x7f338c0a42c0 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: dump: 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:44.646 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 1 -- 172.21.15.105:0/2932150275 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33840a6970 con 0x7f338c0a42c0 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 1 -- 172.21.15.105:0/2932150275 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33840af480 con 0x7f338c0a42c0 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 5 monclient: authenticate success, global_id 4126 2024-09-17T16:27:44.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 4 set_mon_vals no callback set 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f339087d640 10 monclient: dump: 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.649 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 10 monclient: shutdown 2024-09-17T16:27:44.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 -- 172.21.15.105:0/2932150275 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 msgr2=0x7f338c0a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3384009e80 tx=0x7f338401c910 comp rx=0 tx=0).stop 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 -- 172.21.15.105:0/2932150275 shutdown_connections 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f338c0a42c0 0x7f338c0a4700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a25a0 0x7f338c0a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2932150275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f338c09b510 0x7f338c09b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 -- 172.21.15.105:0/2932150275 >> 172.21.15.105:0/2932150275 conn(0x7f338c0a4e70 msgr2=0x7f338c0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 -- 172.21.15.105:0/2932150275 shutdown_connections 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.620+0000 7f3393ae3640 1 -- 172.21.15.105:0/2932150275 wait complete. 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: build_initial_monmap 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: monmap: 2024-09-17T16:27:44.651 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:44.652 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 librados: starting msgr at 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 librados: starting objecter 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 Processor -- start 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- start start 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 librados: setting wanted keys 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 librados: calling monclient init 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: init 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _add_conns ranks=[5,1,7,3,8,2,6,0,4] 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 0x7f338c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-h con 0x7f338c09b510 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f338c0a25a0 0x7f338c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-f con 0x7f338c0a25a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): picked mon.noname-i con 0x7f338c0a42c0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f338c140140 con 0x7f338c0a25a0 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f338c09c040 con 0x7f338c09b510 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f338c1402c0 con 0x7f338c0a42c0 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient(hunting): authenticate will time out at 947.893372s 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 0x7f338c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f338c0a25a0 0x7f338c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:53382/0 (socket says 172.21.15.105:53382) 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 -- 172.21.15.105:0/2144937416 learned_addr learned my addr 172.21.15.105:0/2144937416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): get_auth_request con 0x7f338c0a42c0 auth_method 0 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): get_auth_request con 0x7f338c09b510 auth_method 0 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): get_auth_request con 0x7f338c0a25a0 auth_method 0 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339107e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:44.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient(hunting): handle_auth_done global_id 4210 payload 824 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 -- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 msgr2=0x7f338c073db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 0x7f338c073db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 -- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f338c0a25a0 msgr2=0x7f338c071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f338c0a25a0 0x7f338c071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient: _finish_hunting 0 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 monclient: found mon.noname-i 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 -- 172.21.15.105:0/2144937416 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c08f8c0 con 0x7f338c0a42c0 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f339187f640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 0x7f338c073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3392080640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3384009040 tx=0x7f33840196b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 client.?.objecter ms_handle_connect 0x7f338c0a42c0 2024-09-17T16:27:44.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33840be070 con 0x7f338c0a42c0 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: dump: 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.658 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: _finish_auth 0 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: _check_auth_tickets 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:44.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33840a6b20 con 0x7f338c0a42c0 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3384003e10 con 0x7f338c0a42c0 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3383fff640 4 set_mon_vals no callback set 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 monclient: dump: 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:44.660 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 5 monclient: authenticate success, global_id 4210 2024-09-17T16:27:44.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _renew_subs 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c0a52e0 con 0x7f338c0a42c0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 client.4210.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _renew_subs 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c08f4a0 con 0x7f338c0a42c0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 librados: init done 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33840a6500 con 0x7f338c0a42c0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f33840b5030 con 0x7f338c0a42c0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 10 client.4210.objecter ms_dispatch 0x7f338c000ba0 osd_map(8..8 src has 1..8) 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 3 client.4210.objecter handle_osd_map got epochs [8,8] > 0 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 3 client.4210.objecter handle_osd_map decoding full epoch 8 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f33827fc640 20 client.4210.objecter dump_active .. 0 homeless 2024-09-17T16:27:44.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.624+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338c098d80 con 0x7f338c0a42c0 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.628+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f338c0a52e0 con 0x7f338c0a42c0 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.628+0000 7f33827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:44.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.628+0000 7f33827fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:44.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.688+0000 7f3393ae3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:44.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.688+0000 7f3393ae3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:44.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.688+0000 7f3393ae3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:27:44.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.688+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f338c0a3d80 con 0x7f338c0a42c0 2024-09-17T16:27:44.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f33827fc640 1 -- 172.21.15.105:0/2144937416 <== mon.7 v2:172.21.15.160:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7f338c08f8c0 con 0x7f338c0a42c0 2024-09-17T16:27:44.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f33827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:44.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f33827fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:44.699 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:27:44.699 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:27:44.458290+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:27:44.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 10 librados: watch_flush enter 2024-09-17T16:27:44.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 10 librados: watch_flush exit 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 20 client.4210.objecter shutdown clearing up homeless session... 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 10 client.4210.objecter successfully canceled tick 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 10 monclient: shutdown 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 msgr2=0x7f338c071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3384009040 tx=0x7f33840196b0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 shutdown_connections 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f338c0a42c0 0x7f338c071e50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f338c0a25a0 0x7f338c071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 --2- 172.21.15.105:0/2144937416 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f338c09b510 0x7f338c073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 >> 172.21.15.105:0/2144937416 conn(0x7f338c0a4e70 msgr2=0x7f338c099fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 shutdown_connections 2024-09-17T16:27:44.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 -- 172.21.15.105:0/2144937416 wait complete. 2024-09-17T16:27:44.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:44.692+0000 7f3393ae3640 1 librados: shutdown 2024-09-17T16:27:44.715 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T16:27:44.715 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T16:27:45.321 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.432+0000 7f87eec94640 10 monclient: get_monmap_and_config 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.432+0000 7f87eec94640 10 monclient: build_initial_monmap 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.432+0000 7f87eec94640 10 monclient: monmap: 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:45.441 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:45.438891+0000 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:45.438891+0000 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:45.442 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:45.443 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:45.444 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 Processor -- start 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 -- start start 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient: init 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient: _add_conns ranks=[9,4,0,10,5,2,3,8,7,6,1] 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87e80a46c0 0x7f87e80a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-a con 0x7f87e80a46c0 addr v2:172.21.15.105:3300/0 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-c con 0x7f87e80a4aa0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87e8139870 addr v1:172.21.15.105:6789/0 2024-09-17T16:27:45.454 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87e806f0d0 con 0x7f87e8139870 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f87e806fb50 con 0x7f87e80a46c0 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f87e80a1210 con 0x7f87e80a4aa0 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eec94640 10 monclient(hunting): authenticate will time out at 948.705237s 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87e80a46c0 0x7f87e80a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:58938/0 (socket says 172.21.15.105:58938) 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 -- 172.21.15.105:0/2002694056 learned_addr learned my addr 172.21.15.105:0/2002694056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): get_auth_request con 0x7f87e80a4aa0 auth_method 0 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:45.455 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): get_auth_request con 0x7f87e80a46c0 auth_method 0 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient(hunting): handle_auth_done global_id 4135 payload 274 2024-09-17T16:27:45.456 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 -- 172.21.15.105:0/2002694056 >> v2:172.21.15.105:3300/0 conn(0x7f87e80a46c0 msgr2=0x7f87e80a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 --2- 172.21.15.105:0/2002694056 >> v2:172.21.15.105:3300/0 conn(0x7f87e80a46c0 0x7f87e80a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 -- 172.21.15.105:0/2002694056 >> v1:172.21.15.105:6789/0 conn(0x7f87e8139870 legacy=0x7f87e8139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 monclient: found mon.noname-c 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.436+0000 7f87e7fff640 1 -- 172.21.15.105:0/2002694056 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e806f510 con 0x7f87e80a4aa0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e7fff640 1 --2- 172.21.15.105:0/2002694056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009870 tx=0x7f87dc01c560 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a135b81996bef66c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 1 -- 172.21.15.105:0/2002694056 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87dc0a7070 con 0x7f87e80a4aa0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: dump: 2024-09-17T16:27:45.457 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:45.458 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: _finish_auth 0 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 1 -- 172.21.15.105:0/2002694056 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc0b0070 con 0x7f87e80a4aa0 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 1 -- 172.21.15.105:0/2002694056 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87dc0abd50 con 0x7f87e80a4aa0 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 5 monclient: authenticate success, global_id 4135 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 4 set_mon_vals no callback set 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e77fe640 10 monclient: dump: 2024-09-17T16:27:45.459 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:45.460 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: shutdown 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2002694056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 msgr2=0x7f87e80a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- 172.21.15.105:0/2002694056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009870 tx=0x7f87dc01c560 comp rx=0 tx=0).stop 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2002694056 shutdown_connections 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- 172.21.15.105:0/2002694056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e80a4aa0 0x7f87e80a4ea0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- 172.21.15.105:0/2002694056 >> v2:172.21.15.105:3300/0 conn(0x7f87e80a46c0 0x7f87e80a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2002694056 >> 172.21.15.105:0/2002694056 conn(0x7f87e80a9310 msgr2=0x7f87e80a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2002694056 shutdown_connections 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2002694056 wait complete. 2024-09-17T16:27:45.461 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: build_initial_monmap 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: monmap: 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:created 0.000000 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:45.462 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 librados: starting msgr at 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 librados: starting objecter 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 Processor -- start 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- start start 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 librados: setting wanted keys 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 librados: calling monclient init 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: init 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _add_conns ranks=[2,5,4,1,0,7,6,3,8] 2024-09-17T16:27:45.463 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-b con 0x7f87e80a46c0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f87e80a4aa0 0x7f87e81d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-h con 0x7f87e80a4aa0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 0x7f87e81d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): picked mon.noname-c con 0x7f87e8139870 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f87e8144680 con 0x7f87e80a46c0 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f87e8144500 con 0x7f87e8139870 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f87e8144800 con 0x7f87e80a4aa0 2024-09-17T16:27:45.464 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient(hunting): authenticate will time out at 948.710291s 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 0x7f87e81d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43978/0 (socket says 172.21.15.105:43978) 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 -- 172.21.15.105:0/2379150075 learned_addr learned my addr 172.21.15.105:0/2379150075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 0x7f87e81d2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:58944/0 (socket says 172.21.15.105:58944) 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): get_auth_request con 0x7f87e80a46c0 auth_method 0 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): get_auth_request con 0x7f87e8139870 auth_method 0 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:45.465 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87ed231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient(hunting): handle_auth_done global_id 4169 payload 824 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 -- 172.21.15.105:0/2379150075 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f87e80a4aa0 msgr2=0x7f87e81d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f87e80a4aa0 0x7f87e81d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 -- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 msgr2=0x7f87e81d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 0x7f87e81d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient: _finish_hunting 0 2024-09-17T16:27:45.466 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 monclient: found mon.noname-b 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 -- 172.21.15.105:0/2379150075 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8141280 con 0x7f87e80a46c0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eca30640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e930 tx=0x7f87d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 client.?.objecter ms_handle_connect 0x7f87e80a46c0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87d8017070 con 0x7f87e80a46c0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: dump: 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.467 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:45.468 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: _finish_auth 0 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d8013470 con 0x7f87e80a46c0 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87d8013a90 con 0x7f87e80a46c0 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 monclient: dump: 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.469 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e6ffd640 4 set_mon_vals no callback set 2024-09-17T16:27:45.470 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 5 monclient: authenticate success, global_id 4169 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _renew_subs 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e80a85b0 con 0x7f87e80a46c0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 client.4169.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _renew_subs 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e8140410 con 0x7f87e80a46c0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87eec94640 1 librados: init done 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87d801a070 con 0x7f87e80a46c0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f87d800f040 con 0x7f87e80a46c0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 10 client.4169.objecter ms_dispatch 0x7f87e8000ba0 osd_map(8..8 src has 1..8) 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 3 client.4169.objecter handle_osd_map got epochs [8,8] > 0 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 3 client.4169.objecter handle_osd_map decoding full epoch 8 2024-09-17T16:27:45.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.440+0000 7f87e57fa640 20 client.4169.objecter dump_active .. 0 homeless 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87eec94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87eec94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87eec94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e80a19f0 con 0x7f87e80a46c0 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f87d8021070 con 0x7f87e80a46c0 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:45.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.444+0000 7f87e57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:45.511 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:45.512 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:45.512 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:27:45.512 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87e813ef20 con 0x7f87e80a46c0 2024-09-17T16:27:45.512 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87e57fa640 1 -- 172.21.15.105:0/2379150075 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7f87d800ca00 con 0x7f87e80a46c0 2024-09-17T16:27:45.513 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:45.513 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87e57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:45.513 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-17T16:27:45.513 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":8,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:27:44.458290+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:27:45.514 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 librados: watch_flush enter 2024-09-17T16:27:45.514 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 librados: watch_flush exit 2024-09-17T16:27:45.514 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 20 client.4169.objecter shutdown clearing up homeless session... 2024-09-17T16:27:45.514 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 client.4169.objecter successfully canceled tick 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 10 monclient: shutdown 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 msgr2=0x7f87e81d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87d800e930 tx=0x7f87d800edf0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 shutdown_connections 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f87e8139870 0x7f87e81d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f87e80a4aa0 0x7f87e81d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 --2- 172.21.15.105:0/2379150075 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87e80a46c0 0x7f87e81d1710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 >> 172.21.15.105:0/2379150075 conn(0x7f87e80a9310 msgr2=0x7f87e809f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 shutdown_connections 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.504+0000 7f87eec94640 1 -- 172.21.15.105:0/2379150075 wait complete. 2024-09-17T16:27:45.515 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:45.508+0000 7f87eec94640 1 librados: shutdown 2024-09-17T16:27:45.528 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T16:27:45.599 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:27:45.592+0000 7f51f83487c0 -1 Falling back to public interface 2024-09-17T16:27:45.643 INFO:tasks.ceph.osd.5.smithi160.stderr:2024-09-17T16:27:45.638+0000 7f81275897c0 -1 Falling back to public interface 2024-09-17T16:27:45.672 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-17T16:27:45.664+0000 7f931494e7c0 -1 Falling back to public interface 2024-09-17T16:27:45.809 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T16:27:45.810 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T16:27:45.810 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2024-09-17T16:27:45.848 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-17T16:27:45.840+0000 7f57598827c0 -1 Falling back to public interface 2024-09-17T16:27:45.884 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-09-17T16:27:45.878+0000 7f90f4e627c0 -1 Falling back to public interface 2024-09-17T16:27:46.140 INFO:tasks.ceph.osd.4.smithi160.stderr:2024-09-17T16:27:46.134+0000 7f59146fd7c0 -1 Falling back to public interface 2024-09-17T16:27:48.792 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.786+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-09-17T16:27:48.792 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.786+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T16:27:48.797 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.790+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-09-17T16:27:48.801 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.794+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-09-17T16:27:48.801 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.794+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-09-17T16:27:48.802 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.794+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-09-17T16:27:48.802 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.798+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T16:27:48.802 INFO:tasks.ceph.mgr.x.smithi160.stderr:2024-09-17T16:27:48.798+0000 7f1f35758640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-09-17T16:27:49.585 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:27:49.576+0000 7f51f83487c0 -1 osd.1 0 log_to_monitors true 2024-09-17T16:27:49.690 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-17T16:27:49.684+0000 7f931494e7c0 -1 osd.2 0 log_to_monitors true 2024-09-17T16:27:49.752 INFO:tasks.ceph.osd.5.smithi160.stderr:2024-09-17T16:27:49.746+0000 7f81275897c0 -1 osd.5 0 log_to_monitors true 2024-09-17T16:27:49.957 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-09-17T16:27:49.950+0000 7f90f4e627c0 -1 osd.3 0 log_to_monitors true 2024-09-17T16:27:50.118 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-17T16:27:50.112+0000 7f57598827c0 -1 osd.0 0 log_to_monitors true 2024-09-17T16:27:50.623 INFO:tasks.ceph.osd.4.smithi160.stderr:2024-09-17T16:27:50.618+0000 7f59146fd7c0 -1 osd.4 0 log_to_monitors true 2024-09-17T16:27:50.854 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-17T16:27:50.848+0000 7f93108f7640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T16:27:50.889 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-17T16:27:50.880+0000 7f930c731640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:51.638 INFO:tasks.ceph.osd.5.smithi160.stderr:2024-09-17T16:27:51.634+0000 7f8123d44640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T16:27:51.651 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:27:51.644+0000 7f51f42f1640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T16:27:51.665 INFO:tasks.ceph.osd.5.smithi160.stderr:2024-09-17T16:27:51.658+0000 7f811b364640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:51.682 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:27:51.676+0000 7f51ec123640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:51.853 INFO:tasks.ceph.osd.4.smithi160.stderr:2024-09-17T16:27:51.846+0000 7f59106a6640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T16:27:51.855 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-09-17T16:27:51.850+0000 7f90f0e0b640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T16:27:51.859 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-17T16:27:51.852+0000 7f575582b640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T16:27:51.879 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-17T16:27:51.872+0000 7f5751665640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:51.884 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-09-17T16:27:51.878+0000 7f90ecc45640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:51.884 INFO:tasks.ceph.osd.4.smithi160.stderr:2024-09-17T16:27:51.878+0000 7f590c4e0640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T16:27:52.134 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T16:27:52.256 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 10 monclient: get_monmap_and_config 2024-09-17T16:27:52.256 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 10 monclient: build_initial_monmap 2024-09-17T16:27:52.256 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 10 monclient: monmap: 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:52.254157+0000 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:52.254157+0000 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:52.257 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:52.258 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:52.258 INFO:teuthology.misc.health.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:52.258 INFO:teuthology.misc.health.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:52.258 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 1 Processor -- start 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 1 -- start start 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.248+0000 7f8588d7f640 10 monclient: init 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient: _add_conns ranks=[8,5,6,9,0,7,1,3,2,10,4] 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 0x7f85840a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-e con 0x7f85840a7950 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f85840707d0 0x7f85840a8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-h con 0x7f85840707d0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:27:52.267 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-d con 0x7f8584139700 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8584072340 con 0x7f85840707d0 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8584072590 con 0x7f8584139700 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8584070c50 con 0x7f85840a7950 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient(hunting): authenticate will time out at 955.520193s 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 0x7f85840a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.268 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:52266/0 (socket says 172.21.15.105:52266) 2024-09-17T16:27:52.269 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 0x7f85840a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:55078/0 (socket says 172.21.15.105:55078) 2024-09-17T16:27:52.269 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 -- 172.21.15.105:0/771305779 learned_addr learned my addr 172.21.15.105:0/771305779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:52.269 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): get_auth_request con 0x7f8584139700 auth_method 0 2024-09-17T16:27:52.269 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.269 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.270 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.270 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): get_auth_request con 0x7f85840a7950 auth_method 0 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:52.271 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8581d9b640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f85840707d0 0x7f85840a8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8581d9b640 10 monclient(hunting): get_auth_request con 0x7f85840707d0 auth_method 0 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8581d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8581d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8581d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient(hunting): handle_auth_done global_id 4164 payload 274 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 -- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 msgr2=0x7f85840a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 0x7f85840a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 -- 172.21.15.105:0/771305779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f85840707d0 msgr2=0x7f85840a8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f85840707d0 0x7f85840a8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 monclient: found mon.noname-d 2024-09-17T16:27:52.273 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 -- 172.21.15.105:0/771305779 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584074880 con 0x7f8584139700 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8582d9d640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f85740098a0 tx=0x7f857401c410 comp rx=0 tx=0).ready entity=mon.6 client_cookie=578e28c30df4f1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 1 -- 172.21.15.105:0/771305779 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85740b1070 con 0x7f8584139700 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: dump: 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.274 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:52.275 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: _finish_auth 0 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: _check_auth_tickets 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 5 monclient: authenticate success, global_id 4164 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 1 -- 172.21.15.105:0/771305779 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85740aa7b0 con 0x7f8584139700 2024-09-17T16:27:52.276 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 4 set_mon_vals no callback set 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 1 -- 172.21.15.105:0/771305779 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85740aae50 con 0x7f8584139700 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f858159a640 10 monclient: dump: 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:52.277 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 10 monclient: shutdown 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 msgr2=0x7f8584139b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f85740098a0 tx=0x7f857401c410 comp rx=0 tx=0).stop 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- 172.21.15.105:0/771305779 shutdown_connections 2024-09-17T16:27:52.278 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8584139700 0x7f8584139b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f85840707d0 0x7f85840a8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 --2- 172.21.15.105:0/771305779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f85840a7950 0x7f85840a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- 172.21.15.105:0/771305779 >> 172.21.15.105:0/771305779 conn(0x7f85840ac730 msgr2=0x7f85840a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- 172.21.15.105:0/771305779 shutdown_connections 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.252+0000 7f8588d7f640 1 -- 172.21.15.105:0/771305779 wait complete. 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: build_initial_monmap 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: monmap: 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:created 0.000000 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:52.279 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 librados: starting msgr at 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 librados: starting objecter 2024-09-17T16:27:52.280 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 Processor -- start 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- start start 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 librados: setting wanted keys 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 librados: calling monclient init 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: init 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _add_conns ranks=[6,1,7,8,0,4,2,3,5] 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-d con 0x7f85840a7950 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8584139700 0x7f85841d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-f con 0x7f8584139700 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f858413d5b0 0x7f858413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.281 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): picked mon.noname-i con 0x7f858413d5b0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f85840730b0 con 0x7f8584139700 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8584072f30 con 0x7f85840a7950 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8584143380 con 0x7f858413d5b0 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:52.282 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient(hunting): authenticate will time out at 955.525374s 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:52272/0 (socket says 172.21.15.105:52272) 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 -- 172.21.15.105:0/2353927418 learned_addr learned my addr 172.21.15.105:0/2353927418 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): get_auth_request con 0x7f85840a7950 auth_method 0 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.283 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8582d9d640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f858413d5b0 0x7f858413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8581d9b640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8584139700 0x7f85841d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8581d9b640 10 monclient(hunting): get_auth_request con 0x7f8584139700 auth_method 0 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8581d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8581d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8581d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8582d9d640 10 monclient(hunting): get_auth_request con 0x7f858413d5b0 auth_method 0 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8582d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:52.284 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8582d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8582d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient(hunting): handle_auth_done global_id 4173 payload 824 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 -- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f858413d5b0 msgr2=0x7f858413d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f858413d5b0 0x7f858413d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 -- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8584139700 msgr2=0x7f85841d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8584139700 0x7f85841d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient: _finish_hunting 0 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 monclient: found mon.noname-d 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 -- 172.21.15.105:0/2353927418 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584123d70 con 0x7f85840a7950 2024-09-17T16:27:52.285 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f858259c640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f857c009db0 tx=0x7f857c00c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 client.?.objecter ms_handle_connect 0x7f85840a7950 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f857c018020 con 0x7f85840a7950 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: dump: 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.286 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: _finish_auth 0 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: _check_auth_tickets 2024-09-17T16:27:52.287 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857c00cdb0 con 0x7f85840a7950 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f857c014d60 con 0x7f85840a7950 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8580d99640 4 set_mon_vals no callback set 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 monclient: dump: 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:52.288 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 5 monclient: authenticate success, global_id 4173 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _renew_subs 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.289 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858413f4a0 con 0x7f85840a7950 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 client.4173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _renew_subs 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f858413f000 con 0x7f85840a7950 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 librados: init done 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858406a850 con 0x7f85840a7950 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 4) ==== 48888+0+0 (secure 0 0 0) 0x7f857c01b020 con 0x7f85840a7950 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 0x7f8564040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(11..11 src has 1..11) ==== 2767+0+0 (secure 0 0 0) 0x7f857c04f990 con 0x7f85840a7950 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 10 client.4173.objecter ms_dispatch 0x7f8584000ba0 osd_map(11..11 src has 1..11) 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 3 client.4173.objecter handle_osd_map got epochs [11,11] > 0 2024-09-17T16:27:52.290 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 3 client.4173.objecter handle_osd_map decoding full epoch 11 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.256+0000 7f857affd640 20 client.4173.objecter dump_active .. 0 homeless 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.260+0000 7f8581d9b640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 0x7f8564040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.260+0000 7f8581d9b640 10 monclient: get_auth_request con 0x7f856403e4c0 auth_method 0 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.260+0000 7f857affd640 10 client.4173.objecter ms_handle_connect 0x7f856403e4c0 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.260+0000 7f8581d9b640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 0x7f8564040980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8584070e60 tx=0x7f856c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.264+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f857c010040 con 0x7f85840a7950 2024-09-17T16:27:52.291 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.264+0000 7f857affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:52.292 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.264+0000 7f857affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:52.444 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f8588d7f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:52.444 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f8588d7f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:52.444 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f8588d7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:52.445 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8584139b20 con 0x7f85840a7950 2024-09-17T16:27:52.445 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f857affd640 1 -- 172.21.15.105:0/2353927418 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+7087 (secure 0 0 0) 0x7f857c01cc90 con 0x7f85840a7950 2024-09-17T16:27:52.445 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f857affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:52.445 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.436+0000 7f857affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:52.446 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-17T16:27:52.446 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":11,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:27:51.837855+0000","last_up_change":"2024-09-17T16:27:51.837855+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 10 librados: watch_flush enter 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 10 librados: watch_flush exit 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 20 client.4173.objecter shutdown clearing up homeless session... 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 10 client.4173.objecter successfully canceled tick 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 msgr2=0x7f8564040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:52.448 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 0x7f8564040980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8584070e60 tx=0x7f856c008040 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 10 monclient: shutdown 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 msgr2=0x7f85841d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f857c009db0 tx=0x7f857c00c490 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 shutdown_connections 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f856403e4c0 0x7f8564040980 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f858413d5b0 0x7f858413d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8584139700 0x7f85841d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 --2- 172.21.15.105:0/2353927418 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f85840a7950 0x7f85841d16b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 >> 172.21.15.105:0/2353927418 conn(0x7f85840ac730 msgr2=0x7f85840a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 shutdown_connections 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 -- 172.21.15.105:0/2353927418 wait complete. 2024-09-17T16:27:52.449 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:52.440+0000 7f8588d7f640 1 librados: shutdown 2024-09-17T16:27:52.463 DEBUG:teuthology.misc:3 of 6 OSDs are up 2024-09-17T16:27:59.068 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T16:27:59.189 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.180+0000 7f2a3aae7640 10 monclient: get_monmap_and_config 2024-09-17T16:27:59.189 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.180+0000 7f2a3aae7640 10 monclient: build_initial_monmap 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.180+0000 7f2a3aae7640 10 monclient: monmap: 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:59.187206+0000 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:59.187206+0000 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:59.190 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:59.191 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:59.191 INFO:teuthology.misc.health.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:59.191 INFO:teuthology.misc.health.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:59.191 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.194 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 Processor -- start 2024-09-17T16:27:59.194 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 -- start start 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient: init 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient: _add_conns ranks=[4,1,7,9,10,0,6,3,5,8,2] 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-c con 0x7f2a340a7950 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2a340707d0 addr v1:172.21.15.160:6789/0 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a34139e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-i con 0x7f2a340a8270 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a34074880 con 0x7f2a340707d0 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2a34072340 con 0x7f2a340a7950 2024-09-17T16:27:59.195 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2a34072590 con 0x7f2a340a8270 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient(hunting): authenticate will time out at 962.452942s 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57498/0 (socket says 172.21.15.105:57498) 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 -- 172.21.15.105:0/2042898722 learned_addr learned my addr 172.21.15.105:0/2042898722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): get_auth_request con 0x7f2a340a7950 auth_method 0 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.196 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a39084640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a34139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient(hunting): handle_auth_done global_id 4189 payload 274 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 -- 172.21.15.105:0/2042898722 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 msgr2=0x7f2a34139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a34139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 -- 172.21.15.105:0/2042898722 >> v1:172.21.15.160:6789/0 conn(0x7f2a340707d0 legacy=0x7f2a34139700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient: _finish_hunting 0 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 monclient: found mon.noname-c 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:27:59.197 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 -- 172.21.15.105:0/2042898722 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a34071cf0 con 0x7f2a340a7950 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a39084640 10 monclient: get_auth_request con 0x7f2a340a8270 auth_method 0 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a39084640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a34139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a38883640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098d0 tx=0x7f2a2801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9031f9e441548901 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 1 -- 172.21.15.105:0/2042898722 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a280b0070 con 0x7f2a340a7950 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: dump: 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.198 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: _finish_auth 0 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: _check_auth_tickets 2024-09-17T16:27:59.199 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 1 -- 172.21.15.105:0/2042898722 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a280aa810 con 0x7f2a340a7950 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 5 monclient: authenticate success, global_id 4189 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a3aae7640 4 set_mon_vals no callback set 2024-09-17T16:27:59.200 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 1 -- 172.21.15.105:0/2042898722 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a280a9300 con 0x7f2a340a7950 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.184+0000 7f2a337fe640 10 monclient: dump: 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.201 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: shutdown 2024-09-17T16:27:59.202 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/2042898722 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 msgr2=0x7f2a340a7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a280098d0 tx=0x7f2a2801c440 comp rx=0 tx=0).stop 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/2042898722 shutdown_connections 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a34139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/2042898722 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a340a7950 0x7f2a340a7d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/2042898722 >> 172.21.15.105:0/2042898722 conn(0x7f2a340ac730 msgr2=0x7f2a340a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/2042898722 shutdown_connections 2024-09-17T16:27:59.203 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/2042898722 wait complete. 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: build_initial_monmap 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: monmap: 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:epoch 0 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:created 0.000000 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:59.208 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 librados: starting msgr at 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 librados: starting objecter 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 Processor -- start 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- start start 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 librados: setting wanted keys 2024-09-17T16:27:59.209 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 librados: calling monclient init 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: init 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _add_conns ranks=[6,7,4,2,5,0,8,3,1] 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-d con 0x7f2a340a7950 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a341d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-i con 0x7f2a340a8270 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a3413e170 0x7f2a3413e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3413e170 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2a3413a4d0 con 0x7f2a3413e170 2024-09-17T16:27:59.210 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2a3413a350 con 0x7f2a340a7950 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2a34144390 con 0x7f2a340a8270 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient(hunting): authenticate will time out at 962.458395s 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33828/0 (socket says 172.21.15.105:33828) 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 -- 172.21.15.105:0/1534488281 learned_addr learned my addr 172.21.15.105:0/1534488281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): get_auth_request con 0x7f2a340a7950 auth_method 0 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.211 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a3413e170 0x7f2a3413e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): get_auth_request con 0x7f2a3413e170 auth_method 0 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a33fff640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a341d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.212 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a39084640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a33fff640 10 monclient(hunting): get_auth_request con 0x7f2a340a8270 auth_method 0 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a33fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient(hunting): handle_auth_done global_id 4200 payload 824 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 -- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 msgr2=0x7f2a341d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a341d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 -- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a3413e170 msgr2=0x7f2a3413e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a3413e170 0x7f2a3413e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient: _finish_hunting 0 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 monclient: found mon.noname-d 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:59.213 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 -- 172.21.15.105:0/1534488281 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3413fb70 con 0x7f2a340a7950 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a38883640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a34122710 tx=0x7f2a280191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 client.?.objecter ms_handle_connect 0x7f2a340a7950 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a280b9070 con 0x7f2a340a7950 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: dump: 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.214 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: _finish_auth 0 2024-09-17T16:27:59.215 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: _check_auth_tickets 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 5 monclient: authenticate success, global_id 4200 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a280044b0 con 0x7f2a340a7950 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a280b0040 con 0x7f2a340a7950 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _renew_subs 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a34123920 con 0x7f2a340a7950 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a32ffd640 4 set_mon_vals no callback set 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 client.4200.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _renew_subs 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:59.216 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a340a5440 con 0x7f2a340a7950 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a3aae7640 1 librados: init done 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.188+0000 7f2a317fa640 10 monclient: dump: 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:epoch 1 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.217 INFO:teuthology.misc.health.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr: 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2a28004040 con 0x7f2a340a7950 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a3aae7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a3aae7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a3aae7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:59.218 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3406a850 con 0x7f2a340a7950 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 0x7f2a18040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(15..15 src has 1..15) ==== 3993+0+0 (secure 0 0 0) 0x7f2a280e88c0 con 0x7f2a340a7950 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 10 client.4200.objecter ms_dispatch 0x7f2a34000ba0 osd_map(15..15 src has 1..15) 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 3 client.4200.objecter handle_osd_map got epochs [15,15] > 0 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 3 client.4200.objecter handle_osd_map decoding full epoch 15 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 20 client.4200.objecter dump_active .. 0 homeless 2024-09-17T16:27:59.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a33fff640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 0x7f2a18040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a33fff640 10 monclient: get_auth_request con 0x7f2a1803e550 auth_method 0 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a317fa640 10 client.4200.objecter ms_handle_connect 0x7f2a1803e550 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.192+0000 7f2a33fff640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 0x7f2a18040a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a3413f8d0 tx=0x7f2a20009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.196+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2a28016150 con 0x7f2a340a7950 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.196+0000 7f2a317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.196+0000 7f2a317fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:59.370 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:59.370 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:59.370 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:27:59.370 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a340a7d30 con 0x7f2a340a7950 2024-09-17T16:27:59.371 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a317fa640 1 -- 172.21.15.105:0/1534488281 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10521 (secure 0 0 0) 0x7f2a280bc360 con 0x7f2a340a7950 2024-09-17T16:27:59.372 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:27:59.372 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a317fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:27:59.372 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-17T16:27:59.372 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":15,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:27:55.922865+0000","last_up_change":"2024-09-17T16:27:52.867595+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T16:27:53.825252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6817","nonce":315040748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6819","nonce":315040748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6823","nonce":315040748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6821","nonce":315040748}]},"public_addr":"172.21.15.105:6817/315040748","cluster_addr":"172.21.15.105:6819/315040748","heartbeat_back_addr":"172.21.15.105:6823/315040748","heartbeat_front_addr":"172.21.15.105:6821/315040748","state":["exists","up"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6809","nonce":3218035673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6811","nonce":3218035673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6815","nonce":3218035673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6813","nonce":3218035673}]},"public_addr":"172.21.15.160:6809/3218035673","cluster_addr":"172.21.15.160:6811/3218035673","heartbeat_back_addr":"172.21.15.160:6815/3218035673","heartbeat_front_addr":"172.21.15.160:6813/3218035673","state":["exists","up"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6817","nonce":639753971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6819","nonce":639753971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6823","nonce":639753971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6821","nonce":639753971}]},"public_addr":"172.21.15.160:6817/639753971","cluster_addr":"172.21.15.160:6819/639753971","heartbeat_back_addr":"172.21.15.160:6823/639753971","heartbeat_front_addr":"172.21.15.160:6821/639753971","state":["exists","up"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.073377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.536863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.709983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.944027+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.614775+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.703905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 librados: watch_flush enter 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 librados: watch_flush exit 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 20 client.4200.objecter shutdown clearing up homeless session... 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 client.4200.objecter successfully canceled tick 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 msgr2=0x7f2a18040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 0x7f2a18040a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a3413f8d0 tx=0x7f2a20009040 comp rx=0 tx=0).stop 2024-09-17T16:27:59.374 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 10 monclient: shutdown 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 msgr2=0x7f2a341d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.364+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a34122710 tx=0x7f2a280191e0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 shutdown_connections 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a1803e550 0x7f2a18040a10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2a3413e170 0x7f2a3413e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2a340a8270 0x7f2a341d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 --2- 172.21.15.105:0/1534488281 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2a340a7950 0x7f2a341d15d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 >> 172.21.15.105:0/1534488281 conn(0x7f2a340ac730 msgr2=0x7f2a341411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:59.375 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 shutdown_connections 2024-09-17T16:27:59.376 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 -- 172.21.15.105:0/1534488281 wait complete. 2024-09-17T16:27:59.376 INFO:teuthology.misc.health.smithi105.stderr:2024-09-17T16:27:59.368+0000 7f2a3aae7640 1 librados: shutdown 2024-09-17T16:27:59.388 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-17T16:27:59.388 INFO:tasks.ceph:Creating RBD pool 2024-09-17T16:27:59.389 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.488+0000 7f8e2f18c640 10 monclient: get_monmap_and_config 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.488+0000 7f8e2f18c640 10 monclient: build_initial_monmap 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.488+0000 7f8e2f18c640 10 monclient: monmap: 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:59.495585+0000 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:59.495585+0000 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:27:59.498 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:27:59.499 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 Processor -- start 2024-09-17T16:27:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 -- start start 2024-09-17T16:27:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient: init 2024-09-17T16:27:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient: _add_conns ranks=[7,9,8,5,3,1,0,6,2,4,10] 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 0x7f8e280a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-i con 0x7f8e280a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 0x7f8e280a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-a con 0x7f8e280a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-e con 0x7f8e281396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8e280a3fc0 con 0x7f8e280a4910 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8e280a63c0 con 0x7f8e280a68e0 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8e280a7740 con 0x7f8e281396a0 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2f18c640 10 monclient(hunting): authenticate will time out at 962.761397s 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 0x7f8e280a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34080/0 (socket says 172.21.15.105:34080) 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 -- 172.21.15.105:0/2608570765 learned_addr learned my addr 172.21.15.105:0/2608570765 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): get_auth_request con 0x7f8e280a4910 auth_method 0 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): get_auth_request con 0x7f8e281396a0 auth_method 0 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 0x7f8e280a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient(hunting): handle_auth_done global_id 4202 payload 274 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 -- 172.21.15.105:0/2608570765 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 msgr2=0x7f8e280a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 0x7f8e280a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 -- 172.21.15.105:0/2608570765 >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 msgr2=0x7f8e280a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 --2- 172.21.15.105:0/2608570765 >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 0x7f8e280a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient: _finish_hunting 0 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 monclient: found mon.noname-e 2024-09-17T16:27:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:59.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:27:59.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 -- 172.21.15.105:0/2608570765 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2806ebc0 con 0x7f8e281396a0 2024-09-17T16:27:59.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2cf28640 10 monclient: get_auth_request con 0x7f8e280a68e0 auth_method 0 2024-09-17T16:27:59.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e27fff640 1 --2- 172.21.15.105:0/2608570765 >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 0x7f8e280a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:27:59.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 0x7f8e280a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.492+0000 7f8e2d729640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009e50 tx=0x7f8e2001c8c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8d86b978d9dce4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 1 -- 172.21.15.105:0/2608570765 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e200090d0 con 0x7f8e281396a0 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: dump: 2024-09-17T16:27:59.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.513 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: _finish_auth 0 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: _check_auth_tickets 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 5 monclient: authenticate success, global_id 4202 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 1 -- 172.21.15.105:0/2608570765 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e200a5b90 con 0x7f8e281396a0 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: get_monmap_and_config success 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 4 set_mon_vals no callback set 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 1 -- 172.21.15.105:0/2608570765 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e200a5d80 con 0x7f8e281396a0 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:27:59.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e277fe640 10 monclient: dump: 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.515 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: shutdown 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/2608570765 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 msgr2=0x7f8e28139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009e50 tx=0x7f8e2001c8c0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/2608570765 shutdown_connections 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e281396a0 0x7f8e28139b50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/2608570765 >> v2:172.21.15.105:3300/0 conn(0x7f8e280a4910 0x7f8e280a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/2608570765 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e280a68e0 0x7f8e280a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/2608570765 >> 172.21.15.105:0/2608570765 conn(0x7f8e28068ba0 msgr2=0x7f8e28068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/2608570765 shutdown_connections 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/2608570765 wait complete. 2024-09-17T16:27:59.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: build_initial_monmap 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: monmap: 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:27:59.517 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 librados: starting msgr at 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 librados: starting objecter 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 Processor -- start 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- start start 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 librados: setting wanted keys 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 librados: calling monclient init 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: init 2024-09-17T16:27:59.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _reopen_session rank -1 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _add_conns ranks=[3,4,0,8,5,1,7,2,6] 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 0x7f8e2813d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-g con 0x7f8e280a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 0x7f8e2813dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-c con 0x7f8e280a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): picked mon.noname-a con 0x7f8e281396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2806e630 con 0x7f8e281396a0 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8e280a7af0 con 0x7f8e280a4910 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8e28143320 con 0x7f8e280a68e0 2024-09-17T16:27:59.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): _renew_subs 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient(hunting): authenticate will time out at 962.766250s 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 0x7f8e2813dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45112/0 (socket says 172.21.15.105:45112) 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 0x7f8e2813dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57520/0 (socket says 172.21.15.105:57520) 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 -- 172.21.15.105:0/3962448065 learned_addr learned my addr 172.21.15.105:0/3962448065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): get_auth_request con 0x7f8e281396a0 auth_method 0 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): get_auth_request con 0x7f8e280a68e0 auth_method 0 2024-09-17T16:27:59.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 0x7f8e2813d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient(hunting): handle_auth_done global_id 4203 payload 824 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 -- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 msgr2=0x7f8e2813dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 0x7f8e2813dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 -- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 msgr2=0x7f8e2813d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 0x7f8e2813d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient: _finish_hunting 0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 monclient: found mon.noname-a 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 -- 172.21.15.105:0/3962448065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e28123570 con 0x7f8e281396a0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2cf28640 10 monclient: get_auth_request con 0x7f8e280a4910 auth_method 0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 0x7f8e2813d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2d729640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009040 tx=0x7f8e20019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 client.?.objecter ms_handle_connect 0x7f8e281396a0 2024-09-17T16:27:59.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e200bc020 con 0x7f8e281396a0 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: dump: 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.523 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: _finish_auth 0 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: _check_auth_tickets 2024-09-17T16:27:59.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 5 monclient: authenticate success, global_id 4203 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e20009210 con 0x7f8e281396a0 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _renew_subs 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e280ab950 con 0x7f8e281396a0 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:27:59.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e20003bb0 con 0x7f8e281396a0 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 10 monclient: dump: 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:27:59.526 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 client.4203.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e26ffd640 4 set_mon_vals no callback set 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _renew_subs 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:27:59.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e280a9e70 con 0x7f8e281396a0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 librados: init done 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8e200bf020 con 0x7f8e281396a0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.496+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e28139b50 con 0x7f8e281396a0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 0x7f8e14040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3993+0+0 (secure 0 0 0) 0x7f8e200b3040 con 0x7f8e281396a0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 10 client.4203.objecter ms_dispatch 0x7f8e28000ba0 osd_map(15..15 src has 1..15) 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 3 client.4203.objecter handle_osd_map got epochs [15,15] > 0 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 3 client.4203.objecter handle_osd_map decoding full epoch 15 2024-09-17T16:27:59.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 20 client.4203.objecter dump_active .. 0 homeless 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 0x7f8e14040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e2cf28640 10 monclient: get_auth_request con 0x7f8e1403e540 auth_method 0 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e257fa640 10 client.4203.objecter ms_handle_connect 0x7f8e1403e540 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.500+0000 7f8e2cf28640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 0x7f8e14040a00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e18008930 tx=0x7f8e18008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.504+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8e200c6030 con 0x7f8e281396a0 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.504+0000 7f8e257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:27:59.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.504+0000 7f8e257fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:27:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.676+0000 7f8e2f18c640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-17T16:27:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.676+0000 7f8e2f18c640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-17T16:27:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.676+0000 7f8e2f18c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:27:59.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:27:59.676+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f8e28139480 con 0x7f8e281396a0 2024-09-17T16:28:00.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.428+0000 7f8e257fa640 1 -- 172.21.15.105:0/3962448065 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) ==== 109+0+0 (secure 0 0 0) 0x7f8e20016600 con 0x7f8e281396a0 2024-09-17T16:28:00.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.428+0000 7f8e257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-17T16:28:00.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.428+0000 7f8e257fa640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-17T16:28:00.437 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'rbd' created 2024-09-17T16:28:00.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 10 librados: watch_flush enter 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 10 librados: watch_flush exit 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 20 client.4203.objecter shutdown clearing up homeless session... 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 10 client.4203.objecter successfully canceled tick 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 msgr2=0x7f8e14040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 0x7f8e14040a00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e18008930 tx=0x7f8e18008040 comp rx=0 tx=0).stop 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 10 monclient: shutdown 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 msgr2=0x7f8e2813e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009040 tx=0x7f8e20019660 comp rx=0 tx=0).stop 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 shutdown_connections 2024-09-17T16:28:00.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e1403e540 0x7f8e14040a00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e281396a0 0x7f8e2813e260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8e280a68e0 0x7f8e2813dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 --2- 172.21.15.105:0/3962448065 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e280a4910 0x7f8e2813d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 >> 172.21.15.105:0/3962448065 conn(0x7f8e28068ba0 msgr2=0x7f8e280a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 shutdown_connections 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 -- 172.21.15.105:0/3962448065 wait complete. 2024-09-17T16:28:00.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.432+0000 7f8e2f18c640 1 librados: shutdown 2024-09-17T16:28:00.458 DEBUG:teuthology.orchestra.run.smithi105:> rbd --cluster ceph pool init rbd 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.504+0000 7f84373fd100 10 monclient: get_monmap_and_config 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.504+0000 7f84373fd100 10 monclient: build_initial_monmap 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.504+0000 7f84373fd100 10 monclient: monmap: 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:00.511674+0000 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:00.511674+0000 2024-09-17T16:28:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:00.516 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 Processor -- start 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 -- start start 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient: init 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient: _reopen_session rank -1 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient: _add_conns ranks=[3,9,2,4,5,10,7,8,0,6,1] 2024-09-17T16:28:00.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x556ad722d7a0 0x556ad722db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-g con 0x556ad722d7a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 --2- >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-a con 0x556ad7232d90 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-b con 0x556ad7236310 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x556ad7101160 con 0x556ad7232d90 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x556ad7111d30 con 0x556ad7236310 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x556ad7104f70 con 0x556ad722d7a0 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): _renew_subs 2024-09-17T16:28:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f84373fd100 10 monclient(hunting): authenticate will time out at 963.777618s 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59426/0 (socket says 172.21.15.105:59426) 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45116/0 (socket says 172.21.15.105:45116) 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 -- 172.21.15.105:0/4093699757 learned_addr learned my addr 172.21.15.105:0/4093699757 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): get_auth_request con 0x556ad7236310 auth_method 0 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): get_auth_request con 0x556ad7232d90 auth_method 0 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435f06640 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x556ad722d7a0 0x556ad722db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435f06640 10 monclient(hunting): get_auth_request con 0x556ad722d7a0 auth_method 0 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435f06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435f06640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435f06640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient(hunting): handle_auth_done global_id 4205 payload 274 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 -- 172.21.15.105:0/4093699757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x556ad722d7a0 msgr2=0x556ad722db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x556ad722d7a0 0x556ad722db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 -- 172.21.15.105:0/4093699757 >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 msgr2=0x556ad7235dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 --2- 172.21.15.105:0/4093699757 >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient: _finish_hunting 0 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 monclient: found mon.noname-b 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8437168640 1 -- 172.21.15.105:0/4093699757 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad707fc80 con 0x556ad7236310 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.508+0000 7f8435705640 1 --2- 172.21.15.105:0/4093699757 >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f842800a850 tx=0x7f842801d7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3ba434444566cc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 1 -- 172.21.15.105:0/4093699757 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84280ae070 con 0x556ad7236310 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: dump: 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:00.529 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: _finish_auth 0 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: _check_auth_tickets 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 1 -- 172.21.15.105:0/4093699757 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84280aa8c0 con 0x556ad7236310 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 5 monclient: authenticate success, global_id 4205 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: get_monmap_and_config success 2024-09-17T16:28:00.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 1 -- 172.21.15.105:0/4093699757 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84280a93b0 con 0x556ad7236310 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 4 set_mon_vals no callback set 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8434f04640 10 monclient: dump: 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:00.531 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: shutdown 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/4093699757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 msgr2=0x556ad7236790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f842800a850 tx=0x7f842801d7f0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/4093699757 shutdown_connections 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x556ad7236310 0x556ad7236790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- 172.21.15.105:0/4093699757 >> v2:172.21.15.105:3300/0 conn(0x556ad7232d90 0x556ad7235dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- 172.21.15.105:0/4093699757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x556ad722d7a0 0x556ad722db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/4093699757 >> 172.21.15.105:0/4093699757 conn(0x556ad7217b20 msgr2=0x556ad7217f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/4093699757 shutdown_connections 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/4093699757 wait complete. 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: build_initial_monmap 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: monmap: 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:00.533 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 librados: starting msgr at 2024-09-17T16:28:00.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 librados: starting objecter 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 Processor -- start 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- start start 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 librados: setting wanted keys 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 librados: calling monclient init 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: init 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _reopen_session rank -1 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _add_conns ranks=[8,0,6,7,3,4,1,2,5] 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 0x556ad72dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-e con 0x556ad722d7a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 0x556ad72dd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-a con 0x556ad7236310 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x556ad72b84a0 0x556ad72b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): picked mon.noname-d con 0x556ad72b84a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556ad7233f40 con 0x556ad7236310 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x556ad70a6a70 con 0x556ad72b84a0 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x556ad722e0c0 con 0x556ad722d7a0 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): _renew_subs 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient(hunting): authenticate will time out at 963.782597s 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 0x556ad72dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 0x556ad72dd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 0x556ad72dcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34086/0 (socket says 172.21.15.105:34086) 2024-09-17T16:28:00.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x556ad72b84a0 0x556ad72b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 0x556ad72dd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45118/0 (socket says 172.21.15.105:45118) 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 1 -- 172.21.15.105:0/1172645659 learned_addr learned my addr 172.21.15.105:0/1172645659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): get_auth_request con 0x556ad722d7a0 auth_method 0 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): get_auth_request con 0x556ad7236310 auth_method 0 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): get_auth_request con 0x556ad72b84a0 auth_method 0 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:00.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient(hunting): handle_auth_done global_id 4221 payload 824 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 -- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 msgr2=0x556ad72dcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 0x556ad72dcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 -- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x556ad72b84a0 msgr2=0x556ad72b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:00.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x556ad72b84a0 0x556ad72b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient: _finish_hunting 0 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 monclient: found mon.noname-a 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad72bc2f0 con 0x556ad7236310 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8437168640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x556ad72b84a0 0x556ad72b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435f06640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x556ad722d7a0 0x556ad72dcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8435705640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 0x556ad72dd4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x556ad721b870 tx=0x7f8420008c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 client.?.objecter ms_handle_connect 0x556ad7236310 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f842000dcf0 con 0x556ad7236310 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:00.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: dump: 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:00.540 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8420018070 con 0x556ad7236310 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 5 monclient: authenticate success, global_id 4221 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _renew_subs 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ad722f590 con 0x556ad7236310 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:00.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f842000de90 con 0x556ad7236310 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8427fff640 4 set_mon_vals no callback set 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f8425ffb640 10 monclient: dump: 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:00.542 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 client.4221.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _renew_subs 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad72b9f90 con 0x556ad7236310 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 1 librados: init done 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.512+0000 7f84373fd100 10 librados: wait_for_osdmap waiting 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8420012070 con 0x556ad7236310 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841403c820 0x7f841403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:00.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4335+0+0 (secure 0 0 0) 0x7f842004e8f0 con 0x556ad7236310 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_map(16..16 src has 1..16) 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map got epochs [16,16] > 0 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map decoding full epoch 16 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 20 client.4221.objecter dump_active .. 0 homeless 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f84373fd100 10 librados: wait_for_osdmap done waiting 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f84373fd100 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f84373fd100 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f84373fd100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x556ad72baaf0 con 0x556ad7236310 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8435f06640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841403c820 0x7f841403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8435f06640 10 monclient: get_auth_request con 0x7f841403c820 auth_method 0 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8435f06640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841403c820 0x7f841403ece0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f842c00ad50 tx=0x7f842c005c30 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:00.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:00.516+0000 7f8425ffb640 10 client.4221.objecter ms_handle_connect 0x7f841403c820 2024-09-17T16:28:01.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.432+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v17) ==== 141+0+0 (secure 0 0 0) 0x7f842001b370 con 0x556ad7236310 2024-09-17T16:28:01.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.432+0000 7f8425ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-17T16:28:01.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.432+0000 7f8425ffb640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-17T16:28:01.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.432+0000 7f84373fd100 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:01.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.432+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84280aa8c0 con 0x556ad7236310 2024-09-17T16:28:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f842001ccc0 con 0x556ad7236310 2024-09-17T16:28:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-17T16:28:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8427fff640 10 client.4221.objecter _get_latest_version latest 17, waiting 2024-09-17T16:28:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8427fff640 10 client.4221.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8427fff640 10 monclient: _renew_subs 2024-09-17T16:28:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8427fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8427fff640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=17}) -- 0x7f8410003e10 con 0x556ad7236310 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 8 ==== osd_map(17..17 src has 1..17) ==== 658+0+0 (secure 0 0 0) 0x7f8420012dd0 con 0x556ad7236310 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_map(17..17 src has 1..17) 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map got epochs [17,17] > 16 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map decoding incremental epoch 17 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8425ffb640 20 client.4221.objecter dump_active .. 0 homeless 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 10 librados: create oid=rbd_trash nspace= 2024-09-17T16:28:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 10 client.4221.objecter _op_submit op 0x556ad72b8e40 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 20 client.4221.objecter _calc_target epoch 17 base rbd_trash @2 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 20 client.4221.objecter _calc_target target rbd_trash @2 -> pgid 2.5a749323 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 10 client.4221.objecter _calc_target raw pgid 2.5a749323 -> actual 2.3 acting [5,2] primary 5 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x556ad72e3f20 0x556ad72e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 20 client.4221.objecter _get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 10 client.4221.objecter _op_submit oid rbd_trash '@2' '@2' [create] tid 1 osd.5 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 20 client.4221.objecter get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 15 client.4221.objecter _session_op_assign 5 1 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 15 client.4221.objecter _send_op 1 to 2.3 on osd.5 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) -- 0x556ad72e6850 con 0x556ad72e3f20 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f84373fd100 5 client.4221.objecter 1 in flight 2024-09-17T16:28:01.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8437168640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x556ad72e3f20 0x556ad72e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:01.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.448+0000 7f8437168640 10 monclient: get_auth_request con 0x556ad72e3f20 auth_method 0 2024-09-17T16:28:01.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.452+0000 7f8437168640 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x556ad72e3f20 0x556ad72e6310 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:01.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.452+0000 7f8425ffb640 10 client.4221.objecter ms_handle_connect 0x556ad72e3f20 2024-09-17T16:28:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 <== osd.5 v2:172.21.15.160:6800/2641095577 1 ==== osd_op_reply(1 rbd_trash [create] v17'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f84280b0030 con 0x556ad72e3f20 2024-09-17T16:28:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_op_reply(1 rbd_trash [create] v17'1 uv1 ondisk = 0) 2024-09-17T16:28:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 10 client.4221.objecter in handle_osd_op_reply 2024-09-17T16:28:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 7 client.4221.objecter handle_osd_op_reply 1 ondisk uv 1 in 2.3 attempt 0 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 10 client.4221.objecter op 0 rval 0 len 0 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 15 client.4221.objecter handle_osd_op_reply completed tid 1 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 15 client.4221.objecter _finish_op 1 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 15 client.4221.objecter _session_op_remove 5 1 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f8437168640 5 client.4221.objecter 0 in flight 2024-09-17T16:28:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.456+0000 7f84373fd100 10 librados: Objecter returned from create r=0 2024-09-17T16:28:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _op_submit op 0x556ad7338860 2024-09-17T16:28:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _calc_target epoch 17 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:28:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [5,1] primary 5 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _op_submit oid rbd_info '@2' '@2' [call rbd.metadata_list in=33b] tid 2 osd.5 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 15 client.4221.objecter _session_op_assign 5 2 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 15 client.4221.objecter _send_op 2 to 2.2 on osd.5 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) -- 0x556ad7338db0 con 0x556ad72e3f20 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 5 client.4221.objecter 1 in flight 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 <== osd.5 v2:172.21.15.160:6800/2641095577 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f84280b0030 con 0x556ad72e3f20 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter in handle_osd_op_reply 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 7 client.4221.objecter handle_osd_op_reply 2 ondisk uv 0 in 2.2 attempt 0 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter op 0 rval 0 len 0 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter handle_osd_op_reply completed tid 2 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter _finish_op 2 2024-09-17T16:28:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter _session_op_remove 5 2 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 5 client.4221.objecter 0 in flight 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _op_submit op 0x556ad73379c0 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _calc_target epoch 17 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [5,1] primary 5 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter _get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 10 client.4221.objecter _op_submit oid rbd_info '@2' '@2' [read 0~0] tid 3 osd.5 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 15 client.4221.objecter _session_op_assign 5 3 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 15 client.4221.objecter _send_op 3 to 2.2 on osd.5 2024-09-17T16:28:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) -- 0x556ad7337e70 con 0x556ad72e3f20 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84373fd100 5 client.4221.objecter 1 in flight 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 <== osd.5 v2:172.21.15.160:6800/2641095577 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f84280b0030 con 0x556ad72e3f20 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter in handle_osd_op_reply 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 7 client.4221.objecter handle_osd_op_reply 3 ondisk uv 0 in 2.2 attempt 0 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 10 client.4221.objecter op 0 rval 0 len 0 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter handle_osd_op_reply completed tid 3 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter _finish_op 3 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 15 client.4221.objecter _session_op_remove 5 3 2024-09-17T16:28:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f8437168640 5 client.4221.objecter 0 in flight 2024-09-17T16:28:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84277fe640 10 client.4221.objecter allocate_selfmanaged_snap; pool: 2 2024-09-17T16:28:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84277fe640 10 client.4221.objecter pool_op_submit 4 2024-09-17T16:28:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84277fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:01.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:01.460+0000 7f84277fe640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f84080030e0 con 0x556ad7236310 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v18) ==== 55+0+0 (secure 0 0 0) 0x7f8420012360 con 0x556ad7236310 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 pool_op_reply(tid 4 (0) Success v18) 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter handle_pool_op_reply pool_op_reply(tid 4 (0) Success v18) 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter have request 4 at 0x7f8408003070 Op: create unmanaged snap 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 20 client.4221.objecter waiting for client to reach epoch 18 before calling back 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 monclient: _renew_subs 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=18}) -- 0x7f84140429a0 con 0x556ad7236310 2024-09-17T16:28:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter done 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 10 ==== osd_map(18..18 src has 1..18) ==== 626+0+0 (secure 0 0 0) 0x7f842001c410 con 0x556ad7236310 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_map(18..18 src has 1..18) 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map got epochs [18,18] > 17 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map decoding incremental epoch 18 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8425ffb640 20 client.4221.objecter dump_active .. 0 homeless 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 librados: queue_aio_write 0x556ad7333df0 completion 0x556ad7100ba0 write_seq 1 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 10 client.4221.objecter _op_submit op 0x7f8410006500 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 client.4221.objecter _calc_target epoch 18 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 client.4221.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 10 client.4221.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [5,1] primary 5 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 client.4221.objecter _get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 10 client.4221.objecter _op_submit oid rbd_info '@2' '@2' [create,write 0~8 in=8b] tid 5 osd.5 2024-09-17T16:28:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 client.4221.objecter get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 15 client.4221.objecter _session_op_assign 5 5 2024-09-17T16:28:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 15 client.4221.objecter _send_op 5 to 2.2 on osd.5 2024-09-17T16:28:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x7f8410006b20 con 0x556ad72e3f20 2024-09-17T16:28:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.440+0000 7f8427fff640 5 client.4221.objecter 1 in flight 2024-09-17T16:28:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 <== osd.5 v2:172.21.15.160:6800/2641095577 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v18'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f84280b0030 con 0x556ad72e3f20 2024-09-17T16:28:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_op_reply(5 rbd_info [create,write 0~8] v18'1 uv1 ondisk = 0) 2024-09-17T16:28:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 10 client.4221.objecter in handle_osd_op_reply 2024-09-17T16:28:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 7 client.4221.objecter handle_osd_op_reply 5 ondisk uv 1 in 2.2 attempt 0 2024-09-17T16:28:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 10 client.4221.objecter op 0 rval 0 len 0 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 10 client.4221.objecter op 1 rval 0 len 0 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 15 client.4221.objecter handle_osd_op_reply completed tid 5 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 15 client.4221.objecter _finish_op 5 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 15 client.4221.objecter _session_op_remove 5 5 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 5 client.4221.objecter 0 in flight 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f8437168640 20 librados: complete_aio_write 0x556ad7100ba0 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f84277fe640 10 client.4221.objecter delete_selfmanaged_snap; pool: 2; snap: 2 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f84277fe640 10 client.4221.objecter pool_op_submit 6 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f84277fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:02.452+0000 7f84277fe640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v18) -- 0x7f8408004530 con 0x556ad7236310 2024-09-17T16:28:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.440+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v19) ==== 43+0+0 (secure 0 0 0) 0x7f842002b070 con 0x556ad7236310 2024-09-17T16:28:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.440+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 pool_op_reply(tid 6 (0) Success v19) 2024-09-17T16:28:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 client.4221.objecter handle_pool_op_reply pool_op_reply(tid 6 (0) Success v19) 2024-09-17T16:28:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 client.4221.objecter have request 6 at 0x7f8408003070 Op: delete unmanaged snap 2024-09-17T16:28:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 20 client.4221.objecter waiting for client to reach epoch 19 before calling back 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 client.4221.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 monclient: _renew_subs 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=19}) -- 0x7f8414042a50 con 0x556ad7236310 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 client.4221.objecter done 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 1 -- 172.21.15.105:0/1172645659 <== mon.0 v2:172.21.15.105:3300/0 12 ==== osd_map(19..19 src has 1..19) ==== 654+0+0 (secure 0 0 0) 0x7f84200126e0 con 0x556ad7236310 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_map(19..19 src has 1..19) 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map got epochs [19,19] > 18 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 3 client.4221.objecter handle_osd_map decoding incremental epoch 19 2024-09-17T16:28:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8425ffb640 20 client.4221.objecter dump_active .. 0 homeless 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 librados: queue_aio_write 0x556ad7333df0 completion 0x7f8408001d70 write_seq 2 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 10 client.4221.objecter _op_submit op 0x7f8410006500 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 client.4221.objecter _calc_target epoch 19 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 client.4221.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 10 client.4221.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [5,1] primary 5 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 client.4221.objecter _get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 10 client.4221.objecter _op_submit oid rbd_info '@2' '@2' [write 0~19 in=19b] tid 7 osd.5 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 client.4221.objecter get_session s=0x556ad72dda20 osd=5 3 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 15 client.4221.objecter _session_op_assign 5 7 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 15 client.4221.objecter _send_op 7 to 2.2 on osd.5 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 1 -- 172.21.15.105:0/1172645659 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) -- 0x7f8410006ee0 con 0x556ad72e3f20 2024-09-17T16:28:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:03.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.444+0000 7f8427fff640 5 client.4221.objecter 1 in flight 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 <== osd.5 v2:172.21.15.160:6800/2641095577 5 ==== osd_op_reply(7 rbd_info [write 0~19] v19'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84280b0030 con 0x556ad72e3f20 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 10 client.4221.objecter ms_dispatch 0x556ad7119a30 osd_op_reply(7 rbd_info [write 0~19] v19'2 uv2 ondisk = 0) 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 10 client.4221.objecter in handle_osd_op_reply 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 7 client.4221.objecter handle_osd_op_reply 7 ondisk uv 2 in 2.2 attempt 0 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 10 client.4221.objecter op 0 rval 0 len 0 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 15 client.4221.objecter handle_osd_op_reply completed tid 7 2024-09-17T16:28:03.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 15 client.4221.objecter _finish_op 7 2024-09-17T16:28:03.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 4 2024-09-17T16:28:03.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 15 client.4221.objecter _session_op_remove 5 7 2024-09-17T16:28:03.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 5 client.4221.objecter 0 in flight 2024-09-17T16:28:03.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 20 librados: complete_aio_write 0x7f8408001d70 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 10 librados: watch_flush enter 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 10 librados: watch_flush exit 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 10 client.4221.objecter close_session for osd.5 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x556ad72e3f20 msgr2=0x556ad72e6310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x556ad72e3f20 0x556ad72e6310 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 20 client.4221.objecter put_session s=0x556ad72dda20 osd=5 1 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 20 client.4221.objecter shutdown clearing up homeless session... 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 10 client.4221.objecter successfully canceled tick 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841403c820 msgr2=0x7f841403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841403c820 0x7f841403ece0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f842c00ad50 tx=0x7f842c005c30 comp rx=0 tx=0).stop 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 10 monclient: shutdown 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 msgr2=0x556ad72dd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 --2- 172.21.15.105:0/1172645659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556ad7236310 0x556ad72dd4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x556ad721b870 tx=0x7f8420008c80 comp rx=0 tx=0).stop 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f8437168640 1 -- 172.21.15.105:0/1172645659 reap_dead start 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 shutdown_connections 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 >> 172.21.15.105:0/1172645659 conn(0x556ad7217b20 msgr2=0x556ad722e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.456+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 shutdown_connections 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.460+0000 7f84373fd100 1 -- 172.21.15.105:0/1172645659 wait complete. 2024-09-17T16:28:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.460+0000 7f84373fd100 1 librados: shutdown 2024-09-17T16:28:03.475 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T16:28:03.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T16:28:03.476 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T16:28:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: get_monmap_and_config 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: build_initial_monmap 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: monmap: 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:03.602431+0000 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:03.602431+0000 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:03.606 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 Processor -- start 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 -- start start 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: init 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient: _add_conns ranks=[1,9,4,10,0,5,3,6,8,7,2] 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8a2c0a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-a con 0x7f8a2c0a4ab0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8a2c0a7520 0x7f8a2c140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-c con 0x7f8a2c0a7520 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:03.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a2c0a4080 con 0x7f8a2c0a6a30 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c0a7c40 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8a2c0a37e0 con 0x7f8a2c0a7520 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a30fe9640 10 monclient(hunting): authenticate will time out at 966.867849s 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a29d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.596+0000 7f8a2ad9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8a2c0a7520 0x7f8a2c140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45130/0 (socket says 172.21.15.105:45130) 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 -- 172.21.15.105:0/4065440842 learned_addr learned my addr 172.21.15.105:0/4065440842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): get_auth_request con 0x7f8a2c0a4ab0 auth_method 0 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): get_auth_request con 0x7f8a2c0a7520 auth_method 0 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_done global_id 4230 payload 274 2024-09-17T16:28:03.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 -- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8a2c0a7520 msgr2=0x7f8a2c140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 --2- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8a2c0a7520 0x7f8a2c140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 -- 172.21.15.105:0/4065440842 >> v1:172.21.15.160:6789/0 conn(0x7f8a2c0a6a30 legacy=0x7f8a2c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 monclient: found mon.noname-a 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 -- 172.21.15.105:0/4065440842 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0a3ac0 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a29d9b640 1 --2- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009870 tx=0x7f8a1c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d835423a404c82ad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 1 -- 172.21.15.105:0/4065440842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8a1c0b0070 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: dump: 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.617 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.618 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: _finish_auth 0 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 1 -- 172.21.15.105:0/4065440842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0aa810 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 5 monclient: authenticate success, global_id 4230 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 4 set_mon_vals no callback set 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 1 -- 172.21.15.105:0/4065440842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8a1c0a9300 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a2959a640 10 monclient: dump: 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.620 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 10 monclient: shutdown 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 -- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2c0a4ab0 msgr2=0x7f8a2c13ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009870 tx=0x7f8a1c01c420 comp rx=0 tx=0).stop 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 -- 172.21.15.105:0/4065440842 shutdown_connections 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8a2c0a7520 0x7f8a2c140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/4065440842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c13ea30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 -- 172.21.15.105:0/4065440842 >> 172.21.15.105:0/4065440842 conn(0x7f8a2c0a8d00 msgr2=0x7f8a2c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 -- 172.21.15.105:0/4065440842 shutdown_connections 2024-09-17T16:28:03.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 1 -- 172.21.15.105:0/4065440842 wait complete. 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 10 monclient: build_initial_monmap 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.600+0000 7f8a30fe9640 10 monclient: monmap: 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:03.622 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 librados: starting msgr at 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 librados: starting objecter 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 Processor -- start 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- start start 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 librados: setting wanted keys 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 librados: calling monclient init 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: init 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:03.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _add_conns ranks=[1,6,2,5,4,7,0,3,8] 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c128a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-f con 0x7f8a2c0a4ab0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 0x7f8a2c1d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-d con 0x7f8a2c0a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): picked mon.noname-b con 0x7f8a2c0a7520 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c143e60 con 0x7f8a2c0a4ab0 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c143ce0 con 0x7f8a2c0a7520 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8a2c143fe0 con 0x7f8a2c0a6a30 2024-09-17T16:28:03.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient(hunting): authenticate will time out at 966.872938s 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 0x7f8a2c1d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59432/0 (socket says 172.21.15.105:59432) 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 -- 172.21.15.105:0/2040490303 learned_addr learned my addr 172.21.15.105:0/2040490303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): get_auth_request con 0x7f8a2c0a7520 auth_method 0 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): get_auth_request con 0x7f8a2c0a6a30 auth_method 0 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c128a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 10 monclient(hunting): get_auth_request con 0x7f8a2c0a4ab0 auth_method 0 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 -- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 msgr2=0x7f8a2c1d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 0x7f8a2c1d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 -- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a2c0a4ab0 msgr2=0x7f8a2c128a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c128a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 monclient: found mon.noname-b 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 -- 172.21.15.105:0/2040490303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c129410 con 0x7f8a2c0a7520 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a29d9b640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 0x7f8a2c1d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2ad9d640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400e820 tx=0x7f8a2400ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 client.?.objecter ms_handle_connect 0x7f8a2c0a7520 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8a24017070 con 0x7f8a2c0a7520 2024-09-17T16:28:03.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: dump: 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.628 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: _finish_auth 0 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 5 monclient: authenticate success, global_id 4214 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2400cba0 con 0x7f8a2c0a7520 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _renew_subs 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c1c8e70 con 0x7f8a2c0a7520 2024-09-17T16:28:03.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8a2400cd50 con 0x7f8a2c0a7520 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 client.4214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a28d99640 4 set_mon_vals no callback set 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 monclient: dump: 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.630 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _renew_subs 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c12a280 con 0x7f8a2c0a7520 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 librados: init done 2024-09-17T16:28:03.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c074850 con 0x7f8a2c0a7520 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8a2401a070 con 0x7f8a2c0a7520 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 0x7f8a0c040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f8a2400f040 con 0x7f8a2c0a7520 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 10 client.4214.objecter ms_dispatch 0x7f8a2c000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 3 client.4214.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 3 client.4214.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a1affd640 20 client.4214.objecter dump_active .. 0 homeless 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 0x7f8a0c040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.604+0000 7f8a2a59c640 10 monclient: get_auth_request con 0x7f8a0c03e480 auth_method 0 2024-09-17T16:28:03.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.608+0000 7f8a1affd640 10 client.4214.objecter ms_handle_connect 0x7f8a0c03e480 2024-09-17T16:28:03.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.608+0000 7f8a2a59c640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 0x7f8a0c040940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005530 tx=0x7f8a20009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.608+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8a24011350 con 0x7f8a2c0a7520 2024-09-17T16:28:03.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.608+0000 7f8a1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.608+0000 7f8a1affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:03.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 start_mon_command cmd=[{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-17T16:28:03.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 monclient: _send_command 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-17T16:28:03.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:03.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f8a2c13ff40 con 0x7f8a2c0a7520 2024-09-17T16:28:03.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a1affd640 1 -- 172.21.15.105:0/2040490303 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v3) ==== 86+0+159 (secure 0 0 0) 0x7f8a24013d20 con 0x7f8a2c0a7520 2024-09-17T16:28:03.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-09-17T16:28:03.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a1affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:03.793 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 librados: watch_flush enter 2024-09-17T16:28:03.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 librados: watch_flush exit 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 20 client.4214.objecter shutdown clearing up homeless session... 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.784+0000 7f8a30fe9640 10 client.4214.objecter successfully canceled tick 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 msgr2=0x7f8a0c040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 0x7f8a0c040940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005530 tx=0x7f8a20009040 comp rx=0 tx=0).stop 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 10 monclient: shutdown 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 msgr2=0x7f8a2c1265f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400e820 tx=0x7f8a2400ece0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 shutdown_connections 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8a0c03e480 0x7f8a0c040940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a2c0a7520 0x7f8a2c1265f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8a2c0a6a30 0x7f8a2c1d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 --2- 172.21.15.105:0/2040490303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a2c0a4ab0 0x7f8a2c128a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 >> 172.21.15.105:0/2040490303 conn(0x7f8a2c0a8d00 msgr2=0x7f8a2c0aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:03.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 shutdown_connections 2024-09-17T16:28:03.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 -- 172.21.15.105:0/2040490303 wait complete. 2024-09-17T16:28:03.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.788+0000 7f8a30fe9640 1 librados: shutdown 2024-09-17T16:28:03.811 INFO:teuthology.orchestra.run.smithi105.stdout:[{"version":3,"timestamp":"2024-09-17T16:27:51.649648+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"9444.823187"}]}] 2024-09-17T16:28:03.811 INFO:tasks.ceph_manager:config epoch is 3 2024-09-17T16:28:03.812 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T16:28:03.812 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T16:28:03.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T16:28:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 10 monclient: get_monmap_and_config 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 10 monclient: build_initial_monmap 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 10 monclient: monmap: 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:03.929618+0000 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:03.929618+0000 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:03.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 1 Processor -- start 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 1 -- start start 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.924+0000 7fa5f9d4f640 10 monclient: init 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient: _add_conns ranks=[8,3,7,1,5,2,6,9,10,0,4] 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-e con 0x7fa5f40a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 0x7fa5f40a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-g con 0x7fa5f40a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 0x7fa5f4139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-i con 0x7fa5f41396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f40a3fc0 con 0x7fa5f40a4910 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa5f40a63c0 con 0x7fa5f41396a0 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa5f40a7740 con 0x7fa5f40a68e0 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient(hunting): authenticate will time out at 967.196213s 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34096/0 (socket says 172.21.15.105:34096) 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 -- 172.21.15.105:0/784370909 learned_addr learned my addr 172.21.15.105:0/784370909 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): get_auth_request con 0x7fa5f40a68e0 auth_method 0 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f3fff640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 0x7fa5f4139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_done global_id 4220 payload 274 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 -- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 msgr2=0x7fa5f4139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 0x7fa5f4139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f2ffd640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 0x7fa5f40a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 -- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 msgr2=0x7fa5f40a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 0x7fa5f40a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 monclient: found mon.noname-e 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 -- 172.21.15.105:0/784370909 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f406ec10 con 0x7fa5f40a68e0 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f3fff640 10 monclient: get_auth_request con 0x7fa5f41396a0 auth_method 0 2024-09-17T16:28:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f2ffd640 10 monclient: get_auth_request con 0x7fa5f40a4910 auth_method 0 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f3fff640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 0x7fa5f4139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f2ffd640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 0x7fa5f40a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009eb0 tx=0x7fa5e401d350 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ea4b74cdd698ae5d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 1 -- 172.21.15.105:0/784370909 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e40090d0 con 0x7fa5f40a68e0 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: dump: 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.943 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:03.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 1 -- 172.21.15.105:0/784370909 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e40a5a50 con 0x7fa5f40a68e0 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 5 monclient: authenticate success, global_id 4220 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 1 -- 172.21.15.105:0/784370909 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e40a5c40 con 0x7fa5f40a68e0 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 4 set_mon_vals no callback set 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f27fc640 10 monclient: dump: 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.945 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 10 monclient: shutdown 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/784370909 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 msgr2=0x7fa5f40a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009eb0 tx=0x7fa5e401d350 comp rx=0 tx=0).stop 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/784370909 shutdown_connections 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5f41396a0 0x7fa5f4139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5f40a4910 0x7fa5f40a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/784370909 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5f40a68e0 0x7fa5f40a6cc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.928+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/784370909 >> 172.21.15.105:0/784370909 conn(0x7fa5f4068ba0 msgr2=0x7fa5f4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/784370909 shutdown_connections 2024-09-17T16:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/784370909 wait complete. 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: build_initial_monmap 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: monmap: 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:03.948 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 librados: starting msgr at 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 librados: starting objecter 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 Processor -- start 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- start start 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 librados: setting wanted keys 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 librados: calling monclient init 2024-09-17T16:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: init 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _add_conns ranks=[0,6,2,8,3,5,7,1,4] 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-a con 0x7fa5f40a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5f40a68e0 0x7fa5f41d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-d con 0x7fa5f40a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f41396a0 0x7fa5f413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): picked mon.noname-b con 0x7fa5f41396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f4143320 con 0x7fa5f40a4910 2024-09-17T16:28:03.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f406e630 con 0x7fa5f41396a0 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa5f41434a0 con 0x7fa5f40a68e0 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient(hunting): authenticate will time out at 967.201651s 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45144/0 (socket says 172.21.15.105:45144) 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 -- 172.21.15.105:0/1941637568 learned_addr learned my addr 172.21.15.105:0/1941637568 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): get_auth_request con 0x7fa5f40a4910 auth_method 0 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f41396a0 0x7fa5f413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f2ffd640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5f40a68e0 0x7fa5f41d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): get_auth_request con 0x7fa5f41396a0 auth_method 0 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f2ffd640 10 monclient(hunting): get_auth_request con 0x7fa5f40a68e0 auth_method 0 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:03.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient(hunting): handle_auth_done global_id 4239 payload 824 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 -- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5f40a68e0 msgr2=0x7fa5f41d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5f40a68e0 0x7fa5f41d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 -- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f41396a0 msgr2=0x7fa5f413d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:03.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f41396a0 0x7fa5f413d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 monclient: found mon.noname-a 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 -- 172.21.15.105:0/1941637568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f4123930 con 0x7fa5f40a4910 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f37fe640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40af040 tx=0x7fa5e4019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 client.?.objecter ms_handle_connect 0x7fa5f40a4910 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e40bc020 con 0x7fa5f40a4910 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: dump: 2024-09-17T16:28:03.958 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.959 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: _finish_auth 0 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 5 monclient: authenticate success, global_id 4239 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e4003e10 con 0x7fa5f40a4910 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _renew_subs 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f4123c10 con 0x7fa5f40a4910 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5e40a58b0 con 0x7fa5f40a4910 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:03.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 client.4239.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f17fa640 4 set_mon_vals no callback set 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 10 monclient: dump: 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:03.961 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _renew_subs 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f40a9e50 con 0x7fa5f40a4910 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 librados: init done 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f4139ae0 con 0x7fa5f40a4910 2024-09-17T16:28:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa5e40bf020 con 0x7fa5f40a4910 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.932+0000 7fa5dbfff640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 0x7fa5e0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fa5e40ea690 con 0x7fa5f40a4910 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 10 client.4239.objecter ms_dispatch 0x7fa5f4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 3 client.4239.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 3 client.4239.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 20 client.4239.objecter dump_active .. 0 homeless 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5f2ffd640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 0x7fa5e0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5f2ffd640 10 monclient: get_auth_request con 0x7fa5e003e550 auth_method 0 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5dbfff640 10 client.4239.objecter ms_handle_connect 0x7fa5e003e550 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.936+0000 7fa5f2ffd640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 0x7fa5e0040a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5f413eca0 tx=0x7fa5e8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.940+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa5e40c6030 con 0x7fa5f40a4910 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.940+0000 7fa5dbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:03.940+0000 7fa5dbfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.164+0000 7fa5f9d4f640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-17T16:28:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.164+0000 7fa5f9d4f640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-17T16:28:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.164+0000 7fa5f9d4f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.164+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa5f40746d0 con 0x7fa5f40a4910 2024-09-17T16:28:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.168+0000 7fa5dbfff640 1 -- 172.21.15.105:0/1941637568 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+93966 (secure 0 0 0) 0x7fa5e4016600 con 0x7fa5f40a4910 2024-09-17T16:28:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.168+0000 7fa5dbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-17T16:28:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.168+0000 7fa5dbfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:04.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:04.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 10 librados: watch_flush enter 2024-09-17T16:28:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 10 librados: watch_flush exit 2024-09-17T16:28:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 20 client.4239.objecter shutdown clearing up homeless session... 2024-09-17T16:28:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 10 client.4239.objecter successfully canceled tick 2024-09-17T16:28:04.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 msgr2=0x7fa5e0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 0x7fa5e0040a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5f413eca0 tx=0x7fa5e8008040 comp rx=0 tx=0).stop 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 10 monclient: shutdown 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 msgr2=0x7fa5f41d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40af040 tx=0x7fa5e4019690 comp rx=0 tx=0).stop 2024-09-17T16:28:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 shutdown_connections 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa5e003e550 0x7fa5e0040a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5f41396a0 0x7fa5f413d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5f40a68e0 0x7fa5f41d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 --2- 172.21.15.105:0/1941637568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5f40a4910 0x7fa5f41d1620 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 >> 172.21.15.105:0/1941637568 conn(0x7fa5f4068ba0 msgr2=0x7fa5f4068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 shutdown_connections 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 -- 172.21.15.105:0/1941637568 wait complete. 2024-09-17T16:28:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.172+0000 7fa5f9d4f640 1 librados: shutdown 2024-09-17T16:28:04.194 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"flags":0,"active_gid":4118,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3499101759},{"type":"v1","addr":"172.21.15.160:6825","nonce":3499101759}]},"active_addr":"172.21.15.160:6825/3499101759","active_change":"2024-09-17T16:27:47.767107+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3617573755}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2948392263}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3554657607}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3851500278}]}]} 2024-09-17T16:28:04.197 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T16:28:04.197 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T16:28:04.197 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T16:28:04.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 10 monclient: get_monmap_and_config 2024-09-17T16:28:04.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 10 monclient: monmap: 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:04.309923+0000 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:04.309923+0000 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.313 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:04.314 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 1 Processor -- start 2024-09-17T16:28:04.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 1 -- start start 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.304+0000 7fa6d9f5a640 10 monclient: init 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient: _add_conns ranks=[9,8,10,4,5,6,0,2,3,7,1] 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fa6d40a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-e con 0x7fa6d40a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa6d41396a0 0x7fa6d4139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-f con 0x7fa6d41396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d40a3fc0 con 0x7fa6d40a68e0 2024-09-17T16:28:04.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fa6d40a63c0 con 0x7fa6d41396a0 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa6d40a7740 con 0x7fa6d40a4910 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient(hunting): authenticate will time out at 967.576227s 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45148/0 (socket says 172.21.15.105:45148) 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 1 -- 172.21.15.105:0/4149032983 learned_addr learned my addr 172.21.15.105:0/4149032983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): get_auth_request con 0x7fa6d40a68e0 auth_method 0 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34100/0 (socket says 172.21.15.105:34100) 2024-09-17T16:28:04.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): get_auth_request con 0x7fa6d40a4910 auth_method 0 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d3fff640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.160:3300/0 conn(0x7fa6d41396a0 0x7fa6d4139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d3fff640 10 monclient(hunting): get_auth_request con 0x7fa6d41396a0 auth_method 0 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient(hunting): handle_auth_done global_id 4229 payload 274 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 -- 172.21.15.105:0/4149032983 >> v2:172.21.15.160:3300/0 conn(0x7fa6d41396a0 msgr2=0x7fa6d4139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.160:3300/0 conn(0x7fa6d41396a0 0x7fa6d4139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 -- 172.21.15.105:0/4149032983 >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 msgr2=0x7fa6d40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 monclient: found mon.noname-e 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:04.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 -- 172.21.15.105:0/4149032983 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d406ebc0 con 0x7fa6d40a4910 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d37fe640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d2ffd640 1 --2- 172.21.15.105:0/4149032983 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40acf60 tx=0x7fa6c801c310 comp rx=0 tx=0).ready entity=mon.8 client_cookie=89eef5c0a6d6e42c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 1 -- 172.21.15.105:0/4149032983 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6c80b0070 con 0x7fa6d40a4910 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: dump: 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.323 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.324 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.324 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.324 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 1 -- 172.21.15.105:0/4149032983 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c80aa810 con 0x7fa6d40a4910 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 5 monclient: authenticate success, global_id 4229 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 4 set_mon_vals no callback set 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 1 -- 172.21.15.105:0/4149032983 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6c80a9300 con 0x7fa6d40a4910 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d27fc640 10 monclient: dump: 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.326 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 10 monclient: shutdown 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/4149032983 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 msgr2=0x7fa6d40a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/4149032983 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40acf60 tx=0x7fa6c801c310 comp rx=0 tx=0).stop 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/4149032983 shutdown_connections 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/4149032983 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa6d40a4910 0x7fa6d40a7200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.160:3300/0 conn(0x7fa6d41396a0 0x7fa6d4139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/4149032983 >> v2:172.21.15.105:3300/0 conn(0x7fa6d40a68e0 0x7fa6d40a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/4149032983 >> 172.21.15.105:0/4149032983 conn(0x7fa6d4068ba0 msgr2=0x7fa6d4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/4149032983 shutdown_connections 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.308+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/4149032983 wait complete. 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: monmap: 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 librados: starting msgr at 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 librados: starting objecter 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 Processor -- start 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- start start 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 librados: setting wanted keys 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 librados: calling monclient init 2024-09-17T16:28:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: init 2024-09-17T16:28:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _add_conns ranks=[6,7,0,3,5,2,8,1,4] 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 0x7fa6d4137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-d con 0x7fa6d40a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 0x7fa6d4137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-i con 0x7fa6d40a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fa6d41396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d406e630 con 0x7fa6d41396a0 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa6d40a7af0 con 0x7fa6d40a4910 2024-09-17T16:28:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa6d4143320 con 0x7fa6d40a68e0 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient(hunting): authenticate will time out at 967.581414s 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 0x7fa6d4137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45150/0 (socket says 172.21.15.105:45150) 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 -- 172.21.15.105:0/3973426615 learned_addr learned my addr 172.21.15.105:0/3973426615 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): get_auth_request con 0x7fa6d41396a0 auth_method 0 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): get_auth_request con 0x7fa6d40a4910 auth_method 0 2024-09-17T16:28:04.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d2ffd640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 0x7fa6d4137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2024-09-17T16:28:04.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 -- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 msgr2=0x7fa6d4137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 0x7fa6d4137f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 -- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 msgr2=0x7fa6d4137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 0x7fa6d4137a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 monclient: found mon.noname-a 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 -- 172.21.15.105:0/3973426615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d40aa070 con 0x7fa6d41396a0 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d2ffd640 10 monclient: get_auth_request con 0x7fa6d40a68e0 auth_method 0 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d2ffd640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 0x7fa6d4137f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d37fe640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 0x7fa6d4137a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d3fff640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc007e90 tx=0x7fa6cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 client.?.objecter ms_handle_connect 0x7fa6d41396a0 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6cc00ed10 con 0x7fa6d41396a0 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: dump: 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc019070 con 0x7fa6d41396a0 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 5 monclient: authenticate success, global_id 4257 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _renew_subs 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d4123d70 con 0x7fa6d41396a0 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6cc014d00 con 0x7fa6d41396a0 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 monclient: dump: 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.342 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d1ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 client.4257.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _renew_subs 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d413efb0 con 0x7fa6d41396a0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 librados: init done 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa6cc014420 con 0x7fa6d41396a0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d4139b50 con 0x7fa6d41396a0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 0x7fa6c0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fa6cc04fe60 con 0x7fa6d41396a0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 10 client.4257.objecter ms_dispatch 0x7fa6d4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 3 client.4257.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 3 client.4257.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.312+0000 7fa6bbfff640 20 client.4257.objecter dump_active .. 0 homeless 2024-09-17T16:28:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.316+0000 7fa6d37fe640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 0x7fa6c0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.316+0000 7fa6d37fe640 10 monclient: get_auth_request con 0x7fa6c003e550 auth_method 0 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.316+0000 7fa6bbfff640 10 client.4257.objecter ms_handle_connect 0x7fa6c003e550 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.316+0000 7fa6d37fe640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 0x7fa6c0040a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009630 tx=0x7fa6c4009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.320+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6d40aa070 con 0x7fa6d41396a0 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.320+0000 7fa6bbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.320+0000 7fa6bbfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:04.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.492+0000 7fa6d9f5a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.492+0000 7fa6d9f5a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.492+0000 7fa6d9f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.492+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6d40746d0 con 0x7fa6d41396a0 2024-09-17T16:28:04.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6bbfff640 1 -- 172.21.15.105:0/3973426615 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12425 (secure 0 0 0) 0x7fa6cc01c360 con 0x7fa6d41396a0 2024-09-17T16:28:04.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6bbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6bbfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:04.503 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:28:03.439097+0000","last_up_change":"2024-09-17T16:27:52.867595+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T16:27:53.825252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T16:27:59.687166+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6817","nonce":315040748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6819","nonce":315040748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6823","nonce":315040748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6821","nonce":315040748}]},"public_addr":"172.21.15.105:6817/315040748","cluster_addr":"172.21.15.105:6819/315040748","heartbeat_back_addr":"172.21.15.105:6823/315040748","heartbeat_front_addr":"172.21.15.105:6821/315040748","state":["exists","up"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6809","nonce":3218035673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6811","nonce":3218035673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6815","nonce":3218035673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6813","nonce":3218035673}]},"public_addr":"172.21.15.160:6809/3218035673","cluster_addr":"172.21.15.160:6811/3218035673","heartbeat_back_addr":"172.21.15.160:6815/3218035673","heartbeat_front_addr":"172.21.15.160:6813/3218035673","state":["exists","up"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6817","nonce":639753971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6819","nonce":639753971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6823","nonce":639753971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6821","nonce":639753971}]},"public_addr":"172.21.15.160:6817/639753971","cluster_addr":"172.21.15.160:6819/639753971","heartbeat_back_addr":"172.21.15.160:6823/639753971","heartbeat_front_addr":"172.21.15.160:6821/639753971","state":["exists","up"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.073377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.536863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.709983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.944027+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.614775+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.703905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 10 librados: watch_flush enter 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 10 librados: watch_flush exit 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 20 client.4257.objecter shutdown clearing up homeless session... 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 10 client.4257.objecter successfully canceled tick 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 msgr2=0x7fa6c0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 0x7fa6c0040a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009630 tx=0x7fa6c4009240 comp rx=0 tx=0).stop 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 10 monclient: shutdown 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 msgr2=0x7fa6d4138490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc007e90 tx=0x7fa6cc00e4b0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 shutdown_connections 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6c003e550 0x7fa6c0040a10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d41396a0 0x7fa6d4138490 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa6d40a68e0 0x7fa6d4137f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 --2- 172.21.15.105:0/3973426615 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa6d40a4910 0x7fa6d4137a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 >> 172.21.15.105:0/3973426615 conn(0x7fa6d4068ba0 msgr2=0x7fa6d40a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 shutdown_connections 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 -- 172.21.15.105:0/3973426615 wait complete. 2024-09-17T16:28:04.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.496+0000 7fa6d9f5a640 1 librados: shutdown 2024-09-17T16:28:04.521 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T16:28:04.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.628+0000 7fde764ea640 10 monclient: get_monmap_and_config 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.628+0000 7fde764ea640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.628+0000 7fde764ea640 10 monclient: monmap: 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:04.635077+0000 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:04.635077+0000 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:04.639 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 Processor -- start 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 -- start start 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient: init 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient: _add_conns ranks=[7,3,0,6,9,1,4,10,8,5,2] 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fde700a4750 0x7fde700a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-i con 0x7fde700a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde7009e160 0x7fde700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-g con 0x7fde7009e160 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fde70136500 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7009e5e0 con 0x7fde70136500 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fde7009f680 con 0x7fde7009e160 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fde700a55b0 con 0x7fde700a4750 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 10 monclient(hunting): authenticate will time out at 967.900960s 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fde70136500 0x7fde70136900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37196/0 (socket says 172.21.15.105:37196) 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 1 -- 172.21.15.105:0/3529449444 learned_addr learned my addr 172.21.15.105:0/3529449444 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1682319722 0 0) 0x7fde600014a0 con 0x7fde70136500 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): _negotiate 2024-09-17T16:28:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): my global_id is 4266 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde58003530 con 0x7fde70136500 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde7009e160 0x7fde700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fde700a4750 0x7fde700a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request con 0x7fde7009e160 auth_method 0 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): get_auth_request con 0x7fde700a4750 auth_method 0 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1785149096 0 0) 0x7fde58003530 con 0x7fde70136500 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fde700a4750 msgr2=0x7fde700a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fde700a4750 0x7fde700a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde7009e160 msgr2=0x7fde700a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde7009e160 0x7fde700a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7009fc40 con 0x7fde70136500 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 5 monclient: authenticate success, global_id 4266 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fde60001ec0 con 0x7fde70136500 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde7009e160 0x7fde700a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: dump: 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 >> v1:172.21.15.105:6789/0 conn(0x7fde70136500 legacy=0x7fde70136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient: _add_conns ranks=[4,0,1,2,5,6,3,8,7] 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004210 0x7fde58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): picked mon.c con 0x7fde58004210 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 0x7fde58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): picked mon.a con 0x7fde58004da0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde58009b60 0x7fde5800bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): picked mon.f con 0x7fde58009b60 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fde7009f680 con 0x7fde58004da0 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fde700a55b0 con 0x7fde58009b60 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fde5800c900 con 0x7fde58004210 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6effd640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde764ea640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004210 0x7fde58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): get_auth_request con 0x7fde58004210 auth_method 0 2024-09-17T16:28:04.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 0x7fde58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request con 0x7fde58004da0 auth_method 0 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde58009b60 0x7fde5800bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 10 monclient(hunting): get_auth_request con 0x7fde58009b60 auth_method 0 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_done global_id 4266 payload 293 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 -- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004210 msgr2=0x7fde58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004210 0x7fde58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 -- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde58009b60 msgr2=0x7fde5800bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde58009b60 0x7fde5800bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 monclient: found mon.a 2024-09-17T16:28:04.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde74a87640 1 -- 172.21.15.105:0/3529449444 reap_dead start 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 1 -- 172.21.15.105:0/3529449444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde58003530 con 0x7fde58004da0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.632+0000 7fde6ffff640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004210 0x7fde58004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 0x7fde58005200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde640098d0 tx=0x7fde6401c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d49308d8b28e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde6400f040 con 0x7fde58004da0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: dump: 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.653 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.654 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde640af070 con 0x7fde58004da0 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 1 -- 172.21.15.105:0/3529449444 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde7009ea00 con 0x7fde58004da0 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 4 set_mon_vals no callback set 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6effd640 10 monclient: dump: 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.655 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: shutdown 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 msgr2=0x7fde58005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 0x7fde58005200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde640098d0 tx=0x7fde6401c5b0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- 172.21.15.105:0/3529449444 shutdown_connections 2024-09-17T16:28:04.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 --2- 172.21.15.105:0/3529449444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde58004da0 0x7fde58005200 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- 172.21.15.105:0/3529449444 >> 172.21.15.105:0/3529449444 conn(0x7fde700a9540 msgr2=0x7fde700a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- 172.21.15.105:0/3529449444 shutdown_connections 2024-09-17T16:28:04.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- 172.21.15.105:0/3529449444 wait complete. 2024-09-17T16:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: monmap: 2024-09-17T16:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 librados: starting msgr at 2024-09-17T16:28:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 librados: starting objecter 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 Processor -- start 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- start start 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 librados: setting wanted keys 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 librados: calling monclient init 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: init 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient: _add_conns ranks=[4,6,8,5,7,2,1,0,3] 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-c con 0x7fde58004da0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-d con 0x7fde70125150 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fde701cf550 0x7fde701cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): picked mon.noname-e con 0x7fde701cf550 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fde70136c70 con 0x7fde58004da0 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fde70136af0 con 0x7fde70125150 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fde70136df0 con 0x7fde701cf550 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde764ea640 10 monclient(hunting): authenticate will time out at 967.907274s 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57554/0 (socket says 172.21.15.105:57554) 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 1 -- 172.21.15.105:0/4231876488 learned_addr learned my addr 172.21.15.105:0/4231876488 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33880/0 (socket says 172.21.15.105:33880) 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): get_auth_request con 0x7fde58004da0 auth_method 0 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fde701cf550 0x7fde701cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request con 0x7fde70125150 auth_method 0 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): get_auth_request con 0x7fde701cf550 auth_method 0 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde74a87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient(hunting): handle_auth_done global_id 4245 payload 824 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 -- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fde701cf550 msgr2=0x7fde701cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fde701cf550 0x7fde701cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 -- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 msgr2=0x7fde70124c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6f7fe640 1 -- 172.21.15.105:0/4231876488 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde700a8400 con 0x7fde70125150 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.636+0000 7fde6ffff640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6f7fe640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c8a0 tx=0x7fde58005840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 client.?.objecter ms_handle_connect 0x7fde70125150 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde640b3860 con 0x7fde70125150 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: dump: 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde640a6040 con 0x7fde70125150 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 5 monclient: authenticate success, global_id 4245 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _renew_subs 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde70127950 con 0x7fde70125150 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde640bb3b0 con 0x7fde70125150 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 monclient: dump: 2024-09-17T16:28:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6e7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 client.4245.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _renew_subs 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fde70126ae0 con 0x7fde70125150 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 1 librados: init done 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde70137180 con 0x7fde70125150 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fde640b33f0 con 0x7fde70125150 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 0x7fde5c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fde640b9070 con 0x7fde70125150 2024-09-17T16:28:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 client.4245.objecter ms_dispatch 0x7fde70000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 3 client.4245.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 3 client.4245.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 20 client.4245.objecter dump_active .. 0 homeless 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6ffff640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 0x7fde5c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6ffff640 10 monclient: get_auth_request con 0x7fde5c03e550 auth_method 0 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6cff9640 10 client.4245.objecter ms_handle_connect 0x7fde5c03e550 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.640+0000 7fde6ffff640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 0x7fde5c040a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde68004660 tx=0x7fde68008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.644+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fde640c1030 con 0x7fde70125150 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.644+0000 7fde6cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.644+0000 7fde6cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.812+0000 7fde764ea640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.812+0000 7fde764ea640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.812+0000 7fde764ea640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.812+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde7006cbf0 con 0x7fde70125150 2024-09-17T16:28:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde6cff9640 1 -- 172.21.15.105:0/4231876488 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12425 (secure 0 0 0) 0x7fde700a8400 con 0x7fde70125150 2024-09-17T16:28:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde6cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde6cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:04.823 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:04.823 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:28:03.439097+0000","last_up_change":"2024-09-17T16:27:52.867595+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T16:27:53.825252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T16:27:59.687166+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6817","nonce":315040748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6819","nonce":315040748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6823","nonce":315040748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6821","nonce":315040748}]},"public_addr":"172.21.15.105:6817/315040748","cluster_addr":"172.21.15.105:6819/315040748","heartbeat_back_addr":"172.21.15.105:6823/315040748","heartbeat_front_addr":"172.21.15.105:6821/315040748","state":["exists","up"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6809","nonce":3218035673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6811","nonce":3218035673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6815","nonce":3218035673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6813","nonce":3218035673}]},"public_addr":"172.21.15.160:6809/3218035673","cluster_addr":"172.21.15.160:6811/3218035673","heartbeat_back_addr":"172.21.15.160:6815/3218035673","heartbeat_front_addr":"172.21.15.160:6813/3218035673","state":["exists","up"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6817","nonce":639753971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6819","nonce":639753971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6823","nonce":639753971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6821","nonce":639753971}]},"public_addr":"172.21.15.160:6817/639753971","cluster_addr":"172.21.15.160:6819/639753971","heartbeat_back_addr":"172.21.15.160:6823/639753971","heartbeat_front_addr":"172.21.15.160:6821/639753971","state":["exists","up"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.073377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.536863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.709983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.944027+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.614775+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.703905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 10 librados: watch_flush enter 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 10 librados: watch_flush exit 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 20 client.4245.objecter shutdown clearing up homeless session... 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 10 client.4245.objecter successfully canceled tick 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 msgr2=0x7fde5c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 0x7fde5c040a10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde68004660 tx=0x7fde68008040 comp rx=0 tx=0).stop 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 10 monclient: shutdown 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 msgr2=0x7fde701255b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c8a0 tx=0x7fde58005840 comp rx=0 tx=0).stop 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 shutdown_connections 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fde5c03e550 0x7fde5c040a10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fde70125150 0x7fde701255b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fde701cf550 0x7fde701cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 --2- 172.21.15.105:0/4231876488 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fde58004da0 0x7fde70124c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 >> 172.21.15.105:0/4231876488 conn(0x7fde700a9540 msgr2=0x7fde700a0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 shutdown_connections 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 -- 172.21.15.105:0/4231876488 wait complete. 2024-09-17T16:28:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.816+0000 7fde764ea640 1 librados: shutdown 2024-09-17T16:28:04.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T16:28:04.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T16:28:04.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T16:28:04.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T16:28:04.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T16:28:04.845 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T16:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: get_monmap_and_config 2024-09-17T16:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: monmap: 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:04.956725+0000 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:04.956725+0000 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 Processor -- start 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 -- start start 2024-09-17T16:28:04.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: init 2024-09-17T16:28:04.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient: _add_conns ranks=[1,3,9,0,4,6,10,5,2,7,8] 2024-09-17T16:28:04.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7faff40a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:04.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a4ab0 0x7faff413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-g con 0x7faff40a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faff40a7520 0x7faff4140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-a con 0x7faff40a7520 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faff40a7520 0x7faff4140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faff40a7520 0x7faff4140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45168/0 (socket says 172.21.15.105:45168) 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 -- 172.21.15.105:0/3531171801 learned_addr learned my addr 172.21.15.105:0/3531171801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff40a4080 con 0x7faff40a6a30 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7faff40a7c40 con 0x7faff40a7520 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7faff40a37e0 con 0x7faff40a4ab0 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a4ab0 0x7faff413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffbf55640 10 monclient(hunting): authenticate will time out at 968.222903s 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): get_auth_request con 0x7faff40a7520 auth_method 0 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): get_auth_request con 0x7faff40a4ab0 auth_method 0 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7f7e54a2c640 10 monclient: get_monmap_and_config 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7f7e54a2c640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7f7e54a2c640 10 monclient: monmap: 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.966 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:04.959254+0000 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:04.959254+0000 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.967 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 -- 172.21.15.105:0/3531171801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a4ab0 msgr2=0x7faff413ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a4ab0 0x7faff413ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 -- 172.21.15.105:0/3531171801 >> v1:172.21.15.160:6789/0 conn(0x7faff40a6a30 legacy=0x7faff40a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 monclient: found mon.noname-a 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.952+0000 7faffa4f2640 1 -- 172.21.15.105:0/3531171801 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff40a3ac0 con 0x7faff40a7520 2024-09-17T16:28:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 Processor -- start 2024-09-17T16:28:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 -- start start 2024-09-17T16:28:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient: init 2024-09-17T16:28:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient: _add_conns ranks=[8,9,2,4,0,3,10,5,6,1,7] 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7e500a6a30 0x7f7e500a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-e con 0x7f7e500a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e500a4ab0 0x7f7e500a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-a con 0x7f7e500a4ab0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e501397f0 0x7f7e50139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-b con 0x7f7e501397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7e500a7c40 con 0x7f7e500a4ab0 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7e500a7890 con 0x7f7e501397f0 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f7e500a4080 con 0x7f7e500a6a30 2024-09-17T16:28:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e54a2c640 10 monclient(hunting): authenticate will time out at 968.224869s 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e500a4ab0 0x7f7e500a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7e500a4ab0 0x7f7e500a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45180/0 (socket says 172.21.15.105:45180) 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 -- 172.21.15.105:0/1956345509 learned_addr learned my addr 172.21.15.105:0/1956345509 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): get_auth_request con 0x7f7e500a4ab0 auth_method 0 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e501397f0 0x7f7e50139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): get_auth_request con 0x7f7e501397f0 auth_method 0 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7e500a6a30 0x7f7e500a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): get_auth_request con 0x7f7e500a6a30 auth_method 0 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffa4f2640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faff40a7520 0x7faff4140f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fafec009e50 tx=0x7fafec01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af2e95e17ba428 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_done global_id 4284 payload 274 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 -- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7e500a6a30 msgr2=0x7f7e500a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7e500a6a30 0x7f7e500a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 -- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e501397f0 msgr2=0x7f7e50139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e501397f0 0x7f7e50139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 monclient: found mon.noname-a 2024-09-17T16:28:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7f7e4dd9b640 1 -- 172.21.15.105:0/1956345509 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e500a37e0 con 0x7f7e500a4ab0 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 1 -- 172.21.15.105:0/3531171801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafec0090d0 con 0x7faff40a7520 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: dump: 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.977 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 1 -- 172.21.15.105:0/3531171801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0a5c40 con 0x7faff40a7520 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 5 monclient: authenticate success, global_id 4275 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 1 -- 172.21.15.105:0/3531171801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafec0a5e30 con 0x7faff40a7520 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 4 set_mon_vals no callback set 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faff8cef640 10 monclient: dump: 2024-09-17T16:28:04.978 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 10 monclient: shutdown 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faff40a7520 msgr2=0x7faff4140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faff40a7520 0x7faff4140f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fafec009e50 tx=0x7fafec01c8b0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 shutdown_connections 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faff40a7520 0x7faff4140f80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 --2- 172.21.15.105:0/3531171801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a4ab0 0x7faff413ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 >> 172.21.15.105:0/3531171801 conn(0x7faff40a8d00 msgr2=0x7faff40a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 shutdown_connections 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 1 -- 172.21.15.105:0/3531171801 wait complete. 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 10 monclient: build_initial_monmap 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.956+0000 7faffbf55640 10 monclient: monmap: 2024-09-17T16:28:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 librados: starting msgr at 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 librados: starting objecter 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 Processor -- start 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- start start 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 librados: setting wanted keys 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 librados: calling monclient init 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: init 2024-09-17T16:28:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e500a4ab0 0x7f7e500a7350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e4401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e82bcf6587e7c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _add_conns ranks=[6,1,3,4,0,8,7,2,5] 2024-09-17T16:28:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 0x7faff41261c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-d con 0x7faff40a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faff40a6a30 0x7faff4126700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-f con 0x7faff40a6a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a7520 0x7faff4126c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): picked mon.noname-g con 0x7faff40a7520 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 1 -- 172.21.15.105:0/1956345509 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e440b0070 con 0x7f7e500a4ab0 2024-09-17T16:28:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 0x7faff41261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faff4143e60 con 0x7faff40a6a30 2024-09-17T16:28:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 0x7faff41261c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33896/0 (socket says 172.21.15.105:33896) 2024-09-17T16:28:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 -- 172.21.15.105:0/3955978936 learned_addr learned my addr 172.21.15.105:0/3955978936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: dump: 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 1 -- 172.21.15.105:0/1956345509 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e440aa810 con 0x7f7e500a4ab0 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 1 -- 172.21.15.105:0/1956345509 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e440a9300 con 0x7f7e500a4ab0 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e4d59a640 10 monclient: dump: 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7faff4143ce0 con 0x7faff40a7520 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7faff4143fe0 con 0x7faff40a4ab0 2024-09-17T16:28:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 5 monclient: authenticate success, global_id 4284 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient(hunting): authenticate will time out at 968.229340s 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): get_auth_request con 0x7faff40a4ab0 auth_method 0 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 4 set_mon_vals no callback set 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faff40a6a30 0x7faff4126700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a7520 0x7faff4126c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): get_auth_request con 0x7faff40a6a30 auth_method 0 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): get_auth_request con 0x7faff40a7520 auth_method 0 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffa4f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff94f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient(hunting): handle_auth_done global_id 4254 payload 824 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 -- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a7520 msgr2=0x7faff4126c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faff40a7520 0x7faff4126c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 -- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faff40a6a30 msgr2=0x7faff4126700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faff40a6a30 0x7faff4126700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient: _finish_hunting 0 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 monclient: found mon.noname-d 2024-09-17T16:28:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff4129010 con 0x7faff40a4ab0 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faff9cf1640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 0x7faff41261c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faff000c4e0 tx=0x7faff000c9a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 client.?.objecter ms_handle_connect 0x7faff40a4ab0 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 10 monclient: shutdown 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 -- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e500a4ab0 msgr2=0x7f7e500a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:04.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e500a4ab0 0x7f7e500a7350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e4401c3d0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff0012630 con 0x7faff40a4ab0 2024-09-17T16:28:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: dump: 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 -- 172.21.15.105:0/1956345509 shutdown_connections 2024-09-17T16:28:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7e501397f0 0x7f7e50139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff0012810 con 0x7faff40a4ab0 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff0012e30 con 0x7faff40a4ab0 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e500a4ab0 0x7f7e500a7350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/1956345509 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7e500a6a30 0x7f7e500a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 5 monclient: authenticate success, global_id 4254 2024-09-17T16:28:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 -- 172.21.15.105:0/1956345509 >> 172.21.15.105:0/1956345509 conn(0x7f7e500a8d00 msgr2=0x7f7e500a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafebfff640 4 set_mon_vals no callback set 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7fafea7fc640 10 monclient: dump: 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _renew_subs 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff41295e0 con 0x7faff40a4ab0 2024-09-17T16:28:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 client.4254.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 -- 172.21.15.105:0/1956345509 shutdown_connections 2024-09-17T16:28:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7f7e54a2c640 1 -- 172.21.15.105:0/1956345509 wait complete. 2024-09-17T16:28:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _renew_subs 2024-09-17T16:28:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7faff4129e80 con 0x7faff40a4ab0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.960+0000 7faffbf55640 1 librados: init done 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7faff00129c0 con 0x7faff40a4ab0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 10 client.4254.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 15 client.4254.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 20 client.4254.objecter _get_session osd=-1 returning homeless 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 10 client.4254.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faff406dc60 con 0x7faff40a4ab0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafd803e550 0x7fafd8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7faff0050d80 con 0x7faff40a4ab0 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter ms_dispatch 0x7faff4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 3 client.4254.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 3 client.4254.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter checking command 1 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fafd80443c0 0x7fafd8046800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter _get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 15 client.4254.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter _get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 15 client.4254.objecter _session_command_op_assign 3 1 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 4 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter _send_command 1 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faff0010e50 con 0x7fafd80443c0 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7faff0054060 con 0x7faff40a4ab0 2024-09-17T16:28:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-17T16:28:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faff94f0640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafd803e550 0x7fafd8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faff94f0640 10 monclient: get_auth_request con 0x7fafd803e550 auth_method 0 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: monmap: 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffa4f2640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fafd80443c0 0x7fafd8046800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffa4f2640 10 monclient: get_auth_request con 0x7fafd80443c0 auth_method 0 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 librados: starting msgr at 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 librados: starting objecter 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 Processor -- start 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- start start 2024-09-17T16:28:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 librados: setting wanted keys 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 librados: calling monclient init 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: init 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faff94f0640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafd803e550 0x7fafd8040a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faff4127920 tx=0x7fafe4009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter ms_handle_connect 0x7fafd803e550 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter ms_handle_connect 0x7fafd80443c0 2024-09-17T16:28:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7faffa4f2640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fafd80443c0 0x7fafd8046800 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _add_conns ranks=[5,6,4,8,1,7,3,2,0] 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7e500a4ab0 0x7f7e501d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-h con 0x7f7e500a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 0x7f7e501d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-d con 0x7f7e500a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7e501397f0 0x7f7e5013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): picked mon.noname-c con 0x7f7e501397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7e5006e770 con 0x7f7e501397f0 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f7e500a2fc0 con 0x7f7e500a4ab0 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f7e50143260 con 0x7f7e500a6a30 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient(hunting): authenticate will time out at 968.234594s 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 0x7f7e501d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7e501397f0 0x7f7e5013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 0x7f7e501d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33902/0 (socket says 172.21.15.105:33902) 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 1 -- 172.21.15.105:0/207657501 learned_addr learned my addr 172.21.15.105:0/207657501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): get_auth_request con 0x7f7e500a6a30 auth_method 0 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): get_auth_request con 0x7f7e501397f0 auth_method 0 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7e500a4ab0 0x7f7e501d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): get_auth_request con 0x7f7e500a4ab0 auth_method 0 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient(hunting): handle_auth_done global_id 4234 payload 824 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 -- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 msgr2=0x7f7e501d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 0x7f7e501d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 -- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7e500a4ab0 msgr2=0x7f7e501d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7e500a4ab0 0x7f7e501d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 monclient: found mon.noname-c 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e50123ab0 con 0x7f7e501397f0 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7e500a6a30 0x7f7e501d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4ed9d640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7e501397f0 0x7f7e5013d350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb80 tx=0x7f7e4800c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 client.?.objecter ms_handle_connect 0x7f7e501397f0 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e48018020 con 0x7f7e501397f0 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: dump: 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e48014480 con 0x7f7e501397f0 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e48014aa0 con 0x7f7e501397f0 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e3effd640 10 monclient: dump: 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 5 monclient: authenticate success, global_id 4234 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _renew_subs 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e500ab990 con 0x7f7e501397f0 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 client.4234.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _renew_subs 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501234f0 con 0x7f7e501397f0 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e54a2c640 1 librados: init done 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7f7e4cd99640 4 set_mon_vals no callback set 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== osd.3 v2:172.21.15.160:6808/3218035673 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faff0010e50 con 0x7fafd80443c0 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter ms_dispatch 0x7faff4000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 10 client.4254.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 4 2024-09-17T16:28:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.964+0000 7fafea7fc640 15 client.4254.objecter _session_command_op_remove 3 1 2024-09-17T16:28:05.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 10 client.4234.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 15 client.4234.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 20 client.4234.objecter _get_session osd=-1 returning homeless 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 10 client.4234.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e34002920 con 0x7f7e501397f0 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7e4801b020 con 0x7f7e501397f0 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7e3003e550 0x7f7e30040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f7e48050440 con 0x7f7e501397f0 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 client.4234.objecter ms_dispatch 0x7f7e50000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 3 client.4234.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 3 client.4234.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 client.4234.objecter checking command 1 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f7e300443c0 0x7f7e30046800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter _get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 15 client.4234.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter _get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 15 client.4234.objecter _session_command_op_assign 4 1 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 4 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 client.4234.objecter _send_command 1 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e48021e80 con 0x7f7e300443c0 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 20 client.4234.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7e48057050 con 0x7f7e501397f0 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4e59c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7e3003e550 0x7f7e30040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4e59c640 10 monclient: get_auth_request con 0x7f7e3003e550 auth_method 0 2024-09-17T16:28:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f7e300443c0 0x7f7e30046800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4dd9b640 10 monclient: get_auth_request con 0x7f7e300443c0 auth_method 0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4e59c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7e3003e550 0x7f7e30040a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009b60 tx=0x7f7e40009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 client.4234.objecter ms_handle_connect 0x7f7e3003e550 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e3effd640 10 client.4234.objecter ms_handle_connect 0x7f7e300443c0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.968+0000 7f7e4dd9b640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f7e300443c0 0x7f7e30046800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.972+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== osd.4 v2:172.21.15.160:6816/639753971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e48021e80 con 0x7f7e300443c0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.972+0000 7f7e3effd640 10 client.4234.objecter ms_dispatch 0x7f7e50000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.972+0000 7f7e3effd640 10 client.4234.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.972+0000 7f7e3effd640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 4 2024-09-17T16:28:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.972+0000 7f7e3effd640 15 client.4234.objecter _session_command_op_remove 4 1 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 client.4254.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 15 client.4254.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter _get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter _get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 15 client.4254.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter get_session s=0x7fafd8043cf0 osd=3 3 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 15 client.4254.objecter _session_command_op_assign 3 2 2024-09-17T16:28:05.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 4 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 client.4254.objecter _send_command 2 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 --> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faff4129a50 con 0x7fafd80443c0 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7fafea7fc640 1 -- 172.21.15.105:0/3955978936 <== osd.3 v2:172.21.15.160:6808/3218035673 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fafec000fc0 con 0x7fafd80443c0 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7fafea7fc640 10 client.4254.objecter ms_dispatch 0x7faff4000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7fafea7fc640 10 client.4254.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7fafea7fc640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 4 2024-09-17T16:28:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7fafea7fc640 15 client.4254.objecter _session_command_op_remove 3 2 2024-09-17T16:28:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 librados: watch_flush enter 2024-09-17T16:28:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 librados: watch_flush exit 2024-09-17T16:28:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 client.4254.objecter close_session for osd.3 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fafd80443c0 msgr2=0x7fafd8046800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fafd80443c0 0x7fafd8046800 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter put_session s=0x7fafd8043cf0 osd=3 1 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 client.4254.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 client.4254.objecter successfully canceled tick 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafd803e550 msgr2=0x7fafd8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafd803e550 0x7fafd8040a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faff4127920 tx=0x7fafe4009190 comp rx=0 tx=0).stop 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 10 monclient: shutdown 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 msgr2=0x7faff41261c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.980+0000 7faffbf55640 1 --2- 172.21.15.105:0/3955978936 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faff40a4ab0 0x7faff41261c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faff000c4e0 tx=0x7faff000c9a0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffa4f2640 1 -- 172.21.15.105:0/3955978936 reap_dead start 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 shutdown_connections 2024-09-17T16:28:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 >> 172.21.15.105:0/3955978936 conn(0x7faff40a8d00 msgr2=0x7faff40aabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 shutdown_connections 2024-09-17T16:28:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffbf55640 1 -- 172.21.15.105:0/3955978936 wait complete. 2024-09-17T16:28:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.984+0000 7faffbf55640 1 librados: shutdown 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.992+0000 7f6f3aeaf640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.992+0000 7f6f3aeaf640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.992+0000 7f6f3aeaf640 10 monclient: monmap: 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:04.999571+0000 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:04.999571+0000 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.033 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 Processor -- start 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 -- start start 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient: init 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient: _add_conns ranks=[6,1,7,4,0,10,8,9,2,3,5] 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-d con 0x7f6f340a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6f340a5c40 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6f3413ea30 0x7f6f34140e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-i con 0x7f6f3413ea30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3406dbd0 con 0x7f6f340a5c40 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6f3406e7e0 con 0x7f6f340a5860 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6f340745f0 con 0x7f6f3413ea30 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3aeaf640 10 monclient(hunting): authenticate will time out at 968.265193s 2024-09-17T16:28:05.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33910/0 (socket says 172.21.15.105:33910) 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f33fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6f340a5c40 0x7f6f340a6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:55396/0 (socket says 172.21.15.105:55396) 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6f3413ea30 0x7f6f34140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f33fff640 1 -- 172.21.15.105:0/2193645073 learned_addr learned my addr 172.21.15.105:0/2193645073 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): get_auth_request con 0x7f6f340a5860 auth_method 0 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): get_auth_request con 0x7f6f3413ea30 auth_method 0 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_done global_id 4272 payload 274 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 -- 172.21.15.105:0/2193645073 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6f3413ea30 msgr2=0x7f6f34140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 --2- 172.21.15.105:0/2193645073 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6f3413ea30 0x7f6f34140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 -- 172.21.15.105:0/2193645073 >> v1:172.21.15.160:6789/0 conn(0x7f6f340a5c40 legacy=0x7f6f340a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f337fe640 1 -- 172.21.15.105:0/2193645073 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581757027 0 0) 0x7f6f3406dbd0 con 0x7f6f340a5c40 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 monclient: found mon.noname-d 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f38c4b640 1 -- 172.21.15.105:0/2193645073 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f340a14f0 con 0x7f6f340a5860 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f6f337fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 10 client.4234.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 15 client.4234.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 20 client.4234.objecter _get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 20 client.4234.objecter _get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 15 client.4234.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 20 client.4234.objecter get_session s=0x7f7e30043cf0 osd=4 3 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 15 client.4234.objecter _session_command_op_assign 4 2 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 4 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 10 client.4234.objecter _send_command 2 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 --> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e34001230 con 0x7f7e300443c0 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e3effd640 1 -- 172.21.15.105:0/207657501 <== osd.4 v2:172.21.15.160:6816/639753971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7e34001230 con 0x7f7e300443c0 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e3effd640 10 client.4234.objecter ms_dispatch 0x7f7e50000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e3effd640 10 client.4234.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e3effd640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 4 2024-09-17T16:28:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:04.996+0000 7f7e3effd640 15 client.4234.objecter _session_command_op_remove 4 2 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 10 librados: watch_flush enter 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 10 librados: watch_flush exit 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 10 client.4234.objecter close_session for osd.4 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f7e300443c0 msgr2=0x7f7e30046800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f7e300443c0 0x7f7e30046800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 20 client.4234.objecter put_session s=0x7f7e30043cf0 osd=4 1 2024-09-17T16:28:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 20 client.4234.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 10 client.4234.objecter successfully canceled tick 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7e3003e550 msgr2=0x7f7e30040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7e3003e550 0x7f7e30040a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009b60 tx=0x7f7e40009190 comp rx=0 tx=0).stop 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 10 monclient: shutdown 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7e501397f0 msgr2=0x7f7e5013d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 --2- 172.21.15.105:0/207657501 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7e501397f0 0x7f7e5013d350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb80 tx=0x7f7e4800c490 comp rx=0 tx=0).stop 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e4ed9d640 1 -- 172.21.15.105:0/207657501 reap_dead start 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 shutdown_connections 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 >> 172.21.15.105:0/207657501 conn(0x7f7e500a8d00 msgr2=0x7f7e500a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f38c4b640 1 --2- 172.21.15.105:0/2193645073 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f280097f0 tx=0x7f6f2801ce00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=29c6c411c9ca4fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 shutdown_connections 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 -- 172.21.15.105:0/207657501 wait complete. 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 1 -- 172.21.15.105:0/2193645073 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f280b2070 con 0x7f6f340a5860 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: dump: 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 5 monclient: authenticate success, global_id 4272 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f7e54a2c640 1 librados: shutdown 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 1 -- 172.21.15.105:0/2193645073 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f280ab380 con 0x7f6f340a5860 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 1 -- 172.21.15.105:0/2193645073 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f280b2070 con 0x7f6f340a5860 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f337fe640 10 monclient: dump: 2024-09-17T16:28:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.048 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 4 set_mon_vals no callback set 2024-09-17T16:28:05.048 INFO:teuthology.orchestra.run.smithi105.stdout:51539607557 2024-09-17T16:28:05.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T16:28:05.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 10 monclient: shutdown 2024-09-17T16:28:05.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/2193645073 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 msgr2=0x7f6f340a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/2193645073 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f280097f0 tx=0x7f6f2801ce00 comp rx=0 tx=0).stop 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/2193645073 shutdown_connections 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/2193645073 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6f3413ea30 0x7f6f34140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/2193645073 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f340a1ef0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/2193645073 >> 172.21.15.105:0/2193645073 conn(0x7f6f340aa450 msgr2=0x7f6f340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/2193645073 shutdown_connections 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/2193645073 wait complete. 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 10 monclient: monmap: 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 librados: starting msgr at 2024-09-17T16:28:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 librados: starting objecter 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.000+0000 7f6f3aeaf640 1 Processor -- start 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- start start 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 librados: setting wanted keys 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 librados: calling monclient init 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: init 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _add_conns ranks=[6,5,3,8,4,1,7,0,2] 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f341248b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-d con 0x7f6f340a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f340a5c40 0x7f6f34124df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-h con 0x7f6f340a5c40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f3413ea30 0x7f6f34125330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): picked mon.noname-g con 0x7f6f3413ea30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6f34143f10 con 0x7f6f3413ea30 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6f34143d90 con 0x7f6f340a5c40 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6f34144090 con 0x7f6f340a5860 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient(hunting): authenticate will time out at 968.272741s 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f341248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f341248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33926/0 (socket says 172.21.15.105:33926) 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 -- 172.21.15.105:0/3933400524 learned_addr learned my addr 172.21.15.105:0/3933400524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): get_auth_request con 0x7f6f340a5860 auth_method 0 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f3413ea30 0x7f6f34125330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): get_auth_request con 0x7f6f3413ea30 auth_method 0 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3944c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f33fff640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f340a5c40 0x7f6f34124df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f33fff640 10 monclient(hunting): get_auth_request con 0x7f6f340a5c40 auth_method 0 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f33fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f33fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 -- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f340a5c40 msgr2=0x7f6f34124df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f340a5c40 0x7f6f34124df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 -- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f3413ea30 msgr2=0x7f6f34125330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f3413ea30 0x7f6f34125330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 monclient: found mon.noname-d 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3413c410 con 0x7f6f340a5860 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f38c4b640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f341248b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6f2801aa50 tx=0x7f6f28019140 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 client.?.objecter ms_handle_connect 0x7f6f340a5860 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f280ba020 con 0x7f6f340a5860 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: dump: 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f28003a90 con 0x7f6f340a5860 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 5 monclient: authenticate success, global_id 4281 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _renew_subs 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3413bde0 con 0x7f6f340a5860 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f280b2070 con 0x7f6f340a5860 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f317fa640 10 monclient: dump: 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _renew_subs 2024-09-17T16:28:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f32ffd640 4 set_mon_vals no callback set 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f340a3a70 con 0x7f6f340a5860 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 librados: init done 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 client.4281.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 15 client.4281.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 20 client.4281.objecter _get_session osd=-1 returning homeless 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.004+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f340a3580 con 0x7f6f340a5860 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6f280bd020 con 0x7f6f340a5860 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f1803e550 0x7f6f18040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f6f280e9840 con 0x7f6f340a5860 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter ms_dispatch 0x7f6f34000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 3 client.4281.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 3 client.4281.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter checking command 1 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f6f180443c0 0x7f6f18046800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter _get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 15 client.4281.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter _get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 15 client.4281.objecter _session_command_op_assign 2 1 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 4 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter _send_command 1 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f280c0e50 con 0x7f6f180443c0 2024-09-17T16:28:05.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6f280e9ca0 con 0x7f6f340a5860 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f3944c640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f6f180443c0 0x7f6f18046800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f33fff640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f1803e550 0x7f6f18040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f3944c640 10 monclient: get_auth_request con 0x7f6f180443c0 auth_method 0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f33fff640 10 monclient: get_auth_request con 0x7f6f1803e550 auth_method 0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter ms_handle_connect 0x7f6f180443c0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f3944c640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f6f180443c0 0x7f6f18046800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f33fff640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f1803e550 0x7f6f18040a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f3409f650 tx=0x7f6f20007b60 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter ms_handle_connect 0x7f6f1803e550 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== osd.2 v2:172.21.15.105:6808/1006010180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f280c0e50 con 0x7f6f180443c0 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter ms_dispatch 0x7f6f34000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 10 client.4281.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 4 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.008+0000 7f6f317fa640 15 client.4281.objecter _session_command_op_remove 2 1 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: monmap: 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.016239+0000 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.016239+0000 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: monmap: 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.016247+0000 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.016247+0000 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.071 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 Processor -- start 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 -- start start 2024-09-17T16:28:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: init 2024-09-17T16:28:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 Processor -- start 2024-09-17T16:28:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 -- start start 2024-09-17T16:28:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: init 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient: _add_conns ranks=[8,6,3,0,2,1,5,10,9,7,4] 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 0x7efda40a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-e con 0x7efda40a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 0x7efda40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-d con 0x7efda409e740 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e0b0 0x7efda4136500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-g con 0x7efda409e0b0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 0x7efda40a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 0x7efda40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient: _add_conns ranks=[9,8,10,4,2,1,3,0,7,6,5] 2024-09-17T16:28:05.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14e009f8e0 0x7f14e009b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 0x7efda40a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34114/0 (socket says 172.21.15.105:34114) 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 1 -- 172.21.15.105:0/991996744 learned_addr learned my addr 172.21.15.105:0/991996744 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efda409ef60 con 0x7efda409e0b0 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efda40a31c0 con 0x7efda409e740 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7efda40a55b0 con 0x7efda40a4750 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efdab8df640 10 monclient(hunting): authenticate will time out at 968.282164s 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): get_auth_request con 0x7efda409e740 auth_method 0 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-a con 0x7f14e009f8e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 0x7f14e009b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14e009f8e0 0x7f14e009b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-e con 0x7f14e00a46f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f14e009f8e0 0x7f14e009b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45190/0 (socket says 172.21.15.105:45190) 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 -- 172.21.15.105:0/3457528771 learned_addr learned my addr 172.21.15.105:0/3457528771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3457528771 >> v2:172.21.15.160:3300/0 conn(0x7f14e009bad0 0x7f14e009fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): get_auth_request con 0x7efda40a4750 auth_method 0 2024-09-17T16:28:05.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e0b0 0x7efda4136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): get_auth_request con 0x7efda409e0b0 auth_method 0 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-f con 0x7f14e009bad0 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f14e00a1a50 con 0x7f14e009f8e0 2024-09-17T16:28:05.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f14e00a1ca0 con 0x7f14e009bad0 2024-09-17T16:28:05.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f14e00a4c00 con 0x7f14e00a46f0 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 -- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 msgr2=0x7efda40a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 0x7efda40a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14e5a69640 10 monclient(hunting): authenticate will time out at 968.282741s 2024-09-17T16:28:05.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 -- 172.21.15.105:0/991996744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e0b0 msgr2=0x7efda4136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e0b0 0x7efda4136500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 monclient: found mon.noname-d 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7efda8e7a640 1 -- 172.21.15.105:0/991996744 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40a0a80 con 0x7efda409e740 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): get_auth_request con 0x7f14e009f8e0 auth_method 0 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 10 monclient: monmap: 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.018258+0000 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.018258+0000 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.084 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.085 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 1 --2- 172.21.15.105:0/3457528771 >> v2:172.21.15.160:3300/0 conn(0x7f14e009bad0 0x7f14e009fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): get_auth_request con 0x7f14e009bad0 auth_method 0 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14deffd640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 0x7f14e009b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient(hunting): handle_auth_done global_id 4293 payload 274 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 -- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 msgr2=0x7f14e009b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 0x7f14e009b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 -- 172.21.15.105:0/3457528771 >> v2:172.21.15.160:3300/0 conn(0x7f14e009bad0 msgr2=0x7f14e009fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3457528771 >> v2:172.21.15.160:3300/0 conn(0x7f14e009bad0 0x7f14e009fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 monclient: found mon.noname-a 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14df7fe640 1 -- 172.21.15.105:0/3457528771 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e009f3f0 con 0x7f14e009f8e0 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14deffd640 10 monclient: get_auth_request con 0x7f14e00a46f0 auth_method 0 2024-09-17T16:28:05.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7f14deffd640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 0x7f14e009b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:05.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 1 Processor -- start 2024-09-17T16:28:05.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 1 -- start start 2024-09-17T16:28:05.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.012+0000 7fdb2cc10640 10 monclient: init 2024-09-17T16:28:05.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient: _add_conns ranks=[9,7,2,6,1,4,3,10,8,5,0] 2024-09-17T16:28:05.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb280a6a30 0x7fdb280a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-a con 0x7fdb280a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a4ab0 0x7fdb280a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-i con 0x7fdb280a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 0x7fdb28139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-b con 0x7fdb281397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdb280a7c40 con 0x7fdb280a6a30 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdb280a7890 con 0x7fdb281397f0 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdb280a4080 con 0x7fdb280a4ab0 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2cc10640 10 monclient(hunting): authenticate will time out at 968.284485s 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 0x7fdb28139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb280a6a30 0x7fdb280a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 0x7fdb28139ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59462/0 (socket says 172.21.15.105:59462) 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a4ab0 0x7fdb280a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 1 -- 172.21.15.105:0/1684561112 learned_addr learned my addr 172.21.15.105:0/1684561112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 10 monclient(hunting): get_auth_request con 0x7fdb281397f0 auth_method 0 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb26d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): get_auth_request con 0x7fdb280a6a30 auth_method 0 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): get_auth_request con 0x7fdb280a4ab0 auth_method 0 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient(hunting): handle_auth_done global_id 4302 payload 274 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 -- 172.21.15.105:0/1684561112 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a4ab0 msgr2=0x7fdb280a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a4ab0 0x7fdb280a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 -- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 msgr2=0x7fdb28139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 0x7fdb28139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 monclient: found mon.noname-a 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 -- 172.21.15.105:0/1684561112 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb280a37e0 con 0x7fdb280a6a30 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efda8e7a640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 0x7efda40a5070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd94009820 tx=0x7efd9401e2d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=305bd5f4f909f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 1 -- 172.21.15.105:0/991996744 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd940b2070 con 0x7efda409e740 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: dump: 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.096 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 1 -- 172.21.15.105:0/991996744 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd940ac810 con 0x7efda409e740 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 1 -- 172.21.15.105:0/991996744 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd940ab300 con 0x7efda409e740 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efd9bfff640 10 monclient: dump: 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 5 monclient: authenticate success, global_id 4290 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 4 set_mon_vals no callback set 2024-09-17T16:28:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14e009f8e0 0x7f14e009b050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f14e00a93d0 tx=0x7f14d001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789cff16b3ec459d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 1 -- 172.21.15.105:0/3457528771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14d00b0070 con 0x7f14e009f8e0 2024-09-17T16:28:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: dump: 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.103 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 1 -- 172.21.15.105:0/3457528771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d00aa810 con 0x7f14e009f8e0 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 5 monclient: authenticate success, global_id 4293 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 4 set_mon_vals no callback set 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 1 -- 172.21.15.105:0/3457528771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14d00a9300 con 0x7f14e009f8e0 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14de7fc640 10 monclient: dump: 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.106 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 10 monclient: shutdown 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 msgr2=0x7efda40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 0x7efda40a5070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd94009820 tx=0x7efd9401e2d0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 shutdown_connections 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e0b0 0x7efda4136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efda409e740 0x7efda40a5070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 --2- 172.21.15.105:0/991996744 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efda40a4750 0x7efda40a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 >> 172.21.15.105:0/991996744 conn(0x7efda40a5840 msgr2=0x7efda40a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 shutdown_connections 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 1 -- 172.21.15.105:0/991996744 wait complete. 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 10 monclient: shutdown 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14e009f8e0 msgr2=0x7f14e009b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14e009f8e0 0x7f14e009b050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f14e00a93d0 tx=0x7f14d001c3e0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 shutdown_connections 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14e00a46f0 0x7f14e009b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3457528771 >> v2:172.21.15.160:3300/0 conn(0x7f14e009bad0 0x7f14e009fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3457528771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14e009f8e0 0x7f14e009b050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 >> 172.21.15.105:0/3457528771 conn(0x7f14e00a83e0 msgr2=0x7f14e00a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 shutdown_connections 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb2659c640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280a6a30 0x7fdb280a6e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009a50 tx=0x7fdb1801c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65aec73fd4b21e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7f14e5a69640 1 -- 172.21.15.105:0/3457528771 wait complete. 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7efdab8df640 10 monclient: monmap: 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.110 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.111 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 1 -- 172.21.15.105:0/1684561112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb180a9860 con 0x7fdb280a6a30 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 10 monclient: dump: 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.112 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.016+0000 7fdb25d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 1 -- 172.21.15.105:0/1684561112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb180a9a20 con 0x7fdb280a6a30 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 1 -- 172.21.15.105:0/1684561112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb180a9c10 con 0x7fdb280a6a30 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb25d9b640 10 monclient: dump: 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.114 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 5 monclient: authenticate success, global_id 4302 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 4 set_mon_vals no callback set 2024-09-17T16:28:05.116 INFO:teuthology.orchestra.run.smithi105.stdout:51539607557 2024-09-17T16:28:05.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T16:28:05.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 librados: starting msgr at 2024-09-17T16:28:05.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 librados: starting objecter 2024-09-17T16:28:05.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient: monmap: 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.118 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 Processor -- start 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- start start 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 librados: setting wanted keys 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 librados: calling monclient init 2024-09-17T16:28:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: init 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 librados: starting msgr at 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 librados: starting objecter 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 Processor -- start 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 -- start start 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 librados: setting wanted keys 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 librados: calling monclient init 2024-09-17T16:28:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient: init 2024-09-17T16:28:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: shutdown 2024-09-17T16:28:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280a6a30 msgr2=0x7fdb280a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280a6a30 0x7fdb280a6e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009a50 tx=0x7fdb1801c590 comp rx=0 tx=0).stop 2024-09-17T16:28:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/1684561112 shutdown_connections 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb281397f0 0x7fdb28139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a4ab0 0x7fdb280a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/1684561112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb280a6a30 0x7fdb280a6e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/1684561112 >> 172.21.15.105:0/1684561112 conn(0x7fdb280a8d00 msgr2=0x7fdb280a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/1684561112 shutdown_connections 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/1684561112 wait complete. 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _add_conns ranks=[2,3,7,5,8,6,4,0,1] 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 0x7efda4072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-b con 0x7efda409e0b0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e740 0x7efda4072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-g con 0x7efda409e740 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efda40a4750 0x7efda40732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): picked mon.noname-i con 0x7efda40a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efda406c4e0 con 0x7efda409e0b0 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efda406c360 con 0x7efda409e740 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7efda406c660 con 0x7efda40a4750 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient(hunting): authenticate will time out at 968.289058s 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 0x7efda4072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 0x7efda4072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59476/0 (socket says 172.21.15.105:59476) 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 -- 172.21.15.105:0/3887999123 learned_addr learned my addr 172.21.15.105:0/3887999123 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): get_auth_request con 0x7efda409e0b0 auth_method 0 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e740 0x7efda4072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efda40a4750 0x7efda40732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): get_auth_request con 0x7efda409e740 auth_method 0 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): get_auth_request con 0x7efda40a4750 auth_method 0 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: monmap: 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.127 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda8e7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda9e7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient(hunting): handle_auth_done global_id 4250 payload 824 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 -- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efda40a4750 msgr2=0x7efda40732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efda40a4750 0x7efda40732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 -- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e740 msgr2=0x7efda4072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 librados: starting msgr at 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 librados: starting objecter 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efda409e740 0x7efda4072d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 monclient: found mon.noname-b 2024-09-17T16:28:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda41218a0 con 0x7efda409e0b0 2024-09-17T16:28:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 Processor -- start 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient: _add_conns ranks=[2,5,1,4,0,6,7,8,3] 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 0x7f14e0139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-b con 0x7f14e009bad0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f14e009f8e0 0x7f14e013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-h con 0x7f14e009f8e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14e00a46f0 0x7f14e013a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): picked mon.noname-f con 0x7f14e00a46f0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00a3980 con 0x7f14e00a46f0 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f14e00a3800 con 0x7f14e009bad0 2024-09-17T16:28:05.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f14e00a3b00 con 0x7f14e009f8e0 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14e5a69640 10 monclient(hunting): authenticate will time out at 968.289380s 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 0x7f14e0139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 0x7f14e0139de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59490/0 (socket says 172.21.15.105:59490) 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 -- 172.21.15.105:0/3170588503 learned_addr learned my addr 172.21.15.105:0/3170588503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): get_auth_request con 0x7f14e009bad0 auth_method 0 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14e00a46f0 0x7f14e013a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f14e009f8e0 0x7f14e013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): get_auth_request con 0x7f14e00a46f0 auth_method 0 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): get_auth_request con 0x7f14e009f8e0 auth_method 0 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- start start 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 librados: setting wanted keys 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 librados: calling monclient init 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: init 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient(hunting): handle_auth_done global_id 4259 payload 824 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 -- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f14e009f8e0 msgr2=0x7f14e013a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f14e009f8e0 0x7f14e013a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 -- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14e00a46f0 msgr2=0x7f14e013a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14e00a46f0 0x7f14e013a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 monclient: found mon.noname-b 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e009de30 con 0x7f14e009bad0 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efda967b640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 0x7efda4072840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efda000e7f0 tx=0x7efda000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 client.?.objecter ms_handle_connect 0x7efda409e0b0 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda0017070 con 0x7efda409e0b0 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14df7fe640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 0x7f14e0139de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14d00ae040 tx=0x7f14d0019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 client.?.objecter ms_handle_connect 0x7f14e009bad0 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: dump: 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.137 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda000cdb0 con 0x7efda409e0b0 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda00138e0 con 0x7efda409e0b0 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: _add_conns ranks=[4,7,3,6,2,0,8,5,1] 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 0x7fdb28122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-c con 0x7fdb280a4ab0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a6a30 0x7fdb281d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-i con 0x7fdb280a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb281397f0 0x7fdb281d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): picked mon.noname-g con 0x7fdb281397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2806e770 con 0x7fdb281397f0 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd9b7fe640 4 set_mon_vals no callback set 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efd99ffb640 10 monclient: dump: 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.140 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.141 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdb280a2fc0 con 0x7fdb280a4ab0 2024-09-17T16:28:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdb28143260 con 0x7fdb280a6a30 2024-09-17T16:28:05.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient(hunting): authenticate will time out at 968.290671s 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 0x7fdb28122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 0x7fdb28122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57576/0 (socket says 172.21.15.105:57576) 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 -- 172.21.15.105:0/2746143708 learned_addr learned my addr 172.21.15.105:0/2746143708 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): get_auth_request con 0x7fdb280a4ab0 auth_method 0 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient(hunting): handle_auth_done global_id 4243 payload 824 2024-09-17T16:28:05.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1ffff640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a6a30 0x7fdb281d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 -- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a6a30 msgr2=0x7fdb281d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a6a30 0x7fdb281d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 -- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb281397f0 msgr2=0x7fdb281d1c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdb281397f0 0x7fdb281d1c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 monclient: found mon.noname-c 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 -- 172.21.15.105:0/2746143708 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb28123ba0 con 0x7fdb280a4ab0 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1ffff640 10 monclient: get_auth_request con 0x7fdb280a6a30 auth_method 0 2024-09-17T16:28:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1ffff640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdb280a6a30 0x7fdb281d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14d00b9070 con 0x7f14e009bad0 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: dump: 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.146 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d00044b0 con 0x7f14e009bad0 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14d00b0040 con 0x7f14e009bad0 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14c7fff640 10 monclient: dump: 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2659c640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 0x7fdb28122680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ab040 tx=0x7fdb1801add0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 client.?.objecter ms_handle_connect 0x7fdb280a4ab0 2024-09-17T16:28:05.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7f14ddffb640 4 set_mon_vals no callback set 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb180bf070 con 0x7fdb280a4ab0 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: dump: 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.150 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 5 monclient: authenticate success, global_id 4243 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb180044b0 con 0x7fdb280a4ab0 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: _renew_subs 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb28123e80 con 0x7fdb280a4ab0 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb180a9810 con 0x7fdb280a4ab0 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2cc10640 10 client.4243.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb2559a640 4 set_mon_vals no callback set 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 5 monclient: authenticate success, global_id 4250 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _renew_subs 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda40a28b0 con 0x7efda409e0b0 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _renew_subs 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efda411f540 con 0x7efda409e0b0 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 librados: init done 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 client.4250.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 15 client.4250.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 20 client.4250.objecter _get_session osd=-1 returning homeless 2024-09-17T16:28:05.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efda4136500 con 0x7efda409e0b0 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 5 monclient: authenticate success, global_id 4259 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 monclient: _renew_subs 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e01329d0 con 0x7f14e009bad0 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 client.4259.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 monclient: _renew_subs 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e013b740 con 0x7f14e009bad0 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 1 librados: init done 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.020+0000 7fdb1effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 monclient: dump: 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.155 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efda001a070 con 0x7efda409e0b0 2024-09-17T16:28:05.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdb18004040 con 0x7fdb280a4ab0 2024-09-17T16:28:05.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 10 monclient: _renew_subs 2024-09-17T16:28:05.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 client.4259.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb280aa040 con 0x7fdb280a4ab0 2024-09-17T16:28:05.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 1 librados: init done 2024-09-17T16:28:05.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 15 client.4259.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 20 client.4259.objecter _get_session osd=-1 returning homeless 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 client.4259.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14e009fcc0 con 0x7f14e009bad0 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdb0803e580 0x7fdb08040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fdb180e9e10 con 0x7fdb280a4ab0 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 client.4243.objecter ms_dispatch 0x7fdb28000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 3 client.4243.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 3 client.4243.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 20 client.4243.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd8803e550 0x7efd88040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7efda000f040 con 0x7efda409e0b0 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 client.4250.objecter ms_dispatch 0x7efda4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 3 client.4250.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 3 client.4250.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 client.4250.objecter checking command 1 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7efd880443c0 0x7efd88046800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter _get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 15 client.4250.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter _get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 15 client.4250.objecter _session_command_op_assign 5 1 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 4 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 client.4250.objecter _send_command 1 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efda0021ec0 con 0x7efd880443c0 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 20 client.4250.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda8e7a640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd8803e550 0x7efd88040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1ffff640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdb0803e580 0x7fdb08040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda8e7a640 10 monclient: get_auth_request con 0x7efd8803e550 auth_method 0 2024-09-17T16:28:05.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1ffff640 10 monclient: get_auth_request con 0x7fdb0803e580 auth_method 0 2024-09-17T16:28:05.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 10 client.4243.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_assign -1 1 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fdb2813d9d0 0x7fdb2813ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 20 client.4243.objecter _get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda9e7c640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7efd880443c0 0x7efd88046800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 20 client.4243.objecter _get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 20 client.4243.objecter get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_assign 1 1 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 4 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 10 client.4243.objecter _send_command 1 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb280abbb0 con 0x7fdb2813d9d0 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efda0056050 con 0x7efda409e0b0 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-17T16:28:05.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda9e7c640 10 monclient: get_auth_request con 0x7efd880443c0 auth_method 0 2024-09-17T16:28:05.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb26d9d640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fdb2813d9d0 0x7fdb2813ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb26d9d640 10 monclient: get_auth_request con 0x7fdb2813d9d0 auth_method 0 2024-09-17T16:28:05.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb26d9d640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fdb2813d9d0 0x7fdb2813ddb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f14d0004040 con 0x7f14e009bad0 2024-09-17T16:28:05.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda9e7c640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7efd880443c0 0x7efd88046800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 client.4250.objecter ms_handle_connect 0x7efd880443c0 2024-09-17T16:28:05.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efda8e7a640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd8803e550 0x7efd88040a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd940af000 tx=0x7efd94008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7efd99ffb640 10 client.4250.objecter ms_handle_connect 0x7efd8803e550 2024-09-17T16:28:05.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f14cc03e550 0x7f14cc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f14d00e8c70 con 0x7f14e009bad0 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 client.4259.objecter ms_dispatch 0x7f14e0000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 3 client.4259.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 3 client.4259.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 client.4259.objecter checking command 1 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f14cc044350 0x7f14cc046790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter _get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 15 client.4259.objecter _session_command_op_remove -1 1 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter _get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 client.4243.objecter ms_handle_connect 0x7fdb2813d9d0 2024-09-17T16:28:05.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 client.4243.objecter ms_handle_connect 0x7fdb0803e580 2024-09-17T16:28:05.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1ffff640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdb0803e580 0x7fdb08040a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb2813f5e0 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 15 client.4259.objecter _session_command_op_assign 0 1 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 4 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 client.4259.objecter _send_command 1 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14d00b1e50 con 0x7f14cc044350 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 20 client.4259.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f14d00ed220 con 0x7f14e009bad0 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 monclient: handle_get_version_reply finishing 1 version 19 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14dffff640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f14cc044350 0x7f14cc046790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14dffff640 10 monclient: get_auth_request con 0x7f14cc044350 auth_method 0 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14deffd640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f14cc03e550 0x7f14cc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14deffd640 10 monclient: get_auth_request con 0x7f14cc03e550 auth_method 0 2024-09-17T16:28:05.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== osd.1 v2:172.21.15.105:6800/3243801162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb280abbb0 con 0x7fdb2813d9d0 2024-09-17T16:28:05.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 client.4243.objecter ms_dispatch 0x7fdb28000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 10 client.4243.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 4 2024-09-17T16:28:05.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7fdb1effd640 15 client.4243.objecter _session_command_op_remove 1 1 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 client.4259.objecter ms_handle_connect 0x7f14cc044350 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14dffff640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f14cc044350 0x7f14cc046790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14c7fff640 10 client.4259.objecter ms_handle_connect 0x7f14cc03e550 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.024+0000 7f14deffd640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f14cc03e550 0x7f14cc040a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14e013b4e0 tx=0x7f14d4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== osd.0 v2:172.21.15.105:6816/315040748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14d00b1e50 con 0x7f14cc044350 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f14c7fff640 10 client.4259.objecter ms_dispatch 0x7f14e0000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f14c7fff640 10 client.4259.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f14c7fff640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 4 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f14c7fff640 15 client.4259.objecter _session_command_op_remove 0 1 2024-09-17T16:28:05.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== osd.5 v2:172.21.15.160:6800/2641095577 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efda0021ec0 con 0x7efd880443c0 2024-09-17T16:28:05.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7efd99ffb640 10 client.4250.objecter ms_dispatch 0x7efda4000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:28:05.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7efd99ffb640 10 client.4250.objecter _finish_command 1 = system:0 2024-09-17T16:28:05.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7efd99ffb640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 4 2024-09-17T16:28:05.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7efd99ffb640 15 client.4250.objecter _session_command_op_remove 5 1 2024-09-17T16:28:05.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 client.4281.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 15 client.4281.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter _get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter _get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 15 client.4281.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter get_session s=0x7f6f18043cf0 osd=2 3 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 15 client.4281.objecter _session_command_op_assign 2 2 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 4 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 client.4281.objecter _send_command 2 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 --> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f340a1ef0 con 0x7f6f180443c0 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f317fa640 1 -- 172.21.15.105:0/3933400524 <== osd.2 v2:172.21.15.105:6808/1006010180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6f340a1ef0 con 0x7f6f180443c0 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f317fa640 10 client.4281.objecter ms_dispatch 0x7f6f34000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f317fa640 10 client.4281.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f317fa640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 4 2024-09-17T16:28:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f317fa640 15 client.4281.objecter _session_command_op_remove 2 2 2024-09-17T16:28:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 librados: watch_flush enter 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 librados: watch_flush exit 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 client.4281.objecter close_session for osd.2 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f6f180443c0 msgr2=0x7f6f18046800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f6f180443c0 0x7f6f18046800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter put_session s=0x7f6f18043cf0 osd=2 1 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 client.4281.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 client.4281.objecter successfully canceled tick 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f1803e550 msgr2=0x7f6f18040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f1803e550 0x7f6f18040a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f3409f650 tx=0x7f6f20007b60 comp rx=0 tx=0).stop 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 10 monclient: shutdown 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 msgr2=0x7f6f341248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 --2- 172.21.15.105:0/3933400524 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f340a5860 0x7f6f341248b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6f2801aa50 tx=0x7f6f28019140 comp rx=0 tx=0).stop 2024-09-17T16:28:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3944c640 1 -- 172.21.15.105:0/3933400524 reap_dead start 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 shutdown_connections 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 >> 172.21.15.105:0/3933400524 conn(0x7f6f340aa450 msgr2=0x7f6f340a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 shutdown_connections 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 -- 172.21.15.105:0/3933400524 wait complete. 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.028+0000 7f6f3aeaf640 1 librados: shutdown 2024-09-17T16:28:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 10 client.4250.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 15 client.4250.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 20 client.4250.objecter _get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 20 client.4250.objecter _get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 15 client.4250.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 20 client.4250.objecter get_session s=0x7efd88043cf0 osd=5 3 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 15 client.4250.objecter _session_command_op_assign 5 2 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 4 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 10 client.4250.objecter _send_command 2 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efda4121520 con 0x7efd880443c0 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efd99ffb640 1 -- 172.21.15.105:0/3887999123 <== osd.5 v2:172.21.15.160:6800/2641095577 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efda4121520 con 0x7efd880443c0 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efd99ffb640 10 client.4250.objecter ms_dispatch 0x7efda4000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efd99ffb640 10 client.4250.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efd99ffb640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 4 2024-09-17T16:28:05.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.040+0000 7efd99ffb640 15 client.4250.objecter _session_command_op_remove 5 2 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 10 librados: watch_flush enter 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 10 librados: watch_flush exit 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 10 client.4250.objecter close_session for osd.5 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7efd880443c0 msgr2=0x7efd88046800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7efd880443c0 0x7efd88046800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 20 client.4250.objecter put_session s=0x7efd88043cf0 osd=5 1 2024-09-17T16:28:05.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 20 client.4250.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 10 client.4250.objecter successfully canceled tick 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd8803e550 msgr2=0x7efd88040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd8803e550 0x7efd88040a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd940af000 tx=0x7efd94008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 10 monclient: shutdown 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 msgr2=0x7efda4072840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 --2- 172.21.15.105:0/3887999123 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efda409e0b0 0x7efda4072840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efda000e7f0 tx=0x7efda000ecb0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efda9e7c640 1 -- 172.21.15.105:0/3887999123 reap_dead start 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 shutdown_connections 2024-09-17T16:28:05.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 >> 172.21.15.105:0/3887999123 conn(0x7efda40a5840 msgr2=0x7efda40a1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.181 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:05.181 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T16:28:05.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 shutdown_connections 2024-09-17T16:28:05.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 -- 172.21.15.105:0/3887999123 wait complete. 2024-09-17T16:28:05.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.044+0000 7efdab8df640 1 librados: shutdown 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 client.4243.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter _get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter _get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter get_session s=0x7fdb28123010 osd=1 3 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 15 client.4243.objecter _session_command_op_assign 1 2 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 4 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 client.4243.objecter _send_command 2 2024-09-17T16:28:05.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb28139ca0 con 0x7fdb2813d9d0 2024-09-17T16:28:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb1effd640 1 -- 172.21.15.105:0/2746143708 <== osd.1 v2:172.21.15.105:6800/3243801162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb28139ca0 con 0x7fdb2813d9d0 2024-09-17T16:28:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb1effd640 10 client.4243.objecter ms_dispatch 0x7fdb28000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb1effd640 10 client.4243.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb1effd640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 4 2024-09-17T16:28:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb1effd640 15 client.4243.objecter _session_command_op_remove 1 2 2024-09-17T16:28:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 librados: watch_flush enter 2024-09-17T16:28:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 librados: watch_flush exit 2024-09-17T16:28:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 client.4243.objecter close_session for osd.1 2024-09-17T16:28:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fdb2813d9d0 msgr2=0x7fdb2813ddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fdb2813d9d0 0x7fdb2813ddb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter put_session s=0x7fdb28123010 osd=1 1 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 client.4243.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 client.4243.objecter successfully canceled tick 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdb0803e580 msgr2=0x7fdb08040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdb0803e580 0x7fdb08040a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdb2813f5e0 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 10 monclient: shutdown 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 msgr2=0x7fdb28122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 --2- 172.21.15.105:0/2746143708 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdb280a4ab0 0x7fdb28122680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ab040 tx=0x7fdb1801add0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb26d9d640 1 -- 172.21.15.105:0/2746143708 reap_dead start 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 shutdown_connections 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 >> 172.21.15.105:0/2746143708 conn(0x7fdb280a8d00 msgr2=0x7fdb280a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 shutdown_connections 2024-09-17T16:28:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 -- 172.21.15.105:0/2746143708 wait complete. 2024-09-17T16:28:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7fdb2cc10640 1 librados: shutdown 2024-09-17T16:28:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 10 client.4259.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-09-17T16:28:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 15 client.4259.objecter _session_command_op_assign -1 2 2024-09-17T16:28:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 20 client.4259.objecter _get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 20 client.4259.objecter _get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 15 client.4259.objecter _session_command_op_remove -1 2 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 20 client.4259.objecter get_session s=0x7f14cc043c80 osd=0 3 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 15 client.4259.objecter _session_command_op_assign 0 2 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 4 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 10 client.4259.objecter _send_command 2 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.052+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 --> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14e01327b0 con 0x7f14cc044350 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14c7fff640 1 -- 172.21.15.105:0/3170588503 <== osd.0 v2:172.21.15.105:6816/315040748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f14e01327b0 con 0x7f14cc044350 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14c7fff640 10 client.4259.objecter ms_dispatch 0x7f14e0000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14c7fff640 10 client.4259.objecter _finish_command 2 = system:0 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14c7fff640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 4 2024-09-17T16:28:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14c7fff640 15 client.4259.objecter _session_command_op_remove 0 2 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 10 librados: watch_flush enter 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 10 librados: watch_flush exit 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 10 client.4259.objecter close_session for osd.0 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f14cc044350 msgr2=0x7f14cc046790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f14cc044350 0x7f14cc046790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 20 client.4259.objecter put_session s=0x7f14cc043c80 osd=0 1 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 20 client.4259.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 10 client.4259.objecter successfully canceled tick 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f14cc03e550 msgr2=0x7f14cc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f14cc03e550 0x7f14cc040a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14e013b4e0 tx=0x7f14d4008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 10 monclient: shutdown 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 msgr2=0x7f14e0139de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 --2- 172.21.15.105:0/3170588503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14e009bad0 0x7f14e0139de0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14d00ae040 tx=0x7f14d0019180 comp rx=0 tx=0).stop 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14dffff640 1 -- 172.21.15.105:0/3170588503 reap_dead start 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 shutdown_connections 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 >> 172.21.15.105:0/3170588503 conn(0x7f14e00a83e0 msgr2=0x7f14e013d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 shutdown_connections 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 -- 172.21.15.105:0/3170588503 wait complete. 2024-09-17T16:28:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.056+0000 7f14e5a69640 1 librados: shutdown 2024-09-17T16:28:05.192 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:05.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T16:28:05.193 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:05.193 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T16:28:05.194 INFO:teuthology.orchestra.run.smithi105.stdout:51539607558 2024-09-17T16:28:05.195 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T16:28:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: monmap: 2024-09-17T16:28:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.294171+0000 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.294171+0000 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.298 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 Processor -- start 2024-09-17T16:28:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 -- start start 2024-09-17T16:28:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: init 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient: _add_conns ranks=[5,6,7,3,10,0,2,1,8,9,4] 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 0x7f44280a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): picked mon.noname-h con 0x7f44280a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): picked mon.noname-d con 0x7f442806cb00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4428139530 0x7f4428139930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): picked mon.noname-i con 0x7f4428139530 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f44280a6610 con 0x7f44280a4750 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f44280a55b0 con 0x7f442806cb00 2024-09-17T16:28:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f442806c7f0 con 0x7f4428139530 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442f555640 10 monclient(hunting): authenticate will time out at 968.559742s 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33940/0 (socket says 172.21.15.105:33940) 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 1 -- 172.21.15.105:0/4279736868 learned_addr learned my addr 172.21.15.105:0/4279736868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 10 monclient(hunting): get_auth_request con 0x7f442806cb00 auth_method 0 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.288+0000 7f442caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442d2f1640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 0x7f44280a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 -- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4428139530 msgr2=0x7f4428139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4428139530 0x7f4428139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 -- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 msgr2=0x7f44280a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 0x7f44280a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 monclient: found mon.noname-d 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 -- 172.21.15.105:0/4279736868 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442806c1f0 con 0x7f442806cb00 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442d2f1640 10 monclient: get_auth_request con 0x7f44280a4750 auth_method 0 2024-09-17T16:28:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442d2f1640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 0x7f44280a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442caf0640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f441801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a372c6f47aff5506 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 1 -- 172.21.15.105:0/4279736868 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44180b0070 con 0x7f442806cb00 2024-09-17T16:28:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: dump: 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 1 -- 172.21.15.105:0/4279736868 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44180aa810 con 0x7f442806cb00 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 1 -- 172.21.15.105:0/4279736868 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44180a9300 con 0x7f442806cb00 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f441ffff640 10 monclient: dump: 2024-09-17T16:28:05.315 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.315 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.315 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.315 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.317 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 5 monclient: authenticate success, global_id 4299 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 4 set_mon_vals no callback set 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 10 monclient: shutdown 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 -- 172.21.15.105:0/4279736868 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 msgr2=0x7f44280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f441801c440 comp rx=0 tx=0).stop 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 -- 172.21.15.105:0/4279736868 shutdown_connections 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4428139530 0x7f4428139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f442806cb00 0x7f44280a5070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 --2- 172.21.15.105:0/4279736868 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44280a4750 0x7f44280a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 -- 172.21.15.105:0/4279736868 >> 172.21.15.105:0/4279736868 conn(0x7f44280a93d0 msgr2=0x7f44280a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 -- 172.21.15.105:0/4279736868 shutdown_connections 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.292+0000 7f442f555640 1 -- 172.21.15.105:0/4279736868 wait complete. 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient: monmap: 2024-09-17T16:28:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 librados: starting msgr at 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 librados: starting objecter 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 Processor -- start 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 -- start start 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 librados: setting wanted keys 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 librados: calling monclient init 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient: init 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient: _add_conns ranks=[3,4,2,5,8,7,1,0,6] 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 0x7f4428122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): picked mon.noname-g con 0x7f442806cb00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 0x7f44281d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): picked mon.noname-c con 0x7f44280a4750 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 0x7f442811ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): picked mon.noname-b con 0x7f4428139530 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 0x7f44281d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 0x7f44281d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57590/0 (socket says 172.21.15.105:57590) 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 1 -- 172.21.15.105:0/1596533510 learned_addr learned my addr 172.21.15.105:0/1596533510 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 0x7f442811ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f44281433a0 con 0x7f4428139530 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 0x7f4428122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4428143220 con 0x7f442806cb00 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f4428143520 con 0x7f44280a4750 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 10 monclient(hunting): authenticate will time out at 968.566552s 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): get_auth_request con 0x7f44280a4750 auth_method 0 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): get_auth_request con 0x7f4428139530 auth_method 0 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): get_auth_request con 0x7f442806cb00 auth_method 0 2024-09-17T16:28:05.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442caf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient(hunting): handle_auth_done global_id 4268 payload 824 2024-09-17T16:28:05.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 -- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 msgr2=0x7f44281d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 0x7f44281d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 -- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 msgr2=0x7f4428122440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 0x7f4428122440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 monclient: found mon.noname-b 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428123960 con 0x7f4428139530 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442d2f1640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 0x7f4428122440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442daf2640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 0x7f442811ff20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f442000be20 tx=0x7f442000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 client.?.objecter ms_handle_connect 0x7f4428139530 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4420019070 con 0x7f4428139530 2024-09-17T16:28:05.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: dump: 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.326 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442000cc70 con 0x7f4428139530 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f442000ce20 con 0x7f4428139530 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:05.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441f7fe640 4 set_mon_vals no callback set 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f441dffb640 10 monclient: dump: 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.328 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.296+0000 7f442f555640 5 monclient: authenticate success, global_id 4268 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _renew_subs 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4428123c40 con 0x7f4428139530 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 client.4268.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _renew_subs 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44280a7c50 con 0x7f4428139530 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 1 librados: init done 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f442001c360 con 0x7f4428139530 2024-09-17T16:28:05.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 0x7f440c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f44200534e0 con 0x7f4428139530 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 10 client.4268.objecter ms_dispatch 0x7f4428000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 3 client.4268.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 3 client.4268.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 20 client.4268.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428139930 con 0x7f4428139530 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442d2f1640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 0x7f440c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442d2f1640 10 monclient: get_auth_request con 0x7f440c03e580 auth_method 0 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f442d2f1640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 0x7f440c040a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f442400af30 tx=0x7f4424008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.300+0000 7f441dffb640 10 client.4268.objecter ms_handle_connect 0x7f440c03e580 2024-09-17T16:28:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.308+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442004dfb0 con 0x7f4428139530 2024-09-17T16:28:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.308+0000 7f441dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.308+0000 7f441dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: monmap: 2024-09-17T16:28:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.341271+0000 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.341271+0000 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.345 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 Processor -- start 2024-09-17T16:28:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 -- start start 2024-09-17T16:28:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: init 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient: _add_conns ranks=[6,7,1,10,5,0,2,4,9,8,3] 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-d con 0x7f3e380a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3e380a4ab0 0x7f3e380a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-i con 0x7f3e380a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3e381397f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e380a4080 con 0x7f3e381397f0 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3e380a7c40 con 0x7f3e380a6a30 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3e380a7890 con 0x7f3e380a4ab0 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3ccc5640 10 monclient(hunting): authenticate will time out at 968.606979s 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33952/0 (socket says 172.21.15.105:33952) 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 -- 172.21.15.105:0/459293592 learned_addr learned my addr 172.21.15.105:0/459293592 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): get_auth_request con 0x7f3e380a6a30 auth_method 0 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e35d9b640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3e380a4ab0 0x7f3e380a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e35d9b640 10 monclient(hunting): get_auth_request con 0x7f3e380a4ab0 auth_method 0 2024-09-17T16:28:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e35d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e35d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e35d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient(hunting): handle_auth_done global_id 4308 payload 274 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 -- 172.21.15.105:0/459293592 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3e380a4ab0 msgr2=0x7f3e380a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3e380a4ab0 0x7f3e380a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 -- 172.21.15.105:0/459293592 >> v1:172.21.15.160:6789/0 conn(0x7f3e381397f0 legacy=0x7f3e38139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 monclient: found mon.noname-d 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.336+0000 7f3e3659c640 1 -- 172.21.15.105:0/459293592 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e380a37e0 con 0x7f3e380a6a30 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3659c640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e280098a0 tx=0x7f3e2801c4a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4cca1ef82047310d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 1 -- 172.21.15.105:0/459293592 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e280ab8b0 con 0x7f3e380a6a30 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: dump: 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.351 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 1 -- 172.21.15.105:0/459293592 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e280b0070 con 0x7f3e380a6a30 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3ccc5640 5 monclient: authenticate success, global_id 4308 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3ccc5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 1 -- 172.21.15.105:0/459293592 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e280aa3a0 con 0x7f3e380a6a30 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3559a640 10 monclient: dump: 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.353 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3ccc5640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.340+0000 7f3e3ccc5640 4 set_mon_vals no callback set 2024-09-17T16:28:05.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: shutdown 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/459293592 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 msgr2=0x7f3e380a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e280098a0 tx=0x7f3e2801c4a0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/459293592 shutdown_connections 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3e380a4ab0 0x7f3e380a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- 172.21.15.105:0/459293592 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3e380a6a30 0x7f3e380a6e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/459293592 >> 172.21.15.105:0/459293592 conn(0x7f3e380a8d00 msgr2=0x7f3e380a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/459293592 shutdown_connections 2024-09-17T16:28:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/459293592 wait complete. 2024-09-17T16:28:05.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: monmap: 2024-09-17T16:28:05.360 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.361 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 librados: starting msgr at 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 librados: starting objecter 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 Processor -- start 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- start start 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 librados: setting wanted keys 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 librados: calling monclient init 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: init 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient: _add_conns ranks=[3,8,5,6,7,1,4,2,0] 2024-09-17T16:28:05.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e380a4ab0 0x7f3e381d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-g con 0x7f3e380a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-e con 0x7f3e380a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3e381397f0 0x7f3e381d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): picked mon.noname-h con 0x7f3e381397f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f3e38144270 con 0x7f3e380a4ab0 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34134/0 (socket says 172.21.15.105:34134) 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 1 -- 172.21.15.105:0/3838084847 learned_addr learned my addr 172.21.15.105:0/3838084847 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e380a4ab0 0x7f3e381d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f3e3806e770 con 0x7f3e381397f0 2024-09-17T16:28:05.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3e381443f0 con 0x7f3e380a6a30 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3ccc5640 10 monclient(hunting): authenticate will time out at 968.615639s 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): get_auth_request con 0x7f3e380a6a30 auth_method 0 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3e381397f0 0x7f3e381d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): get_auth_request con 0x7f3e381397f0 auth_method 0 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): get_auth_request con 0x7f3e380a4ab0 auth_method 0 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e35d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e36d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.344+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2024-09-17T16:28:05.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 -- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3e381397f0 msgr2=0x7f3e381d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3e381397f0 0x7f3e381d1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 -- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e380a4ab0 msgr2=0x7f3e381d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e380a4ab0 0x7f3e381d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 monclient: found mon.noname-e 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e38140350 con 0x7f3e380a6a30 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e35d9b640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00d7d0 tx=0x7f3e2c00dc90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 client.?.objecter ms_handle_connect 0x7f3e380a6a30 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e2c00bb30 con 0x7f3e380a6a30 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: dump: 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.367 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c00bcf0 con 0x7f3e380a6a30 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e2c0148a0 con 0x7f3e380a6a30 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:05.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 monclient: dump: 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.369 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 5 monclient: authenticate success, global_id 4265 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e34d99640 4 set_mon_vals no callback set 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 10 monclient: _renew_subs 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e38122de0 con 0x7f3e380a6a30 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 10 monclient: _renew_subs 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e38122b00 con 0x7f3e380a6a30 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3ccc5640 1 librados: init done 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e24ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e24ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e24ff9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e38139c50 con 0x7f3e380a6a30 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3e2c01b030 con 0x7f3e380a6a30 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 0x7f3e18040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f3e2c019020 con 0x7f3e380a6a30 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 client.4265.objecter ms_dispatch 0x7f3e38000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 3 client.4265.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 3 client.4265.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 20 client.4265.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3659c640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 0x7f3e18040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3659c640 10 monclient: get_auth_request con 0x7f3e1803e550 auth_method 0 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e26ffd640 10 client.4265.objecter ms_handle_connect 0x7f3e1803e550 2024-09-17T16:28:05.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.348+0000 7f3e3659c640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 0x7f3e18040a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e280af000 tx=0x7f3e280a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.360+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e2c022060 con 0x7f3e380a6a30 2024-09-17T16:28:05.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.360+0000 7f3e26ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.360+0000 7f3e26ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: monmap: 2024-09-17T16:28:05.375 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.375 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.372836+0000 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.372836+0000 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.376 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.377 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.378 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.378 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.378 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 Processor -- start 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 -- start start 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: init 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient: _add_conns ranks=[3,6,9,4,2,7,5,8,1,10,0] 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1f40a4750 0x7fc1f40a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-g con 0x7fc1f40a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1f409e740 0x7fc1f40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-d con 0x7fc1f409e740 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:05.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1f409e0b0 0x7fc1f4136500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-a con 0x7fc1f409e0b0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f409ef60 con 0x7fc1f409e0b0 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f40a31c0 con 0x7fc1f40a4750 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc1f40a55b0 con 0x7fc1f409e740 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fbb1a640 10 monclient(hunting): authenticate will time out at 968.638516s 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1f409e0b0 0x7fc1f4136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1f409e740 0x7fc1f40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1f409e0b0 0x7fc1f4136500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45200/0 (socket says 172.21.15.105:45200) 2024-09-17T16:28:05.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 -- 172.21.15.105:0/443958299 learned_addr learned my addr 172.21.15.105:0/443958299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): get_auth_request con 0x7fc1f409e0b0 auth_method 0 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): get_auth_request con 0x7fc1f409e740 auth_method 0 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1f90b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient(hunting): handle_auth_done global_id 4311 payload 274 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 -- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1f409e740 msgr2=0x7fc1f40a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1f409e740 0x7fc1f40a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 -- 172.21.15.105:0/443958299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1f40a4750 msgr2=0x7fc1f40a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1f40a4750 0x7fc1f40a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 monclient: found mon.noname-a 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.368+0000 7fc1fa0b7640 1 -- 172.21.15.105:0/443958299 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f40a0a80 con 0x7fc1f409e0b0 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fa0b7640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f4136500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009e50 tx=0x7fc1ec01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44fb26307559379 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 1 -- 172.21.15.105:0/443958299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1ec0ac070 con 0x7fc1f409e0b0 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: dump: 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.390 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 5 monclient: authenticate success, global_id 4311 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 1 -- 172.21.15.105:0/443958299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec0090d0 con 0x7fc1f409e0b0 2024-09-17T16:28:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 1 -- 172.21.15.105:0/443958299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1ec007860 con 0x7fc1f409e0b0 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1f88b4640 10 monclient: dump: 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 4 set_mon_vals no callback set 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 10 monclient: shutdown 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 msgr2=0x7fc1f4136500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f4136500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009e50 tx=0x7fc1ec01c650 comp rx=0 tx=0).stop 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/443958299 shutdown_connections 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f4136500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1f409e740 0x7fc1f40a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/443958299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1f40a4750 0x7fc1f40a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/443958299 >> 172.21.15.105:0/443958299 conn(0x7fc1f40a5840 msgr2=0x7fc1f40a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/443958299 shutdown_connections 2024-09-17T16:28:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/443958299 wait complete. 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 10 monclient: monmap: 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 librados: starting msgr at 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 librados: starting objecter 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.372+0000 7fc1fbb1a640 1 Processor -- start 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- start start 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 librados: setting wanted keys 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 librados: calling monclient init 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: init 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _add_conns ranks=[0,5,7,8,1,4,2,6,3] 2024-09-17T16:28:05.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-a con 0x7fc1f409e0b0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45206/0 (socket says 172.21.15.105:45206) 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 0x7fc1f41d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-h con 0x7fc1f409e740 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 -- 172.21.15.105:0/4248966890 learned_addr learned my addr 172.21.15.105:0/4248966890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc1f40a4750 0x7fc1f41d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): picked mon.noname-i con 0x7fc1f40a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f406c4e0 con 0x7fc1f409e0b0 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc1f406c360 con 0x7fc1f409e740 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc1f406c660 con 0x7fc1f40a4750 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient(hunting): authenticate will time out at 968.645226s 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): get_auth_request con 0x7fc1f409e0b0 auth_method 0 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f90b5640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 0x7fc1f41d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient(hunting): handle_auth_done global_id 4320 payload 824 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fa0b7640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc1f40a4750 0x7fc1f41d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 -- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc1f40a4750 msgr2=0x7fc1f41d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc1f40a4750 0x7fc1f41d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 -- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 msgr2=0x7fc1f41d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 0x7fc1f41d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 monclient: found mon.noname-a 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f40a4f50 con 0x7fc1f409e0b0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f90b5640 10 monclient: get_auth_request con 0x7fc1f409e740 auth_method 0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f90b5640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 0x7fc1f41d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1f98b6640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000b3f0 tx=0x7fc1f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 client.?.objecter ms_handle_connect 0x7fc1f409e0b0 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1f0015020 con 0x7fc1f409e0b0 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: dump: 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.400 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f0011670 con 0x7fc1f409e0b0 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1f0011c90 con 0x7fc1f409e0b0 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ebfff640 4 set_mon_vals no callback set 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1ea7fc640 10 monclient: dump: 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 5 monclient: authenticate success, global_id 4320 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _renew_subs 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f413b320 con 0x7fc1f409e0b0 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 client.4320.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _renew_subs 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f413a0e0 con 0x7fc1f409e0b0 2024-09-17T16:28:05.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.376+0000 7fc1fbb1a640 1 librados: init done 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1fbb1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1fbb1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1fbb1a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f40008e0 con 0x7fc1f409e0b0 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc1f0018020 con 0x7fc1f409e0b0 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fc1f004f810 con 0x7fc1f409e0b0 2024-09-17T16:28:05.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 10 client.4320.objecter ms_dispatch 0x7fc1f4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 3 client.4320.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 3 client.4320.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1ea7fc640 20 client.4320.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1f90b5640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1f90b5640 10 monclient: get_auth_request con 0x7fc1d803e550 auth_method 0 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.380+0000 7fc1f90b5640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1f413d750 tx=0x7fc1e4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7fc1ea7fc640 10 client.4320.objecter ms_handle_connect 0x7fc1d803e550 2024-09-17T16:28:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: monmap: 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.388286+0000 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.388286+0000 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 Processor -- start 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 -- start start 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: init 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient: _add_conns ranks=[8,9,5,0,4,7,6,3,1,2,10] 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-e con 0x7f4de8026c70 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 0x7f4de8027590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-a con 0x7f4de80209e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de80bdb30 0x7f4de80bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-h con 0x7f4de80bdb30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4de8020e60 con 0x7f4de80209e0 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4de8027ad0 con 0x7f4de80bdb30 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4de80207a0 con 0x7f4de8026c70 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4def59e640 10 monclient(hunting): authenticate will time out at 968.654312s 2024-09-17T16:28:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34142/0 (socket says 172.21.15.105:34142) 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 -- 172.21.15.105:0/1452753345 learned_addr learned my addr 172.21.15.105:0/1452753345 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): get_auth_request con 0x7f4de8026c70 auth_method 0 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 1 --2- 172.21.15.105:0/1452753345 >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 0x7f4de8027590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): get_auth_request con 0x7f4de80209e0 auth_method 0 2024-09-17T16:28:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4deed9d640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de80bdb30 0x7f4de80bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4deed9d640 10 monclient(hunting): get_auth_request con 0x7f4de80bdb30 auth_method 0 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4deed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4deed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4deed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 -- 172.21.15.105:0/1452753345 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de80bdb30 msgr2=0x7f4de80bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de80bdb30 0x7f4de80bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 -- 172.21.15.105:0/1452753345 >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 msgr2=0x7f4de8027590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 --2- 172.21.15.105:0/1452753345 >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 0x7f4de8027590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 monclient: found mon.noname-e 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dee59c640 1 -- 172.21.15.105:0/1452753345 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de8022b80 con 0x7f4de8026c70 2024-09-17T16:28:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7f4dedd9b640 1 --2- 172.21.15.105:0/1452753345 >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 0x7f4de8027590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:05.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1f001f020 con 0x7fc1f409e0b0 2024-09-17T16:28:05.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7fc1ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.384+0000 7fc1ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4dee59c640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de001c3f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e8130204cf8db6d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 1 -- 172.21.15.105:0/1452753345 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4de00b0070 con 0x7f4de8026c70 2024-09-17T16:28:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:05.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: dump: 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.419 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 1 -- 172.21.15.105:0/1452753345 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de00aa810 con 0x7f4de8026c70 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4def59e640 5 monclient: authenticate success, global_id 4274 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4def59e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4def59e640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 1 -- 172.21.15.105:0/1452753345 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4de00a9300 con 0x7f4de8026c70 2024-09-17T16:28:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4def59e640 4 set_mon_vals no callback set 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.388+0000 7f4ded59a640 10 monclient: dump: 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.421 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: shutdown 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- 172.21.15.105:0/1452753345 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 msgr2=0x7f4de8027050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de001c3f0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- 172.21.15.105:0/1452753345 shutdown_connections 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de80bdb30 0x7f4de80bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- 172.21.15.105:0/1452753345 >> v2:172.21.15.105:3300/0 conn(0x7f4de80209e0 0x7f4de8027590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- 172.21.15.105:0/1452753345 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de8026c70 0x7f4de8027050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- 172.21.15.105:0/1452753345 >> 172.21.15.105:0/1452753345 conn(0x7f4de8008490 msgr2=0x7f4de80088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- 172.21.15.105:0/1452753345 shutdown_connections 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- 172.21.15.105:0/1452753345 wait complete. 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: monmap: 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 librados: starting msgr at 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 librados: starting objecter 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 Processor -- start 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- start start 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 librados: setting wanted keys 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 librados: calling monclient init 2024-09-17T16:28:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: init 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient: _add_conns ranks=[8,5,3,4,1,6,7,0,2] 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-e con 0x7f4de80209e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de8026c70 0x7f4de801b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-h con 0x7f4de8026c70 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de80bdb30 0x7f4de801b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): picked mon.noname-g con 0x7f4de80bdb30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4de801fb10 con 0x7f4de80bdb30 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4de801f990 con 0x7f4de8026c70 2024-09-17T16:28:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4de801fc90 con 0x7f4de80209e0 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4def59e640 10 monclient(hunting): authenticate will time out at 968.662570s 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34158/0 (socket says 172.21.15.105:34158) 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 -- 172.21.15.105:0/3833269768 learned_addr learned my addr 172.21.15.105:0/3833269768 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): get_auth_request con 0x7f4de80209e0 auth_method 0 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de80bdb30 0x7f4de801b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): get_auth_request con 0x7f4de80bdb30 auth_method 0 2024-09-17T16:28:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4deed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dedd9b640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de8026c70 0x7f4de801b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient(hunting): handle_auth_done global_id 4283 payload 824 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 -- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de8026c70 msgr2=0x7f4de801b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de8026c70 0x7f4de801b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 -- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de80bdb30 msgr2=0x7f4de801b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de80bdb30 0x7f4de801b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 monclient: found mon.noname-e 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 -- 172.21.15.105:0/3833269768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de80bd8a0 con 0x7f4de80209e0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4dee59c640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097c0 tx=0x7f4de0019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4ddeffd640 10 client.?.objecter ms_handle_connect 0x7f4de80209e0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.392+0000 7f4ddeffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4de00b9070 con 0x7f4de80209e0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: dump: 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 5 monclient: authenticate success, global_id 4283 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de00044b0 con 0x7f4de80209e0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 10 monclient: _renew_subs 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 1 -- 172.21.15.105:0/3833269768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de801e300 con 0x7f4de80209e0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4de00b0040 con 0x7f4de80209e0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 monclient: dump: 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 10 client.4283.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 10 monclient: _renew_subs 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 1 -- 172.21.15.105:0/3833269768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de802ae20 con 0x7f4de80209e0 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4def59e640 1 librados: init done 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4decd99640 4 set_mon_vals no callback set 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddcff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddcff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddcff9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc8005090 con 0x7f4de80209e0 2024-09-17T16:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4de0004040 con 0x7f4de80209e0 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 0x7f4dd0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f4de00e89d0 con 0x7f4de80209e0 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 10 client.4283.objecter ms_dispatch 0x7f4df0000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 3 client.4283.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 3 client.4283.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4ddeffd640 20 client.4283.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f15bba71640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f15bba71640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f15bba71640 10 monclient: monmap: 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.403381+0000 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.403381+0000 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4dedd9b640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 0x7f4dd0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.396+0000 7f4dedd9b640 10 monclient: get_auth_request con 0x7f4dd003e550 auth_method 0 2024-09-17T16:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 Processor -- start 2024-09-17T16:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f4dedd9b640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 0x7f4dd0040a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009580 tx=0x7f4de4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f4ddeffd640 10 client.4283.objecter ms_handle_connect 0x7f4dd003e550 2024-09-17T16:28:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 -- start start 2024-09-17T16:28:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient: init 2024-09-17T16:28:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient: _add_conns ranks=[5,10,4,6,8,2,7,3,0,9,1] 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15b40a4750 0x7f15b40a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-h con 0x7f15b40a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15b409e660 0x7f15b40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-f con 0x7f15b409e660 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-c con 0x7f15b4139530 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f15b40a66c0 con 0x7f15b409e660 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f15b40a55b0 con 0x7f15b4139530 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f15b40a61d0 con 0x7f15b40a4750 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15bba71640 10 monclient(hunting): authenticate will time out at 968.669318s 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57602/0 (socket says 172.21.15.105:57602) 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 -- 172.21.15.105:0/1260539243 learned_addr learned my addr 172.21.15.105:0/1260539243 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): get_auth_request con 0x7f15b4139530 auth_method 0 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 1 --2- 172.21.15.105:0/1260539243 >> v2:172.21.15.160:3300/0 conn(0x7f15b409e660 0x7f15b40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): get_auth_request con 0x7f15b409e660 auth_method 0 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b980d640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15b40a4750 0x7f15b40a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b980d640 10 monclient(hunting): get_auth_request con 0x7f15b40a4750 auth_method 0 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b980d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b980d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b980d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient(hunting): handle_auth_done global_id 4261 payload 274 2024-09-17T16:28:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 -- 172.21.15.105:0/1260539243 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15b40a4750 msgr2=0x7f15b40a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15b40a4750 0x7f15b40a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 -- 172.21.15.105:0/1260539243 >> v2:172.21.15.160:3300/0 conn(0x7f15b409e660 msgr2=0x7f15b40a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 --2- 172.21.15.105:0/1260539243 >> v2:172.21.15.160:3300/0 conn(0x7f15b409e660 0x7f15b40a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 monclient: found mon.noname-c 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7f15ba00e640 1 -- 172.21.15.105:0/1260539243 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b40a6410 con 0x7f15b4139530 2024-09-17T16:28:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7fe222a56640 10 monclient: get_monmap_and_config 2024-09-17T16:28:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7fe222a56640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.400+0000 7fe222a56640 10 monclient: monmap: 2024-09-17T16:28:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:05.407105+0000 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:05.407105+0000 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 Processor -- start 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 -- start start 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient: init 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient: _add_conns ranks=[5,1,8,3,10,6,2,4,0,7,9] 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 0x7fe214027050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-h con 0x7fe214026c70 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe2140210a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-e con 0x7fe214027590 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2140254c0 con 0x7fe2140210a0 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe21401fa00 con 0x7fe214026c70 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe2140079b0 con 0x7fe214027590 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe222a56640 10 monclient(hunting): authenticate will time out at 968.673194s 2024-09-17T16:28:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15ba00e640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a401ce60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=51690cf3c9fff4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34166/0 (socket says 172.21.15.105:34166) 2024-09-17T16:28:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 -- 172.21.15.105:0/430596777 learned_addr learned my addr 172.21.15.105:0/430596777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): get_auth_request con 0x7fe214027590 auth_method 0 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 0x7fe214027050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 10 monclient(hunting): get_auth_request con 0x7fe214026c70 auth_method 0 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient(hunting): handle_auth_done global_id 4292 payload 274 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 -- 172.21.15.105:0/430596777 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 msgr2=0x7fe214027050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 0x7fe214027050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 -- 172.21.15.105:0/430596777 >> v1:172.21.15.160:6789/0 conn(0x7fe2140210a0 legacy=0x7fe2140bab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 1 -- 172.21.15.105:0/1260539243 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15a40b0070 con 0x7f15b4139530 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: dump: 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.450 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 1 -- 172.21.15.105:0/1260539243 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a40aa810 con 0x7f15b4139530 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 1 -- 172.21.15.105:0/1260539243 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15a40a9300 con 0x7f15b4139530 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15b880b640 10 monclient: dump: 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 5 monclient: authenticate success, global_id 4261 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 4 set_mon_vals no callback set 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 10 monclient: shutdown 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 -- 172.21.15.105:0/1260539243 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 msgr2=0x7f15b4139970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a401ce60 comp rx=0 tx=0).stop 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 -- 172.21.15.105:0/1260539243 shutdown_connections 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f15b4139530 0x7f15b4139970 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 --2- 172.21.15.105:0/1260539243 >> v2:172.21.15.160:3300/0 conn(0x7f15b409e660 0x7f15b40a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 --2- 172.21.15.105:0/1260539243 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15b40a4750 0x7f15b40a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 -- 172.21.15.105:0/1260539243 >> 172.21.15.105:0/1260539243 conn(0x7f15b40a93f0 msgr2=0x7f15b40a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 -- 172.21.15.105:0/1260539243 shutdown_connections 2024-09-17T16:28:05.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 1 -- 172.21.15.105:0/1260539243 wait complete. 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 monclient: found mon.noname-e 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 -- 172.21.15.105:0/430596777 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe214020730 con 0x7fe214027590 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe21bfff640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 0x7fe214027050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7fe220ff3640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe21c06b150 tx=0x7fe21c0997f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b26735b7625531bb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4de0016130 con 0x7f4de80209e0 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f4ddeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f4ddeffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.404+0000 7f15bba71640 10 monclient: monmap: 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.456 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.457 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 librados: starting msgr at 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 librados: starting objecter 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 Processor -- start 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 1 -- 172.21.15.105:0/430596777 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe21c125960 con 0x7fe214027590 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: dump: 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.458 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.459 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 1 -- 172.21.15.105:0/430596777 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c125b00 con 0x7fe214027590 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 1 -- 172.21.15.105:0/430596777 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe21c125cb0 con 0x7fe214027590 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7fe21affd640 10 monclient: dump: 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.460 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.461 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- start start 2024-09-17T16:28:05.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 librados: setting wanted keys 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 librados: calling monclient init 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: init 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _add_conns ranks=[2,8,1,7,0,5,6,3,4] 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-b con 0x7f15b409e660 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 0x7f15b4137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-e con 0x7f15b40a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b4139530 0x7f15b41384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): picked mon.noname-f con 0x7f15b4139530 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f15b406c950 con 0x7f15b4139530 2024-09-17T16:28:05.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f15b406c7d0 con 0x7f15b409e660 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f15b4143220 con 0x7f15b40a4750 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient(hunting): authenticate will time out at 968.677099s 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59516/0 (socket says 172.21.15.105:59516) 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 -- 172.21.15.105:0/3616353338 learned_addr learned my addr 172.21.15.105:0/3616353338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): get_auth_request con 0x7f15b409e660 auth_method 0 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 0x7f15b4137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): get_auth_request con 0x7f15b40a4750 auth_method 0 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15ba00e640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b4139530 0x7f15b41384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15ba00e640 10 monclient(hunting): get_auth_request con 0x7f15b4139530 auth_method 0 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15ba00e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15ba00e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15ba00e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient(hunting): handle_auth_done global_id 4277 payload 824 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 -- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 msgr2=0x7f15b4137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 0x7f15b4137fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 -- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b4139530 msgr2=0x7f15b41384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b4139530 0x7f15b41384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 monclient: found mon.noname-b 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 -- 172.21.15.105:0/3616353338 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b4077950 con 0x7f15b409e660 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b900c640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 0x7f15b4137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15b980d640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007f20 tx=0x7f15ac00e570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 client.?.objecter ms_handle_connect 0x7f15b409e660 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15ac014890 con 0x7f15b409e660 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: dump: 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.469 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 5 monclient: authenticate success, global_id 4277 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac019070 con 0x7f15b409e660 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _renew_subs 2024-09-17T16:28:05.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b413d600 con 0x7f15b409e660 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15ac013380 con 0x7f15b409e660 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 client.4277.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15aa7fc640 10 monclient: dump: 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.471 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _renew_subs 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b40a80f0 con 0x7f15b409e660 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 librados: init done 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.408+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b4138a20 con 0x7f15b409e660 2024-09-17T16:28:05.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 5 monclient: authenticate success, global_id 4292 2024-09-17T16:28:05.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:05.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 4 set_mon_vals no callback set 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f15ac014420 con 0x7f15b409e660 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 0x7f1598040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f15ac04fe30 con 0x7f15b409e660 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 10 client.4277.objecter ms_dispatch 0x7f15b4000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 3 client.4277.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 3 client.4277.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: shutdown 2024-09-17T16:28:05.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- 172.21.15.105:0/430596777 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 msgr2=0x7fe2140bb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe21c06b150 tx=0x7fe21c0997f0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 20 client.4277.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15abfff640 4 set_mon_vals no callback set 2024-09-17T16:28:05.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15b900c640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 0x7f1598040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- 172.21.15.105:0/430596777 shutdown_connections 2024-09-17T16:28:05.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe214027590 0x7fe2140bb230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- 172.21.15.105:0/430596777 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214026c70 0x7fe214027050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- 172.21.15.105:0/430596777 >> 172.21.15.105:0/430596777 conn(0x7fe214008490 msgr2=0x7fe2140088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15b900c640 10 monclient: get_auth_request con 0x7f159803e550 auth_method 0 2024-09-17T16:28:05.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- 172.21.15.105:0/430596777 shutdown_connections 2024-09-17T16:28:05.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- 172.21.15.105:0/430596777 wait complete. 2024-09-17T16:28:05.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15b900c640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 0x7f1598040a10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15b413dc70 tx=0x7f15b0009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7f15aa7fc640 10 client.4277.objecter ms_handle_connect 0x7f159803e550 2024-09-17T16:28:05.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: build_initial_monmap 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: monmap: 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 librados: starting msgr at 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 librados: starting objecter 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 Processor -- start 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- start start 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 librados: setting wanted keys 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 librados: calling monclient init 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: init 2024-09-17T16:28:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient: _add_conns ranks=[8,7,5,2,1,0,6,4,3] 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-e con 0x7fe2140210a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe214026c70 0x7fe2140a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-i con 0x7fe214026c70 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214027590 0x7fe2140a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): picked mon.noname-h con 0x7fe214027590 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe2140bb770 con 0x7fe214027590 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe2140279f0 con 0x7fe214026c70 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe2140bb8f0 con 0x7fe2140210a0 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe222a56640 10 monclient(hunting): authenticate will time out at 968.683755s 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34192/0 (socket says 172.21.15.105:34192) 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 1 -- 172.21.15.105:0/1429413556 learned_addr learned my addr 172.21.15.105:0/1429413556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 10 monclient(hunting): get_auth_request con 0x7fe2140210a0 auth_method 0 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.412+0000 7fe21bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214027590 0x7fe2140a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): get_auth_request con 0x7fe214027590 auth_method 0 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe214026c70 0x7fe2140a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): get_auth_request con 0x7fe214026c70 auth_method 0 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe220ff3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient(hunting): handle_auth_done global_id 4310 payload 824 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 -- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe214026c70 msgr2=0x7fe2140a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe214026c70 0x7fe2140a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 -- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214027590 msgr2=0x7fe2140a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214027590 0x7fe2140a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 monclient: found mon.noname-e 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 -- 172.21.15.105:0/1429413556 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2140a3ca0 con 0x7fe2140210a0 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21bfff640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e810 tx=0x7fe21000ecd0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 client.?.objecter ms_handle_connect 0x7fe2140210a0 2024-09-17T16:28:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe210017070 con 0x7fe2140210a0 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: dump: 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21000cb40 con 0x7fe2140210a0 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe21000f040 con 0x7fe2140210a0 2024-09-17T16:28:05.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 monclient: dump: 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:05.491 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21a7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 5 monclient: authenticate success, global_id 4310 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 10 monclient: _renew_subs 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 1 -- 172.21.15.105:0/1429413556 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2140a2a30 con 0x7fe2140210a0 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 10 client.4310.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 10 monclient: _renew_subs 2024-09-17T16:28:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 1 -- 172.21.15.105:0/1429413556 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2140a4b10 con 0x7fe2140210a0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe222a56640 1 librados: init done 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe2067fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe2067fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe2067fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f4005090 con 0x7fe2140210a0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe21001a070 con 0x7fe2140210a0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 0x7fe200040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fe21004f450 con 0x7fe2140210a0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 10 client.4310.objecter ms_dispatch 0x7fe21c000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 3 client.4310.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 3 client.4310.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe218ff9640 20 client.4310.objecter dump_active .. 0 homeless 2024-09-17T16:28:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 0x7fe200040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.416+0000 7fe21b7fe640 10 monclient: get_auth_request con 0x7fe20003e550 auth_method 0 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.420+0000 7fe21b7fe640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 0x7fe200040a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2140c18e0 tx=0x7fe208009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.420+0000 7fe218ff9640 10 client.4310.objecter ms_handle_connect 0x7fe20003e550 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.420+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15ac054070 con 0x7f15b409e660 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.420+0000 7f15aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.420+0000 7f15aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.428+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe2140a2a30 con 0x7fe2140210a0 2024-09-17T16:28:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.428+0000 7fe218ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.428+0000 7fe218ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:05.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f442f555640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:05.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f442f555640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:05.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f442f555640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f442806e230 con 0x7f4428139530 2024-09-17T16:28:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f441dffb640 1 -- 172.21.15.105:0/1596533510 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4420012340 con 0x7f4428139530 2024-09-17T16:28:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f441dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.540+0000 7f441dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.550 INFO:teuthology.orchestra.run.smithi105.stdout:51539607555 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 10 librados: watch_flush enter 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 10 librados: watch_flush exit 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 20 client.4268.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 10 client.4268.objecter successfully canceled tick 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 msgr2=0x7f440c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 0x7f440c040a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f442400af30 tx=0x7f4424008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 10 monclient: shutdown 2024-09-17T16:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 msgr2=0x7f442811ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 0x7f442811ff20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f442000be20 tx=0x7f442000bf10 comp rx=0 tx=0).stop 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 shutdown_connections 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f440c03e580 0x7f440c040a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4428139530 0x7f442811ff20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44280a4750 0x7f44281d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 --2- 172.21.15.105:0/1596533510 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f442806cb00 0x7f4428122440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 >> 172.21.15.105:0/1596533510 conn(0x7f44280a93d0 msgr2=0x7f44280a84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 shutdown_connections 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 -- 172.21.15.105:0/1596533510 wait complete. 2024-09-17T16:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.544+0000 7f442f555640 1 librados: shutdown 2024-09-17T16:28:05.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.3 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.588+0000 7f4ddcff9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.588+0000 7f4ddcff9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.588+0000 7f4ddcff9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.588+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4dc8005650 con 0x7f4de80209e0 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.588+0000 7f4ddeffd640 1 -- 172.21.15.105:0/3833269768 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4de00aad70 con 0x7f4de80209e0 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddeffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.598 INFO:teuthology.orchestra.run.smithi105.stdout:47244640260 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 10 librados: watch_flush enter 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 10 librados: watch_flush exit 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 20 client.4283.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 10 client.4283.objecter successfully canceled tick 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 msgr2=0x7f4dd0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 0x7f4dd0040a10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009580 tx=0x7f4de4008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 10 monclient: shutdown 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 msgr2=0x7f4de801ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097c0 tx=0x7f4de0019190 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 shutdown_connections 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4dd003e550 0x7f4dd0040a10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de80bdb30 0x7f4de801b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4de8026c70 0x7f4de801b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 --2- 172.21.15.105:0/3833269768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4de80209e0 0x7f4de801ac70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 >> 172.21.15.105:0/3833269768 conn(0x7f4de8008490 msgr2=0x7f4de80ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 shutdown_connections 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 -- 172.21.15.105:0/3833269768 wait complete. 2024-09-17T16:28:05.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.592+0000 7f4ddcff9640 1 librados: shutdown 2024-09-17T16:28:05.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.2 2024-09-17T16:28:05.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e24ff9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:05.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e24ff9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:05.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e24ff9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3e380008e0 con 0x7f3e380a6a30 2024-09-17T16:28:05.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e26ffd640 1 -- 172.21.15.105:0/3838084847 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3e2c01c3c0 con 0x7f3e380a6a30 2024-09-17T16:28:05.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e26ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:05.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.680+0000 7f3e26ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.690 INFO:teuthology.orchestra.run.smithi105.stdout:47244640260 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 10 librados: watch_flush enter 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 10 librados: watch_flush exit 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 20 client.4265.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 10 client.4265.objecter successfully canceled tick 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 msgr2=0x7f3e18040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 0x7f3e18040a10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3e280af000 tx=0x7f3e280a7000 comp rx=0 tx=0).stop 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 10 monclient: shutdown 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 msgr2=0x7f3e381d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00d7d0 tx=0x7f3e2c00dc90 comp rx=0 tx=0).stop 2024-09-17T16:28:05.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 shutdown_connections 2024-09-17T16:28:05.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e1803e550 0x7f3e18040a10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3e381397f0 0x7f3e381d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3e380a6a30 0x7f3e381d1840 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 --2- 172.21.15.105:0/3838084847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e380a4ab0 0x7f3e381d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 >> 172.21.15.105:0/3838084847 conn(0x7f3e380a8d00 msgr2=0x7f3e380a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 shutdown_connections 2024-09-17T16:28:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 -- 172.21.15.105:0/3838084847 wait complete. 2024-09-17T16:28:05.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.684+0000 7f3e24ff9640 1 librados: shutdown 2024-09-17T16:28:05.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.5 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1fbb1a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1fbb1a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1fbb1a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc1f4139dd0 con 0x7fc1f409e0b0 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1ea7fc640 1 -- 172.21.15.105:0/4248966890 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1f0022350 con 0x7fc1f409e0b0 2024-09-17T16:28:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:05.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.712+0000 7fc1ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.721 INFO:teuthology.orchestra.run.smithi105.stdout:51539607555 2024-09-17T16:28:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 10 librados: watch_flush enter 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 10 librados: watch_flush exit 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 20 client.4320.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 10 client.4320.objecter successfully canceled tick 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 msgr2=0x7fc1d8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1f413d750 tx=0x7fc1e4008040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 10 monclient: shutdown 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 msgr2=0x7fc1f41d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000b3f0 tx=0x7fc1f000b8b0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 shutdown_connections 2024-09-17T16:28:05.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc1f40a4750 0x7fc1f41d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1f409e740 0x7fc1f41d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 --2- 172.21.15.105:0/4248966890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1f409e0b0 0x7fc1f41d0c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 >> 172.21.15.105:0/4248966890 conn(0x7fc1f40a5840 msgr2=0x7fc1f40a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 shutdown_connections 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 -- 172.21.15.105:0/4248966890 wait complete. 2024-09-17T16:28:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.716+0000 7fc1fbb1a640 1 librados: shutdown 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15bba71640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15bba71640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15bba71640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f15b4139970 con 0x7f15b409e660 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15aa7fc640 1 -- 172.21.15.105:0/3616353338 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15ac023020 con 0x7f15b409e660 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.720+0000 7f15aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.730 INFO:teuthology.orchestra.run.smithi105.stdout:51539607556 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 10 librados: watch_flush enter 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 10 librados: watch_flush exit 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 20 client.4277.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 10 client.4277.objecter successfully canceled tick 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 msgr2=0x7f1598040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 0x7f1598040a10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15b413dc70 tx=0x7f15b0009190 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 10 monclient: shutdown 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 msgr2=0x7f15b41d5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007f20 tx=0x7f15ac00e570 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 shutdown_connections 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159803e550 0x7f1598040a10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b4139530 0x7f15b41384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15b40a4750 0x7f15b4137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 --2- 172.21.15.105:0/3616353338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f15b409e660 0x7f15b41d5760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 >> 172.21.15.105:0/3616353338 conn(0x7f15b40a93f0 msgr2=0x7f15b40a05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 shutdown_connections 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 -- 172.21.15.105:0/3616353338 wait complete. 2024-09-17T16:28:05.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.724+0000 7f15bba71640 1 librados: shutdown 2024-09-17T16:28:05.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.4 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe1f4005650 con 0x7fe2140210a0 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe218ff9640 1 -- 172.21.15.105:0/1429413556 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe21000ccf0 con 0x7fe2140210a0 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe218ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe218ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:05.747 INFO:teuthology.orchestra.run.smithi105.stdout:47244640260 2024-09-17T16:28:05.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.0 2024-09-17T16:28:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 librados: watch_flush enter 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 librados: watch_flush exit 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 20 client.4310.objecter shutdown clearing up homeless session... 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 client.4310.objecter successfully canceled tick 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 msgr2=0x7fe200040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 0x7fe200040a10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2140c18e0 tx=0x7fe208009040 comp rx=0 tx=0).stop 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 10 monclient: shutdown 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 msgr2=0x7fe2140a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e810 tx=0x7fe21000ecd0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 shutdown_connections 2024-09-17T16:28:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe20003e550 0x7fe200040a10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe214027590 0x7fe2140a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe214026c70 0x7fe2140a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 --2- 172.21.15.105:0/1429413556 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe2140210a0 0x7fe2140a1990 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 >> 172.21.15.105:0/1429413556 conn(0x7fe214008490 msgr2=0x7fe2140c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 shutdown_connections 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 -- 172.21.15.105:0/1429413556 wait complete. 2024-09-17T16:28:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:05.740+0000 7fe2067fc640 1 librados: shutdown 2024-09-17T16:28:05.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.1 2024-09-17T16:28:06.569 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T16:28:06.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T16:28:06.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: monmap: 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.696234+0000 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.696234+0000 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.699 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.700 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 Processor -- start 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 -- start start 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: init 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient: _add_conns ranks=[7,4,0,1,10,5,3,8,2,9,6] 2024-09-17T16:28:06.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd80a4750 0x7f7dd80a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-i con 0x7f7dd80a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-c con 0x7f7dd809eec0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7dd8139530 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd809df90 con 0x7f7dd8139530 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7dd80a55b0 con 0x7f7dd809eec0 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f7dd809dcb0 con 0x7f7dd80a4750 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7ddd836640 10 monclient(hunting): authenticate will time out at 969.961957s 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57610/0 (socket says 172.21.15.105:57610) 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 -- 172.21.15.105:0/684237995 learned_addr learned my addr 172.21.15.105:0/684237995 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): get_auth_request con 0x7f7dd809eec0 auth_method 0 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_done global_id 4270 payload 274 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 -- 172.21.15.105:0/684237995 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd80a4750 msgr2=0x7f7dd80a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd6ffd640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd80a4750 0x7f7dd80a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd80a4750 0x7f7dd80a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 -- 172.21.15.105:0/684237995 >> v1:172.21.15.105:6789/0 conn(0x7f7dd8139530 legacy=0x7f7dd8139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 monclient: found mon.noname-c 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd5ffb640 1 -- 172.21.15.105:0/684237995 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2124461156 0 0) 0x7f7dd00014a0 con 0x7f7dd8139530 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd67fc640 1 -- 172.21.15.105:0/684237995 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd809e270 con 0x7f7dd809eec0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.692+0000 7f7dd5ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80098a0 tx=0x7f7dc801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d97c2d04e8f20205 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 1 -- 172.21.15.105:0/684237995 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dc80b1070 con 0x7f7dd809eec0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: dump: 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.707 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.708 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 5 monclient: authenticate success, global_id 4270 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 1 -- 172.21.15.105:0/684237995 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc80aa7b0 con 0x7f7dd809eec0 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 4 set_mon_vals no callback set 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 1 -- 172.21.15.105:0/684237995 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dc80aae10 con 0x7f7dd809eec0 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd5ffb640 10 monclient: dump: 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.710 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: shutdown 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/684237995 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 msgr2=0x7f7dd80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80098a0 tx=0x7f7dc801c440 comp rx=0 tx=0).stop 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/684237995 shutdown_connections 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7dd809eec0 0x7f7dd80a5070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- 172.21.15.105:0/684237995 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd80a4750 0x7f7dd80a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/684237995 >> 172.21.15.105:0/684237995 conn(0x7f7dd80a5840 msgr2=0x7f7dd80a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/684237995 shutdown_connections 2024-09-17T16:28:06.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/684237995 wait complete. 2024-09-17T16:28:06.712 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T16:28:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: monmap: 2024-09-17T16:28:06.719 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.720 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 librados: starting msgr at 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 librados: starting objecter 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 Processor -- start 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- start start 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 librados: setting wanted keys 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 librados: calling monclient init 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: init 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _add_conns ranks=[7,6,0,4,8,3,5,1,2] 2024-09-17T16:28:06.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd809eec0 0x7f7dd811f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-i con 0x7f7dd809eec0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-d con 0x7f7dd80a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd8139530 0x7f7dd811fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): picked mon.noname-a con 0x7f7dd8139530 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81443d0 con 0x7f7dd8139530 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f7dd8144550 con 0x7f7dd80a4750 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f7dd81446d0 con 0x7f7dd809eec0 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient(hunting): authenticate will time out at 969.966818s 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd8139530 0x7f7dd811fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33960/0 (socket says 172.21.15.105:33960) 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 -- 172.21.15.105:0/2440457553 learned_addr learned my addr 172.21.15.105:0/2440457553 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): get_auth_request con 0x7f7dd80a4750 auth_method 0 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): get_auth_request con 0x7f7dd8139530 auth_method 0 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient(hunting): handle_auth_done global_id 4326 payload 824 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd6ffd640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd809eec0 0x7f7dd811f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 -- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd809eec0 msgr2=0x7f7dd811f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd809eec0 0x7f7dd811f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 -- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd8139530 msgr2=0x7f7dd811fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd8139530 0x7f7dd811fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 monclient: found mon.noname-d 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 -- 172.21.15.105:0/2440457553 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd8120e70 con 0x7f7dd80a4750 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd67fc640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80af040 tx=0x7f7dc80191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 client.?.objecter ms_handle_connect 0x7f7dd80a4750 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dc80ba070 con 0x7f7dd80a4750 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: dump: 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.725 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 5 monclient: authenticate success, global_id 4326 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc80aadd0 con 0x7f7dd80a4750 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _renew_subs 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd8122810 con 0x7f7dd80a4750 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dc80b1040 con 0x7f7dd80a4750 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 client.4326.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dd57fa640 4 set_mon_vals no callback set 2024-09-17T16:28:06.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7dbffff640 10 monclient: dump: 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.728 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _renew_subs 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd8122530 con 0x7f7dd80a4750 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.696+0000 7f7ddd836640 1 librados: init done 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7ddd836640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7ddd836640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7ddd836640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd8139970 con 0x7f7dd80a4750 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7dc80aa7b0 con 0x7f7dd80a4750 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 0x7f7dc4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f7dc80e8fa0 con 0x7f7dd80a4750 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 10 client.4326.objecter ms_dispatch 0x7f7dd8000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 3 client.4326.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 3 client.4326.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 20 client.4326.objecter dump_active .. 0 homeless 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dd6ffd640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 0x7f7dc4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dd6ffd640 10 monclient: get_auth_request con 0x7f7dc403e550 auth_method 0 2024-09-17T16:28:06.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dbffff640 10 client.4326.objecter ms_handle_connect 0x7f7dc403e550 2024-09-17T16:28:06.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.700+0000 7f7dd6ffd640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 0x7f7dc4040a10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b800 tx=0x7f7dcc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.704+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7dc8016150 con 0x7f7dd80a4750 2024-09-17T16:28:06.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.704+0000 7f7dbffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.704+0000 7f7dbffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:06.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: monmap: 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.740447+0000 2024-09-17T16:28:06.743 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.740447+0000 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 Processor -- start 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 -- start start 2024-09-17T16:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: init 2024-09-17T16:28:06.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient: _add_conns ranks=[8,1,6,10,0,9,3,5,4,7,2] 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-e con 0x7f1e2c0a5860 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1e2c0a18f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-d con 0x7f1e2c0a4070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:06.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2c0a1d70 con 0x7f1e2c0a18f0 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f1e2c0a66c0 con 0x7f1e2c0a4070 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1e2c0a33d0 con 0x7f1e2c0a5860 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e334b1640 10 monclient(hunting): authenticate will time out at 970.006092s 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34194/0 (socket says 172.21.15.105:34194) 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33964/0 (socket says 172.21.15.105:33964) 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 1 -- 172.21.15.105:0/1891195310 learned_addr learned my addr 172.21.15.105:0/1891195310 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): get_auth_request con 0x7f1e2c0a4070 auth_method 0 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): get_auth_request con 0x7f1e2c0a5860 auth_method 0 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_done global_id 4335 payload 274 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 msgr2=0x7f1e2c0a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1891195310 >> v1:172.21.15.160:6789/0 conn(0x7f1e2c0a18f0 legacy=0x7f1e2c137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 monclient: found mon.noname-d 2024-09-17T16:28:06.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1891195310 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c0a36b0 con 0x7f1e2c0a4070 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.736+0000 7f1e3124d640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e31a4e640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009df0 tx=0x7f1e2401c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c01369197468886 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 1 -- 172.21.15.105:0/1891195310 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e240b2070 con 0x7f1e2c0a4070 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: dump: 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.761 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.762 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 1 -- 172.21.15.105:0/1891195310 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240a5a80 con 0x7f1e2c0a4070 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 5 monclient: authenticate success, global_id 4335 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 4 set_mon_vals no callback set 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 1 -- 172.21.15.105:0/1891195310 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e240a5cb0 con 0x7f1e2c0a4070 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e23fff640 10 monclient: dump: 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.764 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 10 monclient: shutdown 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 msgr2=0x7f1e2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009df0 tx=0x7f1e2401c850 comp rx=0 tx=0).stop 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- 172.21.15.105:0/1891195310 shutdown_connections 2024-09-17T16:28:06.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1e2c0a4070 0x7f1e2c0a6180 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1891195310 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a5860 0x7f1e2c0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- 172.21.15.105:0/1891195310 >> 172.21.15.105:0/1891195310 conn(0x7f1e2c0aa450 msgr2=0x7f1e2c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- 172.21.15.105:0/1891195310 shutdown_connections 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- 172.21.15.105:0/1891195310 wait complete. 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 10 monclient: monmap: 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:06.766 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 librados: starting msgr at 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 librados: starting objecter 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 Processor -- start 2024-09-17T16:28:06.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 -- start start 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 librados: setting wanted keys 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 1 librados: calling monclient init 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.740+0000 7f1e334b1640 10 monclient: init 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _add_conns ranks=[8,3,4,0,6,5,2,1,7] 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-e con 0x7f1e2c0a18f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 0x7f1e2c1d2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-g con 0x7f1e2c0a4070 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): picked mon.noname-c con 0x7f1e2c0a5860 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:06.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2c142490 con 0x7f1e2c0a4070 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1e2c142310 con 0x7f1e2c0a5860 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1e2c142610 con 0x7f1e2c0a18f0 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient(hunting): authenticate will time out at 970.012252s 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:57616/0 (socket says 172.21.15.105:57616) 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34210/0 (socket says 172.21.15.105:34210) 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1395899797 learned_addr learned my addr 172.21.15.105:0/1395899797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): get_auth_request con 0x7f1e2c0a5860 auth_method 0 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): get_auth_request con 0x7f1e2c0a18f0 auth_method 0 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 msgr2=0x7f1e2c1d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e30a4c640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 0x7f1e2c1d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 msgr2=0x7f1e2c1d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 0x7f1e2c1d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 monclient: found mon.noname-c 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 -- 172.21.15.105:0/1395899797 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c13ca20 con 0x7f1e2c0a5860 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e30a4c640 10 monclient: get_auth_request con 0x7f1e2c0a4070 auth_method 0 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e30a4c640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 0x7f1e2c1d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:06.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e31a4e640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e2401cd30 tx=0x7f1e240195f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 client.?.objecter ms_handle_connect 0x7f1e2c0a5860 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e240b2050 con 0x7f1e2c0a5860 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: dump: 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.774 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.775 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240be070 con 0x7f1e2c0a5860 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 5 monclient: authenticate success, global_id 4279 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _renew_subs 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c13cd00 con 0x7f1e2c0a5860 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e24003bb0 con 0x7f1e2c0a5860 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 client.4279.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 monclient: dump: 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.777 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.778 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.778 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.778 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.778 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.779 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e237fe640 4 set_mon_vals no callback set 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1e24004040 con 0x7f1e2c0a5860 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _renew_subs 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c0a0420 con 0x7f1e2c0a5860 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 librados: init done 2024-09-17T16:28:06.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 0x7f1e10040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f1e24003d70 con 0x7f1e2c0a5860 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 client.4279.objecter ms_dispatch 0x7f1e2c000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 3 client.4279.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 3 client.4279.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 20 client.4279.objecter dump_active .. 0 homeless 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 0x7f1e10040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 10 monclient: get_auth_request con 0x7f1e1003e550 auth_method 0 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c1375f0 con 0x7f1e2c0a5860 2024-09-17T16:28:06.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e3124d640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 0x7f1e10040a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e280096f0 tx=0x7f1e28009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.744+0000 7f1e21ffb640 10 client.4279.objecter ms_handle_connect 0x7f1e1003e550 2024-09-17T16:28:06.782 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T16:28:06.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.756+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e24016560 con 0x7f1e2c0a5860 2024-09-17T16:28:06.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.756+0000 7f1e21ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.756+0000 7f1e21ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: monmap: 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.841142+0000 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.841142+0000 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.844 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.845 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 Processor -- start 2024-09-17T16:28:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 -- start start 2024-09-17T16:28:06.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: init 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient: _add_conns ranks=[8,4,9,6,2,0,7,5,3,10,1] 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-e con 0x7f880806c110 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f880806c4f0 0x7f880806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-c con 0x7f880806c4f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8808137510 0x7f8808137950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-a con 0x7f8808137510 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f88080a22a0 con 0x7f8808137510 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f88080a10a0 con 0x7f880806c4f0 2024-09-17T16:28:06.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f88080a1380 con 0x7f880806c110 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f880d8fd640 10 monclient(hunting): authenticate will time out at 970.107210s 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f880806c4f0 0x7f880806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34214/0 (socket says 172.21.15.105:34214) 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 -- 172.21.15.105:0/3532874423 learned_addr learned my addr 172.21.15.105:0/3532874423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): get_auth_request con 0x7f880806c110 auth_method 0 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): get_auth_request con 0x7f880806c4f0 auth_method 0 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 1 --2- 172.21.15.105:0/3532874423 >> v2:172.21.15.105:3300/0 conn(0x7f8808137510 0x7f8808137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): get_auth_request con 0x7f8808137510 auth_method 0 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient(hunting): handle_auth_done global_id 4337 payload 274 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 -- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f880806c4f0 msgr2=0x7f880806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 --2- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f880806c4f0 0x7f880806c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 -- 172.21.15.105:0/3532874423 >> v2:172.21.15.105:3300/0 conn(0x7f8808137510 msgr2=0x7f8808137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 --2- 172.21.15.105:0/3532874423 >> v2:172.21.15.105:3300/0 conn(0x7f8808137510 0x7f8808137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 monclient: found mon.noname-e 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.836+0000 7f8806ffd640 1 -- 172.21.15.105:0/3532874423 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88080a1660 con 0x7f880806c110 2024-09-17T16:28:06.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f8806ffd640 1 --2- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009870 tx=0x7f87f801ce80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f0d07a74caf10df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 1 -- 172.21.15.105:0/3532874423 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87f80b0070 con 0x7f880806c110 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: dump: 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.852 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 1 -- 172.21.15.105:0/3532874423 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f80aa810 con 0x7f880806c110 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 1 -- 172.21.15.105:0/3532874423 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87f80a9300 con 0x7f880806c110 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f880d8fd640 5 monclient: authenticate success, global_id 4337 2024-09-17T16:28:06.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f880d8fd640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f880d8fd640 4 set_mon_vals no callback set 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.840+0000 7f88067fc640 10 monclient: dump: 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.854 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: shutdown 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 msgr2=0x7f88080a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009870 tx=0x7f87f801ce80 comp rx=0 tx=0).stop 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- 172.21.15.105:0/3532874423 shutdown_connections 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- 172.21.15.105:0/3532874423 >> v2:172.21.15.105:3300/0 conn(0x7f8808137510 0x7f8808137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f880806c4f0 0x7f880806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- 172.21.15.105:0/3532874423 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f88080a1d60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- 172.21.15.105:0/3532874423 >> 172.21.15.105:0/3532874423 conn(0x7f88080a5840 msgr2=0x7f88080a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- 172.21.15.105:0/3532874423 shutdown_connections 2024-09-17T16:28:06.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- 172.21.15.105:0/3532874423 wait complete. 2024-09-17T16:28:06.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: monmap: 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.860 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 librados: starting msgr at 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 librados: starting objecter 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 Processor -- start 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 -- start start 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 librados: setting wanted keys 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 librados: calling monclient init 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: init 2024-09-17T16:28:06.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient: _add_conns ranks=[8,2,7,5,1,0,6,3,4] 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f880813ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-e con 0x7f880806c110 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f880813ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f8806ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f880813ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34218/0 (socket says 172.21.15.105:34218) 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f87fffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f87fffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59530/0 (socket says 172.21.15.105:59530) 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.844+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-b con 0x7f880806c4f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 1 -- 172.21.15.105:0/711734663 learned_addr learned my addr 172.21.15.105:0/711734663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8808137510 0x7f880813a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): picked mon.noname-i con 0x7f8808137510 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8808143590 con 0x7f880806c4f0 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f88080a4a10 con 0x7f8808137510 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8808143710 con 0x7f880806c110 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient(hunting): authenticate will time out at 970.116337s 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): get_auth_request con 0x7f880806c110 auth_method 0 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): get_auth_request con 0x7f880806c4f0 auth_method 0 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8808137510 0x7f880813a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): get_auth_request con 0x7f8808137510 auth_method 0 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f88077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient(hunting): handle_auth_done global_id 4286 payload 824 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 -- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 msgr2=0x7f880813ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f880813ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 -- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8808137510 msgr2=0x7f880813a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8808137510 0x7f880813a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 monclient: found mon.noname-b 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880811f930 con 0x7f880806c4f0 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f87fffff640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e7f0 tx=0x7f87f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 client.?.objecter ms_handle_connect 0x7f880806c4f0 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:06.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f880806c4f0 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: dump: 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.866 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f000cdb0 con 0x7f880806c4f0 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87f00138e0 con 0x7f880806c4f0 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 monclient: dump: 2024-09-17T16:28:06.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.868 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 5 monclient: authenticate success, global_id 4286 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient: _renew_subs 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f5ce5032640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f5ce5032640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f5ce5032640 10 monclient: monmap: 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.853230+0000 2024-09-17T16:28:06.869 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.853230+0000 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.870 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880811fc10 con 0x7f880806c4f0 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8805ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 client.4286.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient: _renew_subs 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88080a1f20 con 0x7f880806c4f0 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f880d8fd640 1 librados: init done 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f87f001a070 con 0x7f880806c4f0 2024-09-17T16:28:06.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 0x7f87e8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f87f000f040 con 0x7f880806c4f0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 10 client.4286.objecter ms_dispatch 0x7f8808000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 3 client.4286.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 3 client.4286.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8804ff9640 20 client.4286.objecter dump_active .. 0 homeless 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 0x7f87e8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.848+0000 7f8806ffd640 10 monclient: get_auth_request con 0x7f87e803e580 auth_method 0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f880d8fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f880d8fd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f880d8fd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8808137b00 con 0x7f880806c4f0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f8806ffd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 0x7f87e8040a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009870 tx=0x7f87f8004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.852+0000 7f8804ff9640 10 client.4286.objecter ms_handle_connect 0x7f87e803e580 2024-09-17T16:28:06.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87f0011350 con 0x7f880806c4f0 2024-09-17T16:28:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f8804ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f8804ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 Processor -- start 2024-09-17T16:28:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 -- start start 2024-09-17T16:28:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient: init 2024-09-17T16:28:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient: _add_conns ranks=[1,4,9,6,10,3,2,0,7,5,8] 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5cd80244f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5cd80200e0 0x7f5cd80bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-c con 0x7f5cd80200e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd8020500 0x7f5cd80c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-a con 0x7f5cd8020500 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd80248d0 con 0x7f5cd80244f0 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5cd8020a20 con 0x7f5cd8020500 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f5cd8027300 con 0x7f5cd80200e0 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce5032640 10 monclient(hunting): authenticate will time out at 970.126057s 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd8020500 0x7f5cd80c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5cd8020500 0x7f5cd80c17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45244/0 (socket says 172.21.15.105:45244) 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 -- 172.21.15.105:0/1943832695 learned_addr learned my addr 172.21.15.105:0/1943832695 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): get_auth_request con 0x7f5cd8020500 auth_method 0 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient(hunting): handle_auth_done global_id 4365 payload 274 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 -- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5cd80200e0 msgr2=0x7f5cd80bf290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 --2- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5cd80200e0 0x7f5cd80bf290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 -- 172.21.15.105:0/1943832695 >> v1:172.21.15.160:6789/0 conn(0x7f5cd80244f0 legacy=0x7f5cd801f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 monclient: found mon.noname-a 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.856+0000 7f5ce4831640 1 -- 172.21.15.105:0/1943832695 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd8022160 con 0x7f5cd8020500 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.860+0000 7f5ce4831640 1 --2- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd8020500 0x7f5cd80c17e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ce006b150 tx=0x7f5ce009a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e64f9b63535575 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 1 -- 172.21.15.105:0/1943832695 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5ce01278e0 con 0x7f5cd8020500 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: dump: 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 5 monclient: authenticate success, global_id 4365 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 1 -- 172.21.15.105:0/1943832695 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce0127a80 con 0x7f5cd8020500 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 1 -- 172.21.15.105:0/1943832695 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5ce0127cf0 con 0x7f5cd8020500 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5cdeffd640 10 monclient: dump: 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.880 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 4 set_mon_vals no callback set 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 10 monclient: shutdown 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 -- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd8020500 msgr2=0x7f5cd80c17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 --2- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd8020500 0x7f5cd80c17e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ce006b150 tx=0x7f5ce009a060 comp rx=0 tx=0).stop 2024-09-17T16:28:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 -- 172.21.15.105:0/1943832695 shutdown_connections 2024-09-17T16:28:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 --2- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cd8020500 0x7f5cd80c17e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 --2- 172.21.15.105:0/1943832695 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5cd80200e0 0x7f5cd80bf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.868+0000 7f5ce5032640 1 -- 172.21.15.105:0/1943832695 >> 172.21.15.105:0/1943832695 conn(0x7f5cd80077b0 msgr2=0x7f5cd8007bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- 172.21.15.105:0/1943832695 shutdown_connections 2024-09-17T16:28:06.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- 172.21.15.105:0/1943832695 wait complete. 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: monmap: 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:06.891 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 librados: starting msgr at 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 librados: starting objecter 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 Processor -- start 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- start start 2024-09-17T16:28:06.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 librados: setting wanted keys 2024-09-17T16:28:06.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 librados: calling monclient init 2024-09-17T16:28:06.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: init 2024-09-17T16:28:06.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _add_conns ranks=[3,8,1,0,2,7,6,5,4] 2024-09-17T16:28:06.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cd80200e0 0x7f5cd80c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-g con 0x7f5cd80200e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:06.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-e con 0x7f5cd8020500 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cd80244f0 0x7f5cd80bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): picked mon.noname-f con 0x7f5cd80244f0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd801e540 con 0x7f5cd80244f0 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd801e3c0 con 0x7f5cd80200e0 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5cd80c2530 con 0x7f5cd8020500 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient(hunting): authenticate will time out at 970.153697s 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34226/0 (socket says 172.21.15.105:34226) 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 -- 172.21.15.105:0/3289727608 learned_addr learned my addr 172.21.15.105:0/3289727608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f5cd8020500 auth_method 0 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cd80200e0 0x7f5cd80c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): get_auth_request con 0x7f5cd80200e0 auth_method 0 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cd80244f0 0x7f5cd80bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): get_auth_request con 0x7f5cd80244f0 auth_method 0 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce4831640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient(hunting): handle_auth_done global_id 4355 payload 824 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 -- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cd80200e0 msgr2=0x7f5cd80c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cd80200e0 0x7f5cd80c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 -- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cd80244f0 msgr2=0x7f5cd80bd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cd80244f0 0x7f5cd80bd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 monclient: found mon.noname-e 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 -- 172.21.15.105:0/3289727608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd80c06e0 con 0x7f5cd8020500 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdf7fe640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00b450 tx=0x7f5ccc00b910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 client.?.objecter ms_handle_connect 0x7f5cd8020500 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5ccc015020 con 0x7f5cd8020500 2024-09-17T16:28:06.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: dump: 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.900 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ccc011b00 con 0x7f5cd8020500 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 5 monclient: authenticate success, global_id 4355 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _renew_subs 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd802a5e0 con 0x7f5cd8020500 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 client.4355.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _renew_subs 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd80c0400 con 0x7f5cd8020500 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5ce5032640 1 librados: init done 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5ccc011cf0 con 0x7f5cd8020500 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cdcff9640 10 monclient: dump: 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.902 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.884+0000 7f5cde7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5ce5032640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5ce5032640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5ce5032640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc0005090 con 0x7f5cd8020500 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5ccc018020 con 0x7f5cd8020500 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdcff9640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 0x7f5cc4040970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdffff640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 0x7f5cc4040970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdffff640 10 monclient: get_auth_request con 0x7f5cc403e4b0 auth_method 0 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdcff9640 10 client.4355.objecter ms_handle_connect 0x7f5cc403e4b0 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.888+0000 7f5cdffff640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 0x7f5cc4040970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd4009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f5ccc04d780 con 0x7f5cd8020500 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 10 client.4355.objecter ms_dispatch 0x7f5ce0000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 3 client.4355.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 3 client.4355.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 20 client.4355.objecter dump_active .. 0 homeless 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ccc00d040 con 0x7f5cd8020500 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.892+0000 7f5cdcff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:06.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 10 monclient: monmap: 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.918322+0000 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.918322+0000 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.921 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.922 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 1 Processor -- start 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 1 -- start start 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.912+0000 7f4cbea99640 10 monclient: init 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient: _add_conns ranks=[0,5,9,6,7,2,1,10,3,4,8] 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4cb80a6a30 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 0x7f4cb813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-h con 0x7f4cb80a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-a con 0x7f4cb80a7520 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:06.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45256/0 (socket says 172.21.15.105:45256) 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 1 -- 172.21.15.105:0/3689321350 learned_addr learned my addr 172.21.15.105:0/3689321350 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb80a4080 con 0x7f4cb80a6a30 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/3689321350 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 0x7f4cb813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4cb80a7c40 con 0x7f4cb80a7520 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4cb80a37e0 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbea99640 10 monclient(hunting): authenticate will time out at 970.184985s 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): get_auth_request con 0x7f4cb80a7520 auth_method 0 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): get_auth_request con 0x7f4cb80a4ab0 auth_method 0 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_done global_id 4334 payload 274 2024-09-17T16:28:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 -- 172.21.15.105:0/3689321350 >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 msgr2=0x7f4cb8140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/3689321350 >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 -- 172.21.15.105:0/3689321350 >> v1:172.21.15.105:6789/0 conn(0x7f4cb80a6a30 legacy=0x7f4cb80a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 monclient: found mon.noname-h 2024-09-17T16:28:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb77fe640 1 -- 172.21.15.105:0/3689321350 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512018847 0 0) 0x7f4cb80a4080 con 0x7f4cb80a6a30 2024-09-17T16:28:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:28:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb7fff640 1 -- 172.21.15.105:0/3689321350 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb80a3ac0 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cb77fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:28:06.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.916+0000 7f4cbd036640 1 --2- 172.21.15.105:0/3689321350 >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/3689321350 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 0x7f4cb813ea30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40097c0 tx=0x7f4ca401c340 comp rx=0 tx=0).ready entity=mon.5 client_cookie=51a9ecad8f3c8d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 1 -- 172.21.15.105:0/3689321350 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca40b2070 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: dump: 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.933 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 1 -- 172.21.15.105:0/3689321350 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca40ab380 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 5 monclient: authenticate success, global_id 4334 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 1 -- 172.21.15.105:0/3689321350 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca40ab9a0 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cb77fe640 10 monclient: dump: 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.935 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 4 set_mon_vals no callback set 2024-09-17T16:28:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 10 monclient: shutdown 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 msgr2=0x7f4cb813ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 --2- 172.21.15.105:0/3689321350 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 0x7f4cb813ea30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40097c0 tx=0x7f4ca401c340 comp rx=0 tx=0).stop 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 shutdown_connections 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 --2- 172.21.15.105:0/3689321350 >> v2:172.21.15.105:3300/0 conn(0x7f4cb80a7520 0x7f4cb8140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 --2- 172.21.15.105:0/3689321350 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4cb80a4ab0 0x7f4cb813ea30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 >> 172.21.15.105:0/3689321350 conn(0x7f4cb80a8d00 msgr2=0x7f4cb80a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 shutdown_connections 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 1 -- 172.21.15.105:0/3689321350 wait complete. 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.920+0000 7f4cbea99640 10 monclient: monmap: 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 librados: starting msgr at 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 librados: starting objecter 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 Processor -- start 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 -- start start 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 librados: setting wanted keys 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 librados: calling monclient init 2024-09-17T16:28:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient: init 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient: _add_conns ranks=[2,8,1,6,4,0,3,7,5] 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-b con 0x7f4cb80a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34232/0 (socket says 172.21.15.105:34232) 2024-09-17T16:28:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbc835640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbc835640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59538/0 (socket says 172.21.15.105:59538) 2024-09-17T16:28:06.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 -- 172.21.15.105:0/125687271 learned_addr learned my addr 172.21.15.105:0/125687271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-e con 0x7f4cb80a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 0x7f4cb8126390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): picked mon.noname-f con 0x7f4cb80a7520 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8143e60 con 0x7f4cb80a7520 2024-09-17T16:28:06.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb8143ce0 con 0x7f4cb80a4ab0 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4cb8143fe0 con 0x7f4cb80a6a30 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient(hunting): authenticate will time out at 970.193638s 2024-09-17T16:28:06.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): get_auth_request con 0x7f4cb80a6a30 auth_method 0 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbd036640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 0x7f4cb8126390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient(hunting): handle_auth_done global_id 4364 payload 824 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 -- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 msgr2=0x7f4cb81d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 -- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 msgr2=0x7f4cb8126390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 0x7f4cb8126390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 monclient: found mon.noname-e 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb8129a90 con 0x7f4cb80a6a30 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbd036640 10 monclient: get_auth_request con 0x7f4cb80a7520 auth_method 0 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbd036640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 0x7f4cb8126390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb7fff640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4ca401a9f0 tx=0x7f4ca40190e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 client.?.objecter ms_handle_connect 0x7f4cb80a6a30 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbc835640 10 monclient: get_auth_request con 0x7f4cb80a4ab0 auth_method 0 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbc835640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca40ba020 con 0x7f4cb80a6a30 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: dump: 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.952 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca4004340 con 0x7f4cb80a6a30 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca40b2040 con 0x7f4cb80a6a30 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb57fa640 10 monclient: dump: 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 5 monclient: authenticate success, global_id 4364 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cb6ffd640 4 set_mon_vals no callback set 2024-09-17T16:28:06.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient: _renew_subs 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.924+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb8129d70 con 0x7f4cb80a6a30 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4ca40bd020 con 0x7f4cb80a6a30 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 0x7f4c9c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 client.4364.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 monclient: _renew_subs 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb81285d0 con 0x7f4cb80a6a30 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 1 librados: init done 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7f4ca40ab530 con 0x7f4cb80a6a30 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 10 client.4364.objecter ms_dispatch 0x7f4cb8000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 3 client.4364.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 3 client.4364.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:06.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cb57fa640 20 client.4364.objecter dump_active .. 0 homeless 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.928+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb80a6e10 con 0x7f4cb80a6a30 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cbc835640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 0x7f4c9c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cbc835640 10 monclient: get_auth_request con 0x7f4c9c03e550 auth_method 0 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cb57fa640 10 client.4364.objecter ms_handle_connect 0x7f4c9c03e550 2024-09-17T16:28:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cbc835640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 0x7f4c9c040a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cac007fd0 tx=0x7f4cac009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4ca4016050 con 0x7f4cb80a6a30 2024-09-17T16:28:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cb57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.936+0000 7f4cb57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e334b1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e334b1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e334b1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1e2c13b280 con 0x7f1e2c0a5860 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e21ffb640 1 -- 172.21.15.105:0/1395899797 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e24004330 con 0x7f1e2c0a5860 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e21ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-17T16:28:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.948+0000 7f1e21ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:06.960 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 10 librados: watch_flush enter 2024-09-17T16:28:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 10 librados: watch_flush exit 2024-09-17T16:28:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 20 client.4279.objecter shutdown clearing up homeless session... 2024-09-17T16:28:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 10 client.4279.objecter successfully canceled tick 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 msgr2=0x7f1e10040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 0x7f1e10040a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e280096f0 tx=0x7f1e28009240 comp rx=0 tx=0).stop 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 10 monclient: shutdown 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 msgr2=0x7f1e2c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1e2401cd30 tx=0x7f1e240195f0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 shutdown_connections 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1e1003e550 0x7f1e10040a10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e2c0a4070 0x7f1e2c1d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1e2c0a5860 0x7f1e2c06d220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 --2- 172.21.15.105:0/1395899797 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1e2c0a18f0 0x7f1e2c1d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 >> 172.21.15.105:0/1395899797 conn(0x7f1e2c0aa450 msgr2=0x7f1e2c0a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 shutdown_connections 2024-09-17T16:28:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 -- 172.21.15.105:0/1395899797 wait complete. 2024-09-17T16:28:06.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.952+0000 7f1e334b1640 1 librados: shutdown 2024-09-17T16:28:06.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-17T16:28:06.980 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.972+0000 7fc37a830640 10 monclient: get_monmap_and_config 2024-09-17T16:28:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.972+0000 7fc37a830640 10 monclient: build_initial_monmap 2024-09-17T16:28:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.972+0000 7fc37a830640 10 monclient: monmap: 2024-09-17T16:28:06.988 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:06.979661+0000 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:06.979661+0000 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:06.989 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 Processor -- start 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 -- start start 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient: init 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient: _add_conns ranks=[4,0,3,5,9,7,2,6,1,8,10] 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 0x7fc37409b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-c con 0x7fc3740691f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:06.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc3740a2310 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 0x7fc37413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-g con 0x7fc37413ca00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3740a20d0 con 0x7fc3740a2310 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc3737fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc3740a2310 0x7fc37409b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37228/0 (socket says 172.21.15.105:37228) 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 0x7fc37409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc3737fe640 1 -- 172.21.15.105:0/3459193841 learned_addr learned my addr 172.21.15.105:0/3459193841 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc3740a42b0 con 0x7fc37413ca00 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc37409e850 con 0x7fc3740691f0 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc37a830640 10 monclient(hunting): authenticate will time out at 970.245448s 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): get_auth_request con 0x7fc3740691f0 auth_method 0 2024-09-17T16:28:06.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 0x7fc37413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): get_auth_request con 0x7fc37413ca00 auth_method 0 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 1 -- 172.21.15.105:0/3459193841 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655002927 0 0) 0x7fc3740a20d0 con 0x7fc3740a2310 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 10 monclient(hunting): _negotiate 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:06.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 10 monclient(hunting): my global_id is 4392 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc372ffd640 1 -- 172.21.15.105:0/3459193841 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc35c003530 con 0x7fc3740a2310 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient(hunting): handle_auth_done global_id 4297 payload 274 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 -- 172.21.15.105:0/3459193841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 msgr2=0x7fc37413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 0x7fc37413edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 -- 172.21.15.105:0/3459193841 >> v1:172.21.15.105:6789/0 conn(0x7fc3740a2310 legacy=0x7fc37409b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 monclient: found mon.noname-c 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:06.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc373fff640 1 -- 172.21.15.105:0/3459193841 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37409c660 con 0x7fc3740691f0 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.976+0000 7fc378dcd640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 0x7fc37413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc373fff640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 0x7fc37409b050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3640097f0 tx=0x7fc36401c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=12909320d914bf3c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 1 -- 172.21.15.105:0/3459193841 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3640b2070 con 0x7fc3740691f0 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: dump: 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.994 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 1 -- 172.21.15.105:0/3459193841 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3640ab7f0 con 0x7fc3740691f0 2024-09-17T16:28:06.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 5 monclient: authenticate success, global_id 4297 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 1 -- 172.21.15.105:0/3459193841 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3640abe10 con 0x7fc3740691f0 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc372ffd640 10 monclient: dump: 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:06.996 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:06.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 4 set_mon_vals no callback set 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 10 monclient: shutdown 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 msgr2=0x7fc37409b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 0x7fc37409b050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc3640097f0 tx=0x7fc36401c3a0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 shutdown_connections 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc37413ca00 0x7fc37413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 --2- 172.21.15.105:0/3459193841 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740691f0 0x7fc37409b050 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:06.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 >> 172.21.15.105:0/3459193841 conn(0x7fc3740a8620 msgr2=0x7fc3740a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7ddd836640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7ddd836640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7ddd836640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7dd806e230 con 0x7f7dd80a4750 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 shutdown_connections 2024-09-17T16:28:06.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 1 -- 172.21.15.105:0/3459193841 wait complete. 2024-09-17T16:28:07.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7dbffff640 1 -- 172.21.15.105:0/2440457553 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dc80e6c20 con 0x7f7dd80a4750 2024-09-17T16:28:07.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7dbffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-17T16:28:07.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7f7dbffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:07.000 INFO:teuthology.orchestra.run.smithi105.stdout:51539607557 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.980+0000 7fc37a830640 10 monclient: monmap: 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:07.003 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 librados: starting msgr at 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 librados: starting objecter 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 Processor -- start 2024-09-17T16:28:07.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- start start 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 librados: setting wanted keys 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 librados: calling monclient init 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: init 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _add_conns ranks=[4,6,5,0,1,2,7,8,3] 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 0x7fc374070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-c con 0x7fc3740a2310 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-d con 0x7fc37413ca00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 0x7fc37409a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): picked mon.noname-h con 0x7fc374070ad0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc37409fce0 con 0x7fc3740a2310 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc37409fb60 con 0x7fc374070ad0 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc37409fe60 con 0x7fc37413ca00 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient(hunting): authenticate will time out at 970.252920s 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33966/0 (socket says 172.21.15.105:33966) 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 -- 172.21.15.105:0/3643454417 learned_addr learned my addr 172.21.15.105:0/3643454417 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): get_auth_request con 0x7fc37413ca00 auth_method 0 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 0x7fc374070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 10 monclient(hunting): get_auth_request con 0x7fc3740a2310 auth_method 0 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc378dcd640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 0x7fc37409a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient(hunting): handle_auth_done global_id 4344 payload 824 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 -- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 msgr2=0x7fc37409a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 0x7fc37409a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 -- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 msgr2=0x7fc374070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 0x7fc374070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:07.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 -- 172.21.15.105:0/3643454417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc374073cc0 con 0x7fc37413ca00 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc378dcd640 10 monclient: get_auth_request con 0x7fc374070ad0 auth_method 0 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc378dcd640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 0x7fc37409a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc373fff640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 0x7fc374070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3737fe640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc368001c60 tx=0x7fc368014420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 client.?.objecter ms_handle_connect 0x7fc37413ca00 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc368014d40 con 0x7fc37413ca00 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: dump: 2024-09-17T16:28:07.008 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.009 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc368005d30 con 0x7fc37413ca00 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc3727fc640 4 set_mon_vals no callback set 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 10 librados: watch_flush enter 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 10 librados: watch_flush exit 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 20 client.4326.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 10 client.4326.objecter successfully canceled tick 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 msgr2=0x7f7dc4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 0x7f7dc4040a10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b800 tx=0x7f7dcc008040 comp rx=0 tx=0).stop 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 10 monclient: shutdown 2024-09-17T16:28:07.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 msgr2=0x7f7dd811f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dc80af040 tx=0x7f7dc80191e0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 shutdown_connections 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7dc403e550 0x7f7dc4040a10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7dd8139530 0x7f7dd811fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7dd80a4750 0x7f7dd811f8d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 --2- 172.21.15.105:0/2440457553 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7dd809eec0 0x7f7dd811f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 >> 172.21.15.105:0/2440457553 conn(0x7f7dd80a5840 msgr2=0x7f7dd80a6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 shutdown_connections 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 -- 172.21.15.105:0/2440457553 wait complete. 2024-09-17T16:28:07.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7f7ddd836640 1 librados: shutdown 2024-09-17T16:28:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 5 monclient: authenticate success, global_id 4344 2024-09-17T16:28:07.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc368010cc0 con 0x7fc37413ca00 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc370ff9640 10 monclient: dump: 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.014 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _renew_subs 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3740a18e0 con 0x7fc37413ca00 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 client.4344.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _renew_subs 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc374072650 con 0x7fc37413ca00 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 librados: init done 2024-09-17T16:28:07.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.984+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3740008e0 con 0x7fc37413ca00 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc368010850 con 0x7fc37413ca00 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 0x7fc3600409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fc368018030 con 0x7fc37413ca00 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 10 client.4344.objecter ms_dispatch 0x7fc374000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 3 client.4344.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 3 client.4344.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 20 client.4344.objecter dump_active .. 0 homeless 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc373fff640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 0x7fc3600409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc373fff640 10 monclient: get_auth_request con 0x7fc36003e4e0 auth_method 0 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc370ff9640 10 client.4344.objecter ms_handle_connect 0x7fc36003e4e0 2024-09-17T16:28:07.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.988+0000 7fc373fff640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 0x7fc3600409a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc3640af000 tx=0x7fc364008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.996+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc368007110 con 0x7fc37413ca00 2024-09-17T16:28:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.996+0000 7fc370ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:06.996+0000 7fc370ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:07.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.3 2024-09-17T16:28:07.018 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:07.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:07.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5cc0005650 con 0x7f5cd8020500 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5cdcff9640 1 -- 172.21.15.105:0/3289727608 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ccc00f340 con 0x7f5cd8020500 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5cdcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5cdcff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:07.087 INFO:teuthology.orchestra.run.smithi105.stdout:51539607557 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f880d8fd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f880d8fd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f880d8fd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f880806c910 con 0x7f880806c4f0 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f8804ff9640 1 -- 172.21.15.105:0/711734663 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87f0013e10 con 0x7f880806c4f0 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f8804ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f8804ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:07.088 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 librados: watch_flush enter 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 librados: watch_flush exit 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 20 client.4355.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 client.4355.objecter successfully canceled tick 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 msgr2=0x7f5cc4040970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 0x7f5cc4040970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4007fd0 tx=0x7f5cd4009040 comp rx=0 tx=0).stop 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 10 monclient: shutdown 2024-09-17T16:28:07.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 msgr2=0x7f5cd80bcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00b450 tx=0x7f5ccc00b910 comp rx=0 tx=0).stop 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 shutdown_connections 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5cc403e4b0 0x7f5cc4040970 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cd80244f0 0x7f5cd80bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5cd8020500 0x7f5cd80bcc60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 --2- 172.21.15.105:0/3289727608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cd80200e0 0x7f5cd80c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 >> 172.21.15.105:0/3289727608 conn(0x7f5cd80077b0 msgr2=0x7f5cd8028ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 shutdown_connections 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 -- 172.21.15.105:0/3289727608 wait complete. 2024-09-17T16:28:07.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.080+0000 7f5ce5032640 1 librados: shutdown 2024-09-17T16:28:07.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 10 librados: watch_flush enter 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 10 librados: watch_flush exit 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 20 client.4286.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 10 client.4286.objecter successfully canceled tick 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 msgr2=0x7f87e8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 0x7f87e8040a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009870 tx=0x7f87f8004740 comp rx=0 tx=0).stop 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 10 monclient: shutdown 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 msgr2=0x7f88081d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e7f0 tx=0x7f87f000ecb0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 shutdown_connections 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f87e803e580 0x7f87e8040a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8808137510 0x7f880813a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f880806c4f0 0x7f88081d1c10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 --2- 172.21.15.105:0/711734663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f880806c110 0x7f880813ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 >> 172.21.15.105:0/711734663 conn(0x7f88080a5840 msgr2=0x7f88080a7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 shutdown_connections 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 -- 172.21.15.105:0/711734663 wait complete. 2024-09-17T16:28:07.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.084+0000 7f880d8fd640 1 librados: shutdown 2024-09-17T16:28:07.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.4 2024-09-17T16:28:07.107 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:07.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.5 2024-09-17T16:28:07.114 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4cb8140b70 con 0x7f4cb80a6a30 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cb57fa640 1 -- 172.21.15.105:0/125687271 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ca40b5370 con 0x7f4cb80a6a30 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cb57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-17T16:28:07.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cb57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:07.157 INFO:teuthology.orchestra.run.smithi105.stdout:51539607558 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 librados: watch_flush enter 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 librados: watch_flush exit 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 20 client.4364.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 client.4364.objecter successfully canceled tick 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 msgr2=0x7f4c9c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 0x7f4c9c040a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4cac007fd0 tx=0x7f4cac009040 comp rx=0 tx=0).stop 2024-09-17T16:28:07.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 10 monclient: shutdown 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 msgr2=0x7f4cb81d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4ca401a9f0 tx=0x7f4ca40190e0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 shutdown_connections 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4c9c03e550 0x7f4c9c040a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cb80a7520 0x7f4cb8126390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cb80a6a30 0x7f4cb81d16c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 --2- 172.21.15.105:0/125687271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cb80a4ab0 0x7f4cb81d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.148+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 >> 172.21.15.105:0/125687271 conn(0x7f4cb80a8d00 msgr2=0x7f4cb809fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.152+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 shutdown_connections 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.152+0000 7f4cbea99640 1 -- 172.21.15.105:0/125687271 wait complete. 2024-09-17T16:28:07.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.152+0000 7f4cbea99640 1 librados: shutdown 2024-09-17T16:28:07.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc37a830640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:07.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc37a830640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:07.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc37a830640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc374074470 con 0x7fc37413ca00 2024-09-17T16:28:07.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc370ff9640 1 -- 172.21.15.105:0/3643454417 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc368057080 con 0x7fc37413ca00 2024-09-17T16:28:07.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc370ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-17T16:28:07.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.160+0000 7fc370ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:07.169 INFO:teuthology.orchestra.run.smithi105.stdout:47244640261 2024-09-17T16:28:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 10 librados: watch_flush enter 2024-09-17T16:28:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 10 librados: watch_flush exit 2024-09-17T16:28:07.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 20 client.4344.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 10 client.4344.objecter successfully canceled tick 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 msgr2=0x7fc3600409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 0x7fc3600409a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc3640af000 tx=0x7fc364008040 comp rx=0 tx=0).stop 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 10 monclient: shutdown 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 msgr2=0x7fc374070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc368001c60 tx=0x7fc368014420 comp rx=0 tx=0).stop 2024-09-17T16:28:07.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 shutdown_connections 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc36003e4e0 0x7fc3600409a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc374070ad0 0x7fc37409a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc37413ca00 0x7fc374070590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 --2- 172.21.15.105:0/3643454417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc3740a2310 0x7fc374070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 >> 172.21.15.105:0/3643454417 conn(0x7fc3740a8620 msgr2=0x7fc3740a4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 shutdown_connections 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 -- 172.21.15.105:0/3643454417 wait complete. 2024-09-17T16:28:07.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.164+0000 7fc37a830640 1 librados: shutdown 2024-09-17T16:28:07.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.0 2024-09-17T16:28:07.174 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:07.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2024-09-17T16:28:07.186 DEBUG:teuthology.parallel:result is None 2024-09-17T16:28:07.186 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T16:28:07.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.328+0000 7fc44fe36640 10 monclient: get_monmap_and_config 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.328+0000 7fc44fe36640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.328+0000 7fc44fe36640 10 monclient: monmap: 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:07.334538+0000 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:07.334538+0000 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 Processor -- start 2024-09-17T16:28:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 -- start start 2024-09-17T16:28:07.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient: init 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient: _add_conns ranks=[6,9,7,3,1,4,10,2,0,8,5] 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-d con 0x7fc448139a80 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-a con 0x7fc448139e60 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 0x7fc4480a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-i con 0x7fc4480a4c90 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc44806c4e0 con 0x7fc448139e60 2024-09-17T16:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc4480a65b0 con 0x7fc448139a80 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc44809e630 con 0x7fc4480a4c90 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44fe36640 10 monclient(hunting): authenticate will time out at 970.600972s 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45264/0 (socket says 172.21.15.105:45264) 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33974/0 (socket says 172.21.15.105:33974) 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 1 -- 172.21.15.105:0/476617702 learned_addr learned my addr 172.21.15.105:0/476617702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): get_auth_request con 0x7fc448139e60 auth_method 0 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): get_auth_request con 0x7fc448139a80 auth_method 0 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44e3d3640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 0x7fc4480a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_done global_id 4353 payload 274 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 -- 172.21.15.105:0/476617702 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 msgr2=0x7fc4480a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 0x7fc4480a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 -- 172.21.15.105:0/476617702 >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 msgr2=0x7fc4480a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/476617702 >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 monclient: found mon.noname-d 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44dbd2640 1 -- 172.21.15.105:0/476617702 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44806cbe0 con 0x7fc448139a80 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44e3d3640 10 monclient: get_auth_request con 0x7fc4480a4c90 auth_method 0 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44d3d1640 1 --2- 172.21.15.105:0/476617702 >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:07.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.332+0000 7fc44e3d3640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 0x7fc4480a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097c0 tx=0x7fc44401c360 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c99906289465e07c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 1 -- 172.21.15.105:0/476617702 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4440b0070 con 0x7fc448139a80 2024-09-17T16:28:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: dump: 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 5 monclient: authenticate success, global_id 4353 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 1 -- 172.21.15.105:0/476617702 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4440aa810 con 0x7fc448139a80 2024-09-17T16:28:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 4 set_mon_vals no callback set 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 1 -- 172.21.15.105:0/476617702 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4440a4e60 con 0x7fc448139a80 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44cbd0640 10 monclient: dump: 2024-09-17T16:28:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: shutdown 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/476617702 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 msgr2=0x7fc44809ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097c0 tx=0x7fc44401c360 comp rx=0 tx=0).stop 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/476617702 shutdown_connections 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc4480a4c90 0x7fc4480a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- 172.21.15.105:0/476617702 >> v2:172.21.15.105:3300/0 conn(0x7fc448139e60 0x7fc4480a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- 172.21.15.105:0/476617702 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc448139a80 0x7fc44809ea50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/476617702 >> 172.21.15.105:0/476617702 conn(0x7fc4480a94d0 msgr2=0x7fc4480a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/476617702 shutdown_connections 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/476617702 wait complete. 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: monmap: 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 librados: starting msgr at 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 librados: starting objecter 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 Processor -- start 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- start start 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 librados: setting wanted keys 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 librados: calling monclient init 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: init 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _add_conns ranks=[8,7,2,0,1,4,3,6,5] 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-e con 0x7fc4480a4c90 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 0x7fc4481d1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-i con 0x7fc448139a80 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 0x7fc44813a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): picked mon.noname-b con 0x7fc448139e60 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc44813a3c0 con 0x7fc448139e60 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc44813a240 con 0x7fc448139a80 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc448142fa0 con 0x7fc4480a4c90 2024-09-17T16:28:07.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient(hunting): authenticate will time out at 970.606435s 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 0x7fc44813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34244/0 (socket says 172.21.15.105:34244) 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 1 -- 172.21.15.105:0/3709806162 learned_addr learned my addr 172.21.15.105:0/3709806162 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): get_auth_request con 0x7fc4480a4c90 auth_method 0 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): get_auth_request con 0x7fc448139e60 auth_method 0 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44d3d1640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 0x7fc4481d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient(hunting): handle_auth_done global_id 4295 payload 824 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 -- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 msgr2=0x7fc4481d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 -- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 msgr2=0x7fc4481d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 0x7fc4481d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 monclient: found mon.noname-b 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 -- 172.21.15.105:0/3709806162 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4481403b0 con 0x7fc448139e60 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44d3d1640 10 monclient: get_auth_request con 0x7fc448139a80 auth_method 0 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44d3d1640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 0x7fc4481d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44e3d3640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 0x7fc44813a540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc44000cc90 tx=0x7fc44000a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 client.?.objecter ms_handle_connect 0x7fc448139e60 2024-09-17T16:28:07.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc44000f8f0 con 0x7fc448139e60 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: dump: 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44000fad0 con 0x7fc448139e60 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc44000fc80 con 0x7fc448139e60 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:07.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43e7fc640 10 monclient: dump: 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.451 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 5 monclient: authenticate success, global_id 4295 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc43ffff640 4 set_mon_vals no callback set 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _renew_subs 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44813dec0 con 0x7fc448139e60 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 client.4295.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _renew_subs 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc448133640 con 0x7fc448139e60 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.336+0000 7fc44fe36640 1 librados: init done 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44fe36640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44fe36640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44fe36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc448138730 con 0x7fc448139e60 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc44000f480 con 0x7fc448139e60 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 0x7fc42c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4402+0+0 (secure 0 0 0) 0x7fc4400503e0 con 0x7fc448139e60 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 10 client.4295.objecter ms_dispatch 0x7fc448000ba0 osd_map(19..19 src has 1..19) 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 3 client.4295.objecter handle_osd_map got epochs [19,19] > 0 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 3 client.4295.objecter handle_osd_map decoding full epoch 19 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 20 client.4295.objecter dump_active .. 0 homeless 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 0x7fc42c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44dbd2640 10 monclient: get_auth_request con 0x7fc42c03e550 auth_method 0 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc43e7fc640 10 client.4295.objecter ms_handle_connect 0x7fc42c03e550 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.340+0000 7fc44dbd2640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 0x7fc42c040a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc444009710 tx=0x7fc4440a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.344+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc440014030 con 0x7fc448139e60 2024-09-17T16:28:07.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.344+0000 7fc43e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.344+0000 7fc43e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.504+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc44809ea50 con 0x7fc42c03e550 2024-09-17T16:28:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.504+0000 7fc43e7fc640 1 -- 172.21.15.105:0/3709806162 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34074 (secure 0 0 0) 0x7fc4440c3530 con 0x7fc42c03e550 2024-09-17T16:28:07.514 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:07.514 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 10 librados: watch_flush enter 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 10 librados: watch_flush exit 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 20 client.4295.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 10 client.4295.objecter successfully canceled tick 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 msgr2=0x7fc42c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 0x7fc42c040a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc444009710 tx=0x7fc4440a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 10 monclient: shutdown 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 msgr2=0x7fc44813a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 0x7fc44813a540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc44000cc90 tx=0x7fc44000a650 comp rx=0 tx=0).stop 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 shutdown_connections 2024-09-17T16:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc42c03e550 0x7fc42c040a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc448139e60 0x7fc44813a540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc448139a80 0x7fc4481d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 --2- 172.21.15.105:0/3709806162 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc4480a4c90 0x7fc4481d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 >> 172.21.15.105:0/3709806162 conn(0x7fc4480a94d0 msgr2=0x7fc4480a7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 shutdown_connections 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 -- 172.21.15.105:0/3709806162 wait complete. 2024-09-17T16:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.508+0000 7fc44fe36640 1 librados: shutdown 2024-09-17T16:28:07.531 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T16:28:07.363959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164368,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066672,"statfs":{"total":579820584960,"available":579652272128,"internally_reserved":0,"allocated":2277376,"data_stored":1543210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.923700"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452937+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:28:03.452937+0000","last_peered":"2024-09-17T16:28:03.452937+0000","last_clean":"2024-09-17T16:28:03.452937+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:28:03.452937+0000","last_undegraded":"2024-09-17T16:28:03.452937+0000","last_fullsized":"2024-09-17T16:28:03.452937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462830+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:28:03.462830+0000","last_peered":"2024-09-17T16:28:03.462830+0000","last_clean":"2024-09-17T16:28:03.462830+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:28:03.462830+0000","last_undegraded":"2024-09-17T16:28:03.462830+0000","last_fullsized":"2024-09-17T16:28:03.462830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.451997+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:28:03.451997+0000","last_peered":"2024-09-17T16:28:03.451997+0000","last_clean":"2024-09-17T16:28:03.451997+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:28:03.451997+0000","last_undegraded":"2024-09-17T16:28:03.451997+0000","last_fullsized":"2024-09-17T16:28:03.451997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462976+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:28:03.462976+0000","last_peered":"2024-09-17T16:28:03.462976+0000","last_clean":"2024-09-17T16:28:03.462976+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:28:03.462976+0000","last_undegraded":"2024-09-17T16:28:03.462976+0000","last_fullsized":"2024-09-17T16:28:03.462976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.461846+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:28:03.461846+0000","last_peered":"2024-09-17T16:28:03.461846+0000","last_clean":"2024-09-17T16:28:03.461846+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:28:03.461846+0000","last_undegraded":"2024-09-17T16:28:03.461846+0000","last_fullsized":"2024-09-17T16:28:03.461846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.446906+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:28:03.446906+0000","last_peered":"2024-09-17T16:28:03.446906+0000","last_clean":"2024-09-17T16:28:03.446906+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:28:03.446906+0000","last_undegraded":"2024-09-17T16:28:03.446906+0000","last_fullsized":"2024-09-17T16:28:03.446906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452726+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:28:03.452726+0000","last_peered":"2024-09-17T16:28:03.452726+0000","last_clean":"2024-09-17T16:28:03.452726+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:28:03.452726+0000","last_undegraded":"2024-09-17T16:28:03.452726+0000","last_fullsized":"2024-09-17T16:28:03.452726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.456843+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:28:03.456843+0000","last_peered":"2024-09-17T16:28:03.456843+0000","last_clean":"2024-09-17T16:28:03.456843+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:28:03.456843+0000","last_undegraded":"2024-09-17T16:28:03.456843+0000","last_fullsized":"2024-09-17T16:28:03.456843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.450590+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:28:03.450590+0000","last_peered":"2024-09-17T16:28:03.450590+0000","last_clean":"2024-09-17T16:28:03.450590+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:28:03.450590+0000","last_undegraded":"2024-09-17T16:28:03.450590+0000","last_fullsized":"2024-09-17T16:28:03.450590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:28:07.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 10 monclient: get_monmap_and_config 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 10 monclient: monmap: 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:07.646496+0000 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:07.646496+0000 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.649 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:07.650 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 1 Processor -- start 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 1 -- start start 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.640+0000 7f7794417640 10 monclient: init 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient: _add_conns ranks=[2,1,7,6,9,5,0,3,8,4,10] 2024-09-17T16:28:07.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): picked mon.noname-b con 0x7f778c137660 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f778c137f80 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 0x7f778c0a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): picked mon.noname-i con 0x7f778c0a1d50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778c09e790 con 0x7f778c137f80 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f778c06dbe0 con 0x7f778c137660 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f778c0774a0 con 0x7f778c0a1d50 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient(hunting): authenticate will time out at 970.912301s 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59558/0 (socket says 172.21.15.105:59558) 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 -- 172.21.15.105:0/1883287099 learned_addr learned my addr 172.21.15.105:0/1883287099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): get_auth_request con 0x7f778c137660 auth_method 0 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77929b4640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 0x7f778c0a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient(hunting): handle_auth_done global_id 4304 payload 274 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 -- 172.21.15.105:0/1883287099 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 msgr2=0x7f778c0a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 0x7f778c0a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 -- 172.21.15.105:0/1883287099 >> v1:172.21.15.160:6789/0 conn(0x7f778c137f80 legacy=0x7f778c0a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 monclient: found mon.noname-b 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 -- 172.21.15.105:0/1883287099 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c09c300 con 0x7f778c137660 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77929b4640 10 monclient: get_auth_request con 0x7f778c0a1d50 auth_method 0 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77929b4640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 0x7f778c0a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:07.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77921b3640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f778c09c4d0 tx=0x7f778801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3db4ac824a8167d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 1 -- 172.21.15.105:0/1883287099 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77880b0070 con 0x7f778c137660 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: dump: 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 1 -- 172.21.15.105:0/1883287099 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77880aa810 con 0x7f778c137660 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 1 -- 172.21.15.105:0/1883287099 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77880a9300 con 0x7f778c137660 2024-09-17T16:28:07.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 5 monclient: authenticate success, global_id 4304 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 4 set_mon_vals no callback set 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f77911b1640 10 monclient: dump: 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 10 monclient: shutdown 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- 172.21.15.105:0/1883287099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 msgr2=0x7f778c137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f778c09c4d0 tx=0x7f778801c3c0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- 172.21.15.105:0/1883287099 shutdown_connections 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f778c0a1d50 0x7f778c0a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 --2- 172.21.15.105:0/1883287099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137660 0x7f778c137a40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- 172.21.15.105:0/1883287099 >> 172.21.15.105:0/1883287099 conn(0x7f778c0a4bf0 msgr2=0x7f778c0a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- 172.21.15.105:0/1883287099 shutdown_connections 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.644+0000 7f7794417640 1 -- 172.21.15.105:0/1883287099 wait complete. 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: monmap: 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:07.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 librados: starting msgr at 2024-09-17T16:28:07.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 librados: starting objecter 2024-09-17T16:28:07.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 Processor -- start 2024-09-17T16:28:07.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- start start 2024-09-17T16:28:07.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 librados: setting wanted keys 2024-09-17T16:28:07.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 librados: calling monclient init 2024-09-17T16:28:07.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: init 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _add_conns ranks=[8,1,2,7,3,0,4,6,5] 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f778c0a1d50 0x7f778c13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): picked mon.noname-e con 0x7f778c0a1d50 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f778c137660 0x7f778c13e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): picked mon.noname-f con 0x7f778c137660 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): picked mon.noname-b con 0x7f778c137f80 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f778c141e20 con 0x7f778c137660 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f778c0a4960 con 0x7f778c137f80 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f778c141fa0 con 0x7f778c0a1d50 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient(hunting): authenticate will time out at 970.917086s 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59566/0 (socket says 172.21.15.105:59566) 2024-09-17T16:28:07.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 -- 172.21.15.105:0/3585419845 learned_addr learned my addr 172.21.15.105:0/3585419845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): get_auth_request con 0x7f778c137f80 auth_method 0 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f778c0a1d50 0x7f778c13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): get_auth_request con 0x7f778c0a1d50 auth_method 0 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77919b2640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f778c137660 0x7f778c13e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77919b2640 10 monclient(hunting): get_auth_request con 0x7f778c137660 auth_method 0 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77919b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77919b2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77919b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient(hunting): handle_auth_done global_id 4313 payload 824 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 -- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f778c0a1d50 msgr2=0x7f778c13dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f778c0a1d50 0x7f778c13dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 -- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f778c137660 msgr2=0x7f778c13e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f778c137660 0x7f778c13e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 monclient: found mon.noname-b 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 -- 172.21.15.105:0/3585419845 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c0a0be0 con 0x7f778c137f80 2024-09-17T16:28:07.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77929b4640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f778400ce80 tx=0x7f778400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 client.?.objecter ms_handle_connect 0x7f778c137f80 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f778401a070 con 0x7f778c137f80 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: dump: 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778400dab0 con 0x7f778c137f80 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 5 monclient: authenticate success, global_id 4313 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _renew_subs 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c074390 con 0x7f778c137f80 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f778400dca0 con 0x7f778c137f80 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77909b0640 4 set_mon_vals no callback set 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 client.4313.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 monclient: dump: 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.676 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _renew_subs 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c0a1160 con 0x7f778c137f80 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 librados: init done 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778c137a40 con 0x7f778c137f80 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f778400d640 con 0x7f778c137f80 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 0x7f7770040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f778404fef0 con 0x7f778c137f80 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 10 client.4313.objecter ms_dispatch 0x7f778c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 3 client.4313.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 3 client.4313.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f7782ffd640 20 client.4313.objecter dump_active .. 0 homeless 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 0x7f7770040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.648+0000 7f77921b3640 10 monclient: get_auth_request con 0x7f777003e550 auth_method 0 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.652+0000 7f7782ffd640 10 client.4313.objecter ms_handle_connect 0x7f777003e550 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.652+0000 7f77921b3640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 0x7f7770040a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad000 tx=0x7f77880a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.652+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f778401db60 con 0x7f778c137f80 2024-09-17T16:28:07.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.652+0000 7f7782ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.652+0000 7f7782ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:07.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.808+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f778c0a4310 con 0x7f777003e550 2024-09-17T16:28:07.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.808+0000 7f7782ffd640 1 -- 172.21.15.105:0/3585419845 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34074 (secure 0 0 0) 0x7f778c0a4310 con 0x7f777003e550 2024-09-17T16:28:07.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:28:07.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 10 librados: watch_flush enter 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 10 librados: watch_flush exit 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 20 client.4313.objecter shutdown clearing up homeless session... 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 10 client.4313.objecter successfully canceled tick 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 msgr2=0x7f7770040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 0x7f7770040a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad000 tx=0x7f77880a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 10 monclient: shutdown 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 msgr2=0x7f778c13e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f778400ce80 tx=0x7f778400a710 comp rx=0 tx=0).stop 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 shutdown_connections 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f777003e550 0x7f7770040a10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f778c137f80 0x7f778c13e630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f778c137660 0x7f778c13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 --2- 172.21.15.105:0/3585419845 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f778c0a1d50 0x7f778c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 >> 172.21.15.105:0/3585419845 conn(0x7f778c0a4bf0 msgr2=0x7f778c09b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 shutdown_connections 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 -- 172.21.15.105:0/3585419845 wait complete. 2024-09-17T16:28:07.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.812+0000 7f7794417640 1 librados: shutdown 2024-09-17T16:28:07.834 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T16:28:07.788912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164368,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066672,"statfs":{"total":579820584960,"available":579652272128,"internally_reserved":0,"allocated":2277376,"data_stored":1543210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001743"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452937+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:28:03.452937+0000","last_peered":"2024-09-17T16:28:03.452937+0000","last_clean":"2024-09-17T16:28:03.452937+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:28:03.452937+0000","last_undegraded":"2024-09-17T16:28:03.452937+0000","last_fullsized":"2024-09-17T16:28:03.452937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462830+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:28:03.462830+0000","last_peered":"2024-09-17T16:28:03.462830+0000","last_clean":"2024-09-17T16:28:03.462830+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:28:03.462830+0000","last_undegraded":"2024-09-17T16:28:03.462830+0000","last_fullsized":"2024-09-17T16:28:03.462830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.451997+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:28:03.451997+0000","last_peered":"2024-09-17T16:28:03.451997+0000","last_clean":"2024-09-17T16:28:03.451997+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:28:03.451997+0000","last_undegraded":"2024-09-17T16:28:03.451997+0000","last_fullsized":"2024-09-17T16:28:03.451997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462976+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:28:03.462976+0000","last_peered":"2024-09-17T16:28:03.462976+0000","last_clean":"2024-09-17T16:28:03.462976+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:28:03.462976+0000","last_undegraded":"2024-09-17T16:28:03.462976+0000","last_fullsized":"2024-09-17T16:28:03.462976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.461846+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:28:03.461846+0000","last_peered":"2024-09-17T16:28:03.461846+0000","last_clean":"2024-09-17T16:28:03.461846+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:28:03.461846+0000","last_undegraded":"2024-09-17T16:28:03.461846+0000","last_fullsized":"2024-09-17T16:28:03.461846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.446906+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:28:03.446906+0000","last_peered":"2024-09-17T16:28:03.446906+0000","last_clean":"2024-09-17T16:28:03.446906+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:28:03.446906+0000","last_undegraded":"2024-09-17T16:28:03.446906+0000","last_fullsized":"2024-09-17T16:28:03.446906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452726+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:28:03.452726+0000","last_peered":"2024-09-17T16:28:03.452726+0000","last_clean":"2024-09-17T16:28:03.452726+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:28:03.452726+0000","last_undegraded":"2024-09-17T16:28:03.452726+0000","last_fullsized":"2024-09-17T16:28:03.452726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.456843+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:28:03.456843+0000","last_peered":"2024-09-17T16:28:03.456843+0000","last_clean":"2024-09-17T16:28:03.456843+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:28:03.456843+0000","last_undegraded":"2024-09-17T16:28:03.456843+0000","last_fullsized":"2024-09-17T16:28:03.456843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.450590+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:28:03.450590+0000","last_peered":"2024-09-17T16:28:03.450590+0000","last_clean":"2024-09-17T16:28:03.450590+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:28:03.450590+0000","last_undegraded":"2024-09-17T16:28:03.450590+0000","last_fullsized":"2024-09-17T16:28:03.450590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:28:07.835 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T16:28:07.835 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T16:28:07.836 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T16:28:07.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: get_monmap_and_config 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: monmap: 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:07.937841+0000 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:07.937841+0000 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:07.942 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 Processor -- start 2024-09-17T16:28:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 -- start start 2024-09-17T16:28:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: init 2024-09-17T16:28:07.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient: _add_conns ranks=[1,0,6,2,4,7,5,3,9,8,10] 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f77000a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f77000a4ab0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 0x7f7700142160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-d con 0x7f77000a7520 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000a4080 con 0x7f77000a4ab0 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000a37e0 con 0x7f77000a6a30 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f77000a7c40 con 0x7f77000a7520 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f77080bb640 10 monclient(hunting): authenticate will time out at 971.203557s 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 0x7f7700142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7705656640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f77000a4ab0 0x7f770013ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44676/0 (socket says 172.21.15.105:44676) 2024-09-17T16:28:07.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7705656640 1 -- 172.21.15.105:0/815996176 learned_addr learned my addr 172.21.15.105:0/815996176 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): get_auth_request con 0x7f77000a7520 auth_method 0 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.932+0000 7f7704e55640 1 -- 172.21.15.105:0/815996176 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 546638807 0 0) 0x7f77000a4080 con 0x7f77000a4ab0 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient(hunting): _negotiate 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient(hunting): my global_id is 4410 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 1 -- 172.21.15.105:0/815996176 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76ec003530 con 0x7f77000a4ab0 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 10 monclient(hunting): handle_auth_done global_id 4362 payload 274 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 1 -- 172.21.15.105:0/815996176 >> v1:172.21.15.160:6789/0 conn(0x7f77000a6a30 legacy=0x7f77000a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 1 -- 172.21.15.105:0/815996176 >> v1:172.21.15.105:6789/0 conn(0x7f77000a4ab0 legacy=0x7f770013ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 1 monclient: found mon.noname-d 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 1 -- 172.21.15.105:0/815996176 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77000a3ac0 con 0x7f77000a7520 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7706658640 1 --2- 172.21.15.105:0/815996176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 0x7f7700142160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0097f0 tx=0x7f76fc01c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a0c8816b94321db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 1 -- 172.21.15.105:0/815996176 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76fc0b0070 con 0x7f77000a7520 2024-09-17T16:28:07.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: dump: 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 1 -- 172.21.15.105:0/815996176 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc0aa810 con 0x7f77000a7520 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 5 monclient: authenticate success, global_id 4362 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 1 -- 172.21.15.105:0/815996176 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76fc0a9300 con 0x7f77000a7520 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 4 set_mon_vals no callback set 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f7704e55640 10 monclient: dump: 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 10 monclient: shutdown 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- 172.21.15.105:0/815996176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 msgr2=0x7f7700142160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 --2- 172.21.15.105:0/815996176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 0x7f7700142160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0097f0 tx=0x7f76fc01c3a0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- 172.21.15.105:0/815996176 shutdown_connections 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 --2- 172.21.15.105:0/815996176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f77000a7520 0x7f7700142160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- 172.21.15.105:0/815996176 >> 172.21.15.105:0/815996176 conn(0x7f77000a8d00 msgr2=0x7f77000a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- 172.21.15.105:0/815996176 shutdown_connections 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- 172.21.15.105:0/815996176 wait complete. 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 10 monclient: build_initial_monmap 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 10 monclient: monmap: 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 librados: starting msgr at 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 librados: starting objecter 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 Processor -- start 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 -- start start 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 librados: setting wanted keys 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 1 librados: calling monclient init 2024-09-17T16:28:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.936+0000 7f77080bb640 10 monclient: init 2024-09-17T16:28:07.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:07.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _add_conns ranks=[5,2,4,1,6,3,7,8,0] 2024-09-17T16:28:07.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f77000a4ab0 0x7f7700126190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-h con 0x7f77000a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-b con 0x7f77000a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f77000a7520 0x7f7700126c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): picked mon.noname-c con 0x7f77000a7520 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7700145040 con 0x7f77000a6a30 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7700144ec0 con 0x7f77000a7520 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f77001451c0 con 0x7f77000a4ab0 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient(hunting): authenticate will time out at 971.208235s 2024-09-17T16:28:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:46982/0 (socket says 172.21.15.105:46982) 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 -- 172.21.15.105:0/3647218438 learned_addr learned my addr 172.21.15.105:0/3647218438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f77000a7520 0x7f7700126c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): get_auth_request con 0x7f77000a6a30 auth_method 0 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): get_auth_request con 0x7f77000a7520 auth_method 0 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7706658640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient(hunting): handle_auth_done global_id 4322 payload 824 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705e57640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f77000a4ab0 0x7f7700126190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 -- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f77000a4ab0 msgr2=0x7f7700126190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f77000a4ab0 0x7f7700126190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 -- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f77000a7520 msgr2=0x7f7700126c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f77000a7520 0x7f7700126c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient: _finish_hunting 0 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 monclient: found mon.noname-b 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 -- 172.21.15.105:0/3647218438 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7700129c80 con 0x7f77000a6a30 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705656640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f76f0001ec0 tx=0x7f76f000c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 client.?.objecter ms_handle_connect 0x7f77000a6a30 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76f0017070 con 0x7f77000a6a30 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: dump: 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f0007070 con 0x7f77000a6a30 2024-09-17T16:28:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 5 monclient: authenticate success, global_id 4322 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _renew_subs 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7700129f60 con 0x7f77000a6a30 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76f0007710 con 0x7f77000a6a30 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f7fff640 4 set_mon_vals no callback set 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 client.4322.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 monclient: dump: 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:07.967 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _renew_subs 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7700142060 con 0x7f77000a6a30 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 librados: init done 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77000a6e10 con 0x7f77000a6a30 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f76f00072a0 con 0x7f77000a6a30 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 0x7f76e4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f76f004e960 con 0x7f77000a6a30 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 client.4322.objecter ms_dispatch 0x7f7700000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:07.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 3 client.4322.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 3 client.4322.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 20 client.4322.objecter dump_active .. 0 homeless 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705e57640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 0x7f76e4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705e57640 10 monclient: get_auth_request con 0x7f76e403e550 auth_method 0 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f7705e57640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 0x7f76e4040a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004450 tx=0x7f76f8009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.940+0000 7f76f67fc640 10 client.4322.objecter ms_handle_connect 0x7f76e403e550 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.944+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76f0019d00 con 0x7f77000a6a30 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.944+0000 7f76f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:07.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:07.944+0000 7f76f67fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:08.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f77080bb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f77080bb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f77080bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f77001290f0 con 0x7f77000a6a30 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f76f67fc640 1 -- 172.21.15.105:0/3647218438 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f76f0005cd0 con 0x7f77000a6a30 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f76f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-17T16:28:08.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.144+0000 7f76f67fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:08.153 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:08.153 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T16:28:08.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 10 librados: watch_flush enter 2024-09-17T16:28:08.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 10 librados: watch_flush exit 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 20 client.4322.objecter shutdown clearing up homeless session... 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 10 client.4322.objecter successfully canceled tick 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 msgr2=0x7f76e4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 0x7f76e4040a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f76f8004450 tx=0x7f76f8009040 comp rx=0 tx=0).stop 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 10 monclient: shutdown 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 msgr2=0x7f77001266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f76f0001ec0 tx=0x7f76f000c960 comp rx=0 tx=0).stop 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 shutdown_connections 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76e403e550 0x7f76e4040a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f77000a7520 0x7f7700126c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f77000a6a30 0x7f77001266d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 --2- 172.21.15.105:0/3647218438 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f77000a4ab0 0x7f7700126190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 >> 172.21.15.105:0/3647218438 conn(0x7f77000a8d00 msgr2=0x7f77000aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 shutdown_connections 2024-09-17T16:28:08.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 -- 172.21.15.105:0/3647218438 wait complete. 2024-09-17T16:28:08.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.148+0000 7f77080bb640 1 librados: shutdown 2024-09-17T16:28:08.169 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T16:28:08.170 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-17T16:28:08.183 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-17T16:28:08.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.280+0000 7f53eda6d640 10 monclient: get_monmap_and_config 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.280+0000 7f53eda6d640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.280+0000 7f53eda6d640 10 monclient: monmap: 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:08.287129+0000 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:08.287129+0000 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 Processor -- start 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 -- start start 2024-09-17T16:28:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient: init 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient: _add_conns ranks=[2,9,3,7,4,5,6,0,10,1,8] 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-b con 0x7f53e80a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 0x7f53e80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-a con 0x7f53e80a0360 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e81379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-g con 0x7f53e8137510 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f53e80a6700 con 0x7f53e80a0360 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f53e80a55b0 con 0x7f53e80a4750 2024-09-17T16:28:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f53e806c810 con 0x7f53e8137510 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient(hunting): authenticate will time out at 971.552975s 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 0x7f53e80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:46986/0 (socket says 172.21.15.105:46986) 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 -- 172.21.15.105:0/60726014 learned_addr learned my addr 172.21.15.105:0/60726014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): get_auth_request con 0x7f53e80a4750 auth_method 0 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): get_auth_request con 0x7f53e80a0360 auth_method 0 2024-09-17T16:28:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient(hunting): handle_auth_done global_id 4331 payload 274 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 -- 172.21.15.105:0/60726014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 msgr2=0x7f53e81379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 --2- 172.21.15.105:0/60726014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e81379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 -- 172.21.15.105:0/60726014 >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 msgr2=0x7f53e80a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 --2- 172.21.15.105:0/60726014 >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 0x7f53e80a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 monclient: found mon.noname-b 2024-09-17T16:28:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 -- 172.21.15.105:0/60726014 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e80a6240 con 0x7f53e80a4750 2024-09-17T16:28:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e6ffd640 1 --2- 172.21.15.105:0/60726014 >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 0x7f53e80a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e77fe640 1 --2- 172.21.15.105:0/60726014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009870 tx=0x7f53d801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4522a8e2bc4351 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 1 -- 172.21.15.105:0/60726014 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53d80b0070 con 0x7f53e80a4750 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: dump: 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 1 -- 172.21.15.105:0/60726014 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d80aa810 con 0x7f53e80a4750 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 1 -- 172.21.15.105:0/60726014 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53d80a9300 con 0x7f53e80a4750 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 5 monclient: authenticate success, global_id 4331 2024-09-17T16:28:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53eda6d640 4 set_mon_vals no callback set 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.284+0000 7f53e67fc640 10 monclient: dump: 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: shutdown 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/60726014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 msgr2=0x7f53e80a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- 172.21.15.105:0/60726014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009870 tx=0x7f53d801c420 comp rx=0 tx=0).stop 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/60726014 shutdown_connections 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- 172.21.15.105:0/60726014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e81379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- 172.21.15.105:0/60726014 >> v2:172.21.15.105:3300/0 conn(0x7f53e80a0360 0x7f53e80a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- 172.21.15.105:0/60726014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53e80a4750 0x7f53e80a4b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/60726014 >> 172.21.15.105:0/60726014 conn(0x7f53e80a93f0 msgr2=0x7f53e80a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/60726014 shutdown_connections 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/60726014 wait complete. 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: monmap: 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 librados: starting msgr at 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 librados: starting objecter 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 Processor -- start 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- start start 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 librados: setting wanted keys 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 librados: calling monclient init 2024-09-17T16:28:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: init 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _add_conns ranks=[0,6,3,5,2,8,4,7,1] 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-a con 0x7f53e80a0360 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-d con 0x7f53e80a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e813c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): picked mon.noname-g con 0x7f53e8137510 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8141380 con 0x7f53e80a0360 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f53e8141200 con 0x7f53e8137510 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f53e8141500 con 0x7f53e80a4750 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient(hunting): authenticate will time out at 971.557859s 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53174/0 (socket says 172.21.15.105:53174) 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57420/0 (socket says 172.21.15.105:57420) 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 1 -- 172.21.15.105:0/1853019527 learned_addr learned my addr 172.21.15.105:0/1853019527 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): get_auth_request con 0x7f53e80a0360 auth_method 0 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): get_auth_request con 0x7f53e80a4750 auth_method 0 2024-09-17T16:28:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient(hunting): handle_auth_done global_id 4371 payload 824 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e7fff640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e813c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 -- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 msgr2=0x7f53e813c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e813c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 -- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 msgr2=0x7f53e8135c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 monclient: found mon.noname-d 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 -- 172.21.15.105:0/1853019527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81239f0 con 0x7f53e80a4750 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e77fe640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e7fff640 10 monclient: get_auth_request con 0x7f53e8137510 auth_method 0 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e7fff640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e813c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e6ffd640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e980 tx=0x7f53dc00ee40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 client.?.objecter ms_handle_connect 0x7f53e80a4750 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53dc017070 con 0x7f53e80a4750 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: dump: 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc013470 con 0x7f53e80a4750 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 5 monclient: authenticate success, global_id 4371 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _renew_subs 2024-09-17T16:28:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e8123cd0 con 0x7f53e80a4750 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53dc013a90 con 0x7f53e80a4750 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 monclient: dump: 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.316 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53e5ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 client.4371.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _renew_subs 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e80a89a0 con 0x7f53e80a4750 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 librados: init done 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e809e670 con 0x7f53e80a4750 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f53dc01a070 con 0x7f53e80a4750 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 0x7f53d4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f53dc00f040 con 0x7f53e80a4750 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 10 client.4371.objecter ms_dispatch 0x7f53e8000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 3 client.4371.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 3 client.4371.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.288+0000 7f53cffff640 20 client.4371.objecter dump_active .. 0 homeless 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.292+0000 7f53e77fe640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 0x7f53d4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.292+0000 7f53e77fe640 10 monclient: get_auth_request con 0x7f53d403e550 auth_method 0 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.292+0000 7f53cffff640 10 client.4371.objecter ms_handle_connect 0x7f53d403e550 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.292+0000 7f53e77fe640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 0x7f53d4040a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d801ccc0 tx=0x7f53d80a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.296+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53dc011e10 con 0x7f53e80a4750 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.296+0000 7f53cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.296+0000 7f53cffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:08.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53eda6d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:08.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53eda6d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:08.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53eda6d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53e806e230 con 0x7f53e80a4750 2024-09-17T16:28:08.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53cffff640 1 -- 172.21.15.105:0/1853019527 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+12382 (secure 0 0 0) 0x7f53dc011b50 con 0x7f53e80a4750 2024-09-17T16:28:08.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:28:08.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.452+0000 7f53cffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:08.461 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:08.462 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:28:07.354603+0000","last_up_change":"2024-09-17T16:27:52.867595+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T16:27:53.825252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T16:27:59.687166+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6817","nonce":315040748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6819","nonce":315040748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6823","nonce":315040748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6821","nonce":315040748}]},"public_addr":"172.21.15.105:6817/315040748","cluster_addr":"172.21.15.105:6819/315040748","heartbeat_back_addr":"172.21.15.105:6823/315040748","heartbeat_front_addr":"172.21.15.105:6821/315040748","state":["exists","up"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6809","nonce":3218035673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6811","nonce":3218035673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6815","nonce":3218035673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6813","nonce":3218035673}]},"public_addr":"172.21.15.160:6809/3218035673","cluster_addr":"172.21.15.160:6811/3218035673","heartbeat_back_addr":"172.21.15.160:6815/3218035673","heartbeat_front_addr":"172.21.15.160:6813/3218035673","state":["exists","up"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6817","nonce":639753971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6819","nonce":639753971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6823","nonce":639753971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6821","nonce":639753971}]},"public_addr":"172.21.15.160:6817/639753971","cluster_addr":"172.21.15.160:6819/639753971","heartbeat_back_addr":"172.21.15.160:6823/639753971","heartbeat_front_addr":"172.21.15.160:6821/639753971","state":["exists","up"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.073377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.536863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.709983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.944027+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.614775+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.703905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 10 librados: watch_flush enter 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 10 librados: watch_flush exit 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 20 client.4371.objecter shutdown clearing up homeless session... 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 10 client.4371.objecter successfully canceled tick 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 msgr2=0x7f53d4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 0x7f53d4040a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53d801ccc0 tx=0x7f53d80a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 10 monclient: shutdown 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 msgr2=0x7f53e8136190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e980 tx=0x7f53dc00ee40 comp rx=0 tx=0).stop 2024-09-17T16:28:08.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 shutdown_connections 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53d403e550 0x7f53d4040a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53e8137510 0x7f53e813c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53e80a4750 0x7f53e8136190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 --2- 172.21.15.105:0/1853019527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53e80a0360 0x7f53e8135c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 >> 172.21.15.105:0/1853019527 conn(0x7f53e80a93f0 msgr2=0x7f53e80a1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 shutdown_connections 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 -- 172.21.15.105:0/1853019527 wait complete. 2024-09-17T16:28:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.456+0000 7f53eda6d640 1 librados: shutdown 2024-09-17T16:28:08.481 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T16:27:53.825252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T16:27:59.687166+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T16:28:08.482 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T16:28:08.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: get_monmap_and_config 2024-09-17T16:28:08.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: monmap: 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:08.596129+0000 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:08.596129+0000 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.599 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:08.600 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:08.600 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 Processor -- start 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 -- start start 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: init 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient: _add_conns ranks=[8,3,5,9,4,7,1,0,6,10,2] 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-e con 0x7fcbc00a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 0x7fcbc00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-g con 0x7fcbc009eec0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:08.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc0139530 0x7fcbc0139950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-h con 0x7fcbc0139530 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00a55b0 con 0x7fcbc009eec0 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fcbc009df90 con 0x7fcbc0139530 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fcbc009dcb0 con 0x7fcbc00a4750 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc673a640 10 monclient(hunting): authenticate will time out at 971.861832s 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34776/0 (socket says 172.21.15.105:34776) 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 -- 172.21.15.105:0/1889701538 learned_addr learned my addr 172.21.15.105:0/1889701538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): get_auth_request con 0x7fcbc00a4750 auth_method 0 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient(hunting): handle_auth_done global_id 4391 payload 274 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbf7fe640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 0x7fcbc00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbc4cd7640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc0139530 0x7fcbc0139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 -- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc0139530 msgr2=0x7fcbc0139950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc0139530 0x7fcbc0139950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 -- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 msgr2=0x7fcbc00a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 0x7fcbc00a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 monclient: found mon.noname-e 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbffff640 1 -- 172.21.15.105:0/1889701538 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc009e270 con 0x7fcbc00a4750 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbf7fe640 10 monclient: get_auth_request con 0x7fcbc009eec0 auth_method 0 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.592+0000 7fcbbf7fe640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 0x7fcbc00a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbffff640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098a0 tx=0x7fcbb001c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e2fc44ccb4244394 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 1 -- 172.21.15.105:0/1889701538 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcbb00b0070 con 0x7fcbc00a4750 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: dump: 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.606 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 5 monclient: authenticate success, global_id 4391 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 1 -- 172.21.15.105:0/1889701538 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb00aa810 con 0x7fcbc00a4750 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 4 set_mon_vals no callback set 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 1 -- 172.21.15.105:0/1889701538 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcbb00a9300 con 0x7fcbc00a4750 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbeffd640 10 monclient: dump: 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.609 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: shutdown 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/1889701538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 msgr2=0x7fcbc00a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098a0 tx=0x7fcbb001c440 comp rx=0 tx=0).stop 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/1889701538 shutdown_connections 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc0139530 0x7fcbc0139950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc009eec0 0x7fcbc00a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- 172.21.15.105:0/1889701538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fcbc00a4750 0x7fcbc00a4b30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/1889701538 >> 172.21.15.105:0/1889701538 conn(0x7fcbc00a5840 msgr2=0x7fcbc00a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/1889701538 shutdown_connections 2024-09-17T16:28:08.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/1889701538 wait complete. 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: monmap: 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.615 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 librados: starting msgr at 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 librados: starting objecter 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 Processor -- start 2024-09-17T16:28:08.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- start start 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 librados: setting wanted keys 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 librados: calling monclient init 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: init 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _add_conns ranks=[5,6,0,2,4,8,3,7,1] 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 0x7fcbc01224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-h con 0x7fcbc009eec0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-d con 0x7fcbc00a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): picked mon.noname-a con 0x7fcbc0139530 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01431b0 con 0x7fcbc0139530 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fcbc0143330 con 0x7fcbc009eec0 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fcbc01434b0 con 0x7fcbc00a4750 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient(hunting): authenticate will time out at 971.866925s 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57434/0 (socket says 172.21.15.105:57434) 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53184/0 (socket says 172.21.15.105:53184) 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 -- 172.21.15.105:0/2065297691 learned_addr learned my addr 172.21.15.105:0/2065297691 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): get_auth_request con 0x7fcbc00a4750 auth_method 0 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): get_auth_request con 0x7fcbc0139530 auth_method 0 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbffff640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 0x7fcbc01224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient(hunting): handle_auth_done global_id 4380 payload 824 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 -- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 msgr2=0x7fcbc01224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 0x7fcbc01224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 -- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 msgr2=0x7fcbc011fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 -- 172.21.15.105:0/2065297691 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc00a7c50 con 0x7fcbc00a4750 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbffff640 10 monclient: get_auth_request con 0x7fcbc009eec0 auth_method 0 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbffff640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 0x7fcbc01224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc4cd7640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbf7fe640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c7c0 tx=0x7fcbb400cc80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 client.?.objecter ms_handle_connect 0x7fcbc00a4750 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcbb4015070 con 0x7fcbc00a4750 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: dump: 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.621 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 5 monclient: authenticate success, global_id 4380 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb4007ca0 con 0x7fcbc00a4750 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _renew_subs 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01236f0 con 0x7fcbc00a4750 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcbb4011860 con 0x7fcbc00a4750 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbcff9640 10 monclient: dump: 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.623 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 client.4380.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbbe7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _renew_subs 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01c8340 con 0x7fcbc00a4750 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.596+0000 7fcbc673a640 1 librados: init done 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbc673a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbc673a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbc673a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc00a8180 con 0x7fcbc00a4750 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fcbb4018070 con 0x7fcbc00a4750 2024-09-17T16:28:08.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 0x7fcbac040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fcbb404f870 con 0x7fcbc00a4750 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 10 client.4380.objecter ms_dispatch 0x7fcbc0000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 3 client.4380.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 3 client.4380.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 20 client.4380.objecter dump_active .. 0 homeless 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbffff640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 0x7fcbac040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbffff640 10 monclient: get_auth_request con 0x7fcbac03e550 auth_method 0 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbcff9640 10 client.4380.objecter ms_handle_connect 0x7fcbac03e550 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.600+0000 7fcbbffff640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 0x7fcbac040a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00ad000 tx=0x7fcbb00a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.604+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcbc00a7c50 con 0x7fcbc00a4750 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.604+0000 7fcbbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.604+0000 7fcbbcff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:08.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbc673a640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbc673a640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbc673a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbc0121950 con 0x7fcbc00a4750 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbbcff9640 1 -- 172.21.15.105:0/2065297691 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcbb4010350 con 0x7fcbc00a4750 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbbcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-17T16:28:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.760+0000 7fcbbcff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:08.770 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-09-17T16:28:08.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 10 librados: watch_flush enter 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 10 librados: watch_flush exit 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 20 client.4380.objecter shutdown clearing up homeless session... 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 10 client.4380.objecter successfully canceled tick 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 msgr2=0x7fcbac040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 0x7fcbac040a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00ad000 tx=0x7fcbb00a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 10 monclient: shutdown 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 msgr2=0x7fcbc0138310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c7c0 tx=0x7fcbb400cc80 comp rx=0 tx=0).stop 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 shutdown_connections 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcbac03e550 0x7fcbac040a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc0139530 0x7fcbc011fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcbc00a4750 0x7fcbc0138310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 --2- 172.21.15.105:0/2065297691 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcbc009eec0 0x7fcbc01224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 >> 172.21.15.105:0/2065297691 conn(0x7fcbc00a5840 msgr2=0x7fcbc00a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 shutdown_connections 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 -- 172.21.15.105:0/2065297691 wait complete. 2024-09-17T16:28:08.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.764+0000 7fcbc673a640 1 librados: shutdown 2024-09-17T16:28:08.787 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: get_monmap_and_config 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: monmap: 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:08.903815+0000 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:08.903815+0000 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:08.907 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 Processor -- start 2024-09-17T16:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 -- start start 2024-09-17T16:28:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: init 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient: _add_conns ranks=[1,8,4,0,10,5,6,7,2,3,9] 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8d000a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-e con 0x7f8d000a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-c con 0x7f8d000a7520 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d000a4080 con 0x7f8d000a6a30 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8d000a7c40 con 0x7f8d000a7520 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8d000a37e0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d063fc640 10 monclient(hunting): authenticate will time out at 972.169262s 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34786/0 (socket says 172.21.15.105:34786) 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46310/0 (socket says 172.21.15.105:46310) 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 -- 172.21.15.105:0/4100118300 learned_addr learned my addr 172.21.15.105:0/4100118300 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): get_auth_request con 0x7f8d000a4ab0 auth_method 0 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): get_auth_request con 0x7f8d000a7520 auth_method 0 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient(hunting): handle_auth_done global_id 4400 payload 274 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 -- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 msgr2=0x7f8d00140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 -- 172.21.15.105:0/4100118300 >> v1:172.21.15.160:6789/0 conn(0x7f8d000a6a30 legacy=0x7f8d000a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 monclient: found mon.noname-e 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 -- 172.21.15.105:0/4100118300 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d000a3ac0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8d04999640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.900+0000 7f8cff7fe640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40098a0 tx=0x7f8cf401c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2839da8b3b8ef9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 1 -- 172.21.15.105:0/4100118300 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf40b2070 con 0x7f8d000a4ab0 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: dump: 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 5 monclient: authenticate success, global_id 4400 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 1 -- 172.21.15.105:0/4100118300 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf40ab7f0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 4 set_mon_vals no callback set 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 1 -- 172.21.15.105:0/4100118300 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf40abe50 con 0x7f8d000a4ab0 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfeffd640 10 monclient: dump: 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.917 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: shutdown 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 msgr2=0x7f8d0013ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40098a0 tx=0x7f8cf401c400 comp rx=0 tx=0).stop 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/4100118300 shutdown_connections 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8d000a7520 0x7f8d00140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- 172.21.15.105:0/4100118300 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8d000a4ab0 0x7f8d0013ea30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/4100118300 >> 172.21.15.105:0/4100118300 conn(0x7f8d000a8d00 msgr2=0x7f8d000a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/4100118300 shutdown_connections 2024-09-17T16:28:08.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/4100118300 wait complete. 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: build_initial_monmap 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: monmap: 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:08.922 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 librados: starting msgr at 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 librados: starting objecter 2024-09-17T16:28:08.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 Processor -- start 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- start start 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 librados: setting wanted keys 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 librados: calling monclient init 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: init 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _add_conns ranks=[6,5,1,4,2,3,7,0,8] 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-d con 0x7f8d000a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8d000a6a30 0x7f8d00126840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-h con 0x7f8d000a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d000a7520 0x7f8d00126d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): picked mon.noname-f con 0x7f8d000a7520 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:08.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00143e60 con 0x7f8d000a7520 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8d00143ce0 con 0x7f8d000a6a30 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8d00143fe0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient(hunting): authenticate will time out at 972.174087s 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57436/0 (socket says 172.21.15.105:57436) 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 -- 172.21.15.105:0/3431767474 learned_addr learned my addr 172.21.15.105:0/3431767474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): get_auth_request con 0x7f8d000a4ab0 auth_method 0 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:08.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient(hunting): handle_auth_done global_id 4389 payload 824 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 -- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8d000a6a30 msgr2=0x7f8d00126840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8d000a6a30 0x7f8d00126840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 -- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d000a7520 msgr2=0x7f8d00126d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d04999640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d000a7520 0x7f8d00126d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d000a7520 0x7f8d00126d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 monclient: found mon.noname-d 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:08.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 -- 172.21.15.105:0/3431767474 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0012a120 con 0x7f8d000a4ab0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfffff640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000c480 tx=0x7f8cf000c940 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 client.?.objecter ms_handle_connect 0x7f8d000a4ab0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf0012aa0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: dump: 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:08.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0017070 con 0x7f8d000a4ab0 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 5 monclient: authenticate success, global_id 4389 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _renew_subs 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d00140bb0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8cf0012c80 con 0x7f8d000a4ab0 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 client.4389.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfe7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 10 monclient: dump: 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:08.929 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:08.930 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _renew_subs 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001292b0 con 0x7f8d000a4ab0 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 librados: init done 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d000a6e10 con 0x7f8d000a4ab0 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.904+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8cf0012630 con 0x7f8d000a4ab0 2024-09-17T16:28:08.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 0x7f8cec040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f8cf004f570 con 0x7f8d000a4ab0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 10 client.4389.objecter ms_dispatch 0x7f8d00000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 3 client.4389.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 3 client.4389.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 20 client.4389.objecter dump_active .. 0 homeless 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cff7fe640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 0x7f8cec040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cff7fe640 10 monclient: get_auth_request con 0x7f8cec03e550 auth_method 0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cfcff9640 10 client.4389.objecter ms_handle_connect 0x7f8cec03e550 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.908+0000 7f8cff7fe640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 0x7f8cec040a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40af000 tx=0x7f8cf4004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.912+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8cf001a360 con 0x7f8d000a4ab0 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.912+0000 7f8cfcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:08.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:08.912+0000 7f8cfcff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7f8d00074850 con 0x7f8d000a4ab0 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8cfcff9640 1 -- 172.21.15.105:0/3431767474 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) ==== 92+0+10 (secure 0 0 0) 0x7f8cf001a540 con 0x7f8d000a4ab0 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8cfcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-17T16:28:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8cfcff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:09.088 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 8 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 librados: watch_flush enter 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 librados: watch_flush exit 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 20 client.4389.objecter shutdown clearing up homeless session... 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 client.4389.objecter successfully canceled tick 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 msgr2=0x7f8cec040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 0x7f8cec040a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40af000 tx=0x7f8cf4004740 comp rx=0 tx=0).stop 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 10 monclient: shutdown 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 msgr2=0x7f8d00126300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000c480 tx=0x7f8cf000c940 comp rx=0 tx=0).stop 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 shutdown_connections 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8cec03e550 0x7f8cec040a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d000a7520 0x7f8d00126d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8d000a6a30 0x7f8d00126840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.080+0000 7f8d063fc640 1 --2- 172.21.15.105:0/3431767474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8d000a4ab0 0x7f8d00126300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.084+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 >> 172.21.15.105:0/3431767474 conn(0x7f8d000a8d00 msgr2=0x7f8d0009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.084+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 shutdown_connections 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.084+0000 7f8d063fc640 1 -- 172.21.15.105:0/3431767474 wait complete. 2024-09-17T16:28:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.084+0000 7f8d063fc640 1 librados: shutdown 2024-09-17T16:28:09.107 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-17T16:28:09.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 10 monclient: get_monmap_and_config 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 10 monclient: monmap: 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:09.222665+0000 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:09.222665+0000 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:09.226 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 1 Processor -- start 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 1 -- start start 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.216+0000 7f4685445640 10 monclient: init 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient: _add_conns ranks=[4,2,0,5,9,3,1,7,10,8,6] 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46800a2620 0x7f46800a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): picked mon.noname-c con 0x7f46800a2620 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): picked mon.noname-b con 0x7f46800a2f40 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4680137940 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468006f2e0 con 0x7f4680137940 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f468006e580 con 0x7f46800a2f40 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f46800a15e0 con 0x7f46800a2620 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient(hunting): authenticate will time out at 972.488449s 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46800a2620 0x7f46800a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4680137940 0x7f468013e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44690/0 (socket says 172.21.15.105:44690) 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467f7fe640 1 -- 172.21.15.105:0/414744199 learned_addr learned my addr 172.21.15.105:0/414744199 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): get_auth_request con 0x7f46800a2620 auth_method 0 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:46996/0 (socket says 172.21.15.105:46996) 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): get_auth_request con 0x7f46800a2f40 auth_method 0 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient(hunting): handle_auth_done global_id 4340 payload 274 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 -- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46800a2620 msgr2=0x7f46800a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 1 -- 172.21.15.105:0/414744199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85991477 0 0) 0x7f468006f2e0 con 0x7f4680137940 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46800a2620 0x7f46800a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 -- 172.21.15.105:0/414744199 >> v1:172.21.15.105:6789/0 conn(0x7f4680137940 legacy=0x7f468013e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 monclient: found mon.noname-b 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 -- 172.21.15.105:0/414744199 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800a3c30 con 0x7f46800a2f40 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:28:09.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467e7fc640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46740097f0 tx=0x7f467401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc52751fe1c8adbc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 1 -- 172.21.15.105:0/414744199 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46740b0070 con 0x7f46800a2f40 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: dump: 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.233 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 1 -- 172.21.15.105:0/414744199 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46740aa810 con 0x7f46800a2f40 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 5 monclient: authenticate success, global_id 4340 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:09.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 4 set_mon_vals no callback set 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 1 -- 172.21.15.105:0/414744199 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46740a9300 con 0x7f46800a2f40 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f467dffb640 10 monclient: dump: 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.235 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.236 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.236 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.236 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.236 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.236 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 10 monclient: shutdown 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 msgr2=0x7f4680137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46740097f0 tx=0x7f467401c3a0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- 172.21.15.105:0/414744199 shutdown_connections 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f46800a2f40 0x7f4680137400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 --2- 172.21.15.105:0/414744199 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46800a2620 0x7f46800a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- 172.21.15.105:0/414744199 >> 172.21.15.105:0/414744199 conn(0x7f4680068ba0 msgr2=0x7f4680068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- 172.21.15.105:0/414744199 shutdown_connections 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.220+0000 7f4685445640 1 -- 172.21.15.105:0/414744199 wait complete. 2024-09-17T16:28:09.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: monmap: 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 librados: starting msgr at 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 librados: starting objecter 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 Processor -- start 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- start start 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 librados: setting wanted keys 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 librados: calling monclient init 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: init 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _add_conns ranks=[3,5,6,7,8,1,2,0,4] 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 0x7f468013e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): picked mon.noname-g con 0x7f46800a2620 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46800a2f40 0x7f468011e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): picked mon.noname-h con 0x7f46800a2f40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): picked mon.noname-d con 0x7f4680137940 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4680069ad0 con 0x7f46800a2620 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4680069950 con 0x7f46800a2f40 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f46800a1dd0 con 0x7f4680137940 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient(hunting): authenticate will time out at 972.493339s 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57440/0 (socket says 172.21.15.105:57440) 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 -- 172.21.15.105:0/2772501877 learned_addr learned my addr 172.21.15.105:0/2772501877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): get_auth_request con 0x7f4680137940 auth_method 0 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467effd640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 0x7f468013e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467e7fc640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46800a2f40 0x7f468011e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 -- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46800a2f40 msgr2=0x7f468011e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46800a2f40 0x7f468011e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 -- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 msgr2=0x7f468013e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 0x7f468013e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 -- 172.21.15.105:0/2772501877 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46801207e0 con 0x7f4680137940 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467effd640 10 monclient: get_auth_request con 0x7f46800a2620 auth_method 0 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467effd640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 0x7f468013e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467f7fe640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f467800c6b0 tx=0x7f467800cb70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 client.?.objecter ms_handle_connect 0x7f4680137940 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:09.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4678018070 con 0x7f4680137940 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: dump: 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.247 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467800a640 con 0x7f4680137940 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 5 monclient: authenticate success, global_id 4398 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f467800ace0 con 0x7f4680137940 2024-09-17T16:28:09.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f467d7fa640 4 set_mon_vals no callback set 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 monclient: dump: 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.249 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _renew_subs 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4680120ac0 con 0x7f4680137940 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _renew_subs 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f468013fd50 con 0x7f4680137940 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 librados: init done 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f467800a870 con 0x7f4680137940 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46800a2a00 con 0x7f4680137940 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 0x7f466c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f467804efd0 con 0x7f4680137940 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 10 client.4398.objecter ms_dispatch 0x7f4680000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 3 client.4398.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 3 client.4398.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.224+0000 7f46677fe640 20 client.4398.objecter dump_active .. 0 homeless 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.228+0000 7f467effd640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 0x7f466c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.228+0000 7f467effd640 10 monclient: get_auth_request con 0x7f466c03e550 auth_method 0 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.228+0000 7f46677fe640 10 client.4398.objecter ms_handle_connect 0x7f466c03e550 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.228+0000 7f467effd640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 0x7f466c040a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4670009ae0 tx=0x7f4670008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.232+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4678011360 con 0x7f4680137940 2024-09-17T16:28:09.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.232+0000 7f46677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.232+0000 7f46677fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:09.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.396+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f468011f3e0 con 0x7f466c03e550 2024-09-17T16:28:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.396+0000 7f46677fe640 1 -- 172.21.15.105:0/2772501877 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34074 (secure 0 0 0) 0x7f468011f3e0 con 0x7f466c03e550 2024-09-17T16:28:09.406 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:09.407 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:28:09.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 10 librados: watch_flush enter 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 10 librados: watch_flush exit 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 20 client.4398.objecter shutdown clearing up homeless session... 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 10 client.4398.objecter successfully canceled tick 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 msgr2=0x7f466c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 0x7f466c040a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4670009ae0 tx=0x7f4670008040 comp rx=0 tx=0).stop 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 10 monclient: shutdown 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 msgr2=0x7f468011e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f467800c6b0 tx=0x7f467800cb70 comp rx=0 tx=0).stop 2024-09-17T16:28:09.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 shutdown_connections 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f466c03e550 0x7f466c040a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4680137940 0x7f468011e7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46800a2f40 0x7f468011e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 --2- 172.21.15.105:0/2772501877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46800a2620 0x7f468013e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 >> 172.21.15.105:0/2772501877 conn(0x7f4680068ba0 msgr2=0x7f46800a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 shutdown_connections 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 -- 172.21.15.105:0/2772501877 wait complete. 2024-09-17T16:28:09.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.400+0000 7f4685445640 1 librados: shutdown 2024-09-17T16:28:09.425 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T16:28:07.788912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164368,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066672,"statfs":{"total":579820584960,"available":579652272128,"internally_reserved":0,"allocated":2277376,"data_stored":1543210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001743"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452937+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:28:03.452937+0000","last_peered":"2024-09-17T16:28:03.452937+0000","last_clean":"2024-09-17T16:28:03.452937+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:28:03.452937+0000","last_undegraded":"2024-09-17T16:28:03.452937+0000","last_fullsized":"2024-09-17T16:28:03.452937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462830+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:28:03.462830+0000","last_peered":"2024-09-17T16:28:03.462830+0000","last_clean":"2024-09-17T16:28:03.462830+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:28:03.462830+0000","last_undegraded":"2024-09-17T16:28:03.462830+0000","last_fullsized":"2024-09-17T16:28:03.462830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.451997+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:28:03.451997+0000","last_peered":"2024-09-17T16:28:03.451997+0000","last_clean":"2024-09-17T16:28:03.451997+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:28:03.451997+0000","last_undegraded":"2024-09-17T16:28:03.451997+0000","last_fullsized":"2024-09-17T16:28:03.451997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462976+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:28:03.462976+0000","last_peered":"2024-09-17T16:28:03.462976+0000","last_clean":"2024-09-17T16:28:03.462976+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:28:03.462976+0000","last_undegraded":"2024-09-17T16:28:03.462976+0000","last_fullsized":"2024-09-17T16:28:03.462976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.461846+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:28:03.461846+0000","last_peered":"2024-09-17T16:28:03.461846+0000","last_clean":"2024-09-17T16:28:03.461846+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:28:03.461846+0000","last_undegraded":"2024-09-17T16:28:03.461846+0000","last_fullsized":"2024-09-17T16:28:03.461846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.446906+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:28:03.446906+0000","last_peered":"2024-09-17T16:28:03.446906+0000","last_clean":"2024-09-17T16:28:03.446906+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:28:03.446906+0000","last_undegraded":"2024-09-17T16:28:03.446906+0000","last_fullsized":"2024-09-17T16:28:03.446906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452726+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:28:03.452726+0000","last_peered":"2024-09-17T16:28:03.452726+0000","last_clean":"2024-09-17T16:28:03.452726+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:28:03.452726+0000","last_undegraded":"2024-09-17T16:28:03.452726+0000","last_fullsized":"2024-09-17T16:28:03.452726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.456843+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:28:03.456843+0000","last_peered":"2024-09-17T16:28:03.456843+0000","last_clean":"2024-09-17T16:28:03.456843+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:28:03.456843+0000","last_undegraded":"2024-09-17T16:28:03.456843+0000","last_fullsized":"2024-09-17T16:28:03.456843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.450590+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:28:03.450590+0000","last_peered":"2024-09-17T16:28:03.450590+0000","last_clean":"2024-09-17T16:28:03.450590+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:28:03.450590+0000","last_undegraded":"2024-09-17T16:28:03.450590+0000","last_fullsized":"2024-09-17T16:28:03.450590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:28:09.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:28:09.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: get_monmap_and_config 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: monmap: 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:09.534172+0000 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:09.534172+0000 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.537 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:09.538 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 Processor -- start 2024-09-17T16:28:09.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 -- start start 2024-09-17T16:28:09.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: init 2024-09-17T16:28:09.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient: _add_conns ranks=[10,7,0,4,5,8,9,6,2,3,1] 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee300a6a30 0x7fee300a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-f con 0x7fee300a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee300a4ab0 0x7fee300a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-i con 0x7fee300a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee301397f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee300a4080 con 0x7fee301397f0 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fee300a7c40 con 0x7fee300a6a30 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fee300a7890 con 0x7fee300a4ab0 2024-09-17T16:28:09.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee34b66640 10 monclient(hunting): authenticate will time out at 972.799817s 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee2ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fee301397f0 0x7fee30139ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44704/0 (socket says 172.21.15.105:44704) 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.528+0000 7fee2ed9d640 1 -- 172.21.15.105:0/2582782408 learned_addr learned my addr 172.21.15.105:0/2582782408 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee300a4ab0 0x7fee300a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 --2- 172.21.15.105:0/2582782408 >> v2:172.21.15.160:3300/0 conn(0x7fee300a6a30 0x7fee300a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823345949 0 0) 0x7fee300a4080 con 0x7fee301397f0 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): _negotiate 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): my global_id is 4455 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18003530 con 0x7fee301397f0 2024-09-17T16:28:09.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request con 0x7fee300a4ab0 auth_method 0 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): get_auth_request con 0x7fee300a6a30 auth_method 0 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4009665589 0 0) 0x7fee18003530 con 0x7fee301397f0 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee300a4ab0 msgr2=0x7fee300a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee300a4ab0 0x7fee300a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 >> v2:172.21.15.160:3300/0 conn(0x7fee300a6a30 msgr2=0x7fee300a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 --2- 172.21.15.105:0/2582782408 >> v2:172.21.15.160:3300/0 conn(0x7fee300a6a30 0x7fee300a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee300a37e0 con 0x7fee301397f0 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fee28001cf0 con 0x7fee301397f0 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: dump: 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.552 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 >> v1:172.21.15.105:6789/0 conn(0x7fee301397f0 legacy=0x7fee30139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: _add_conns ranks=[6,4,5,1,3,7,8,0,2] 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 0x7fee18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): picked mon.d con 0x7fee18004210 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fee18004da0 0x7fee18005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): picked mon.c con 0x7fee18004da0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fee18009ab0 0x7fee1800bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): picked mon.h con 0x7fee18009ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fee300a7c40 con 0x7fee18004da0 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fee300a7890 con 0x7fee18009ab0 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fee1800c850 con 0x7fee18004210 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 0x7fee18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): get_auth_request con 0x7fee18004210 auth_method 0 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fee18004da0 0x7fee18005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request con 0x7fee18004da0 auth_method 0 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient(hunting): handle_auth_done global_id 4455 payload 293 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2ed9d640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fee18009ab0 0x7fee1800bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 -- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fee18009ab0 msgr2=0x7fee1800bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fee18009ab0 0x7fee1800bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 -- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fee18004da0 msgr2=0x7fee18005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fee18004da0 0x7fee18005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 monclient: found mon.d 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 -- 172.21.15.105:0/2582782408 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee18003530 con 0x7fee18004210 2024-09-17T16:28:09.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee34b66640 5 monclient: authenticate success, global_id 4455 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee34b66640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2ed9d640 1 -- 172.21.15.105:0/2582782408 reap_dead start 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2dd9b640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fee18004da0 0x7fee18005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2e59c640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 0x7fee18004810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee24009ab0 tx=0x7fee2401ca00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c7fcad6d9bcc87c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee240b2070 con 0x7fee18004210 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: dump: 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.557 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.558 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee240a5a20 con 0x7fee18004210 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee34b66640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee34b66640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee34b66640 4 set_mon_vals no callback set 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 1 -- 172.21.15.105:0/2582782408 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee240096b0 con 0x7fee18004210 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.532+0000 7fee2d59a640 10 monclient: dump: 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.559 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: shutdown 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 msgr2=0x7fee18004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 0x7fee18004810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee24009ab0 tx=0x7fee2401ca00 comp rx=0 tx=0).stop 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/2582782408 shutdown_connections 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 --2- 172.21.15.105:0/2582782408 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fee18004210 0x7fee18004810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/2582782408 >> 172.21.15.105:0/2582782408 conn(0x7fee300a8d00 msgr2=0x7fee300a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/2582782408 shutdown_connections 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/2582782408 wait complete. 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: monmap: 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:09.561 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 librados: starting msgr at 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 librados: starting objecter 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 Processor -- start 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- start start 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 librados: setting wanted keys 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 librados: calling monclient init 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: init 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _add_conns ranks=[2,7,0,3,4,6,1,5,8] 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 0x7fee30138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-b con 0x7fee18004210 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee30124910 0x7fee30138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-i con 0x7fee30124910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): picked mon.noname-a con 0x7fee30122680 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006e740 con 0x7fee30122680 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fee3006b5b0 con 0x7fee18004210 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fee301261a0 con 0x7fee30124910 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient(hunting): authenticate will time out at 972.806067s 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 0x7fee30138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53188/0 (socket says 172.21.15.105:53188) 2024-09-17T16:28:09.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 -- 172.21.15.105:0/967538 learned_addr learned my addr 172.21.15.105:0/967538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): get_auth_request con 0x7fee30122680 auth_method 0 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): get_auth_request con 0x7fee18004210 auth_method 0 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2dd9b640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee30124910 0x7fee30138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request con 0x7fee30124910 auth_method 0 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient(hunting): handle_auth_done global_id 4464 payload 824 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 -- 172.21.15.105:0/967538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee30124910 msgr2=0x7fee30138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee30124910 0x7fee30138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 -- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 msgr2=0x7fee30138750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 0x7fee30138750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 monclient: found mon.noname-a 2024-09-17T16:28:09.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 -- 172.21.15.105:0/967538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee300a21e0 con 0x7fee30122680 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2e59c640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 0x7fee30138750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2ed9d640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee280012f0 tx=0x7fee2800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 client.?.objecter ms_handle_connect 0x7fee30122680 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee28004d80 con 0x7fee30122680 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: dump: 2024-09-17T16:28:09.567 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.568 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2800bd40 con 0x7fee30122680 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee28007580 con 0x7fee30122680 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 10 monclient: dump: 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.569 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee2cd99640 4 set_mon_vals no callback set 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 5 monclient: authenticate success, global_id 4464 2024-09-17T16:28:09.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _renew_subs 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee30125870 con 0x7fee30122680 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 client.4464.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _renew_subs 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee300aa6e0 con 0x7fee30122680 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 librados: init done 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee300a1cb0 con 0x7fee30122680 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fee2801b020 con 0x7fee30122680 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.536+0000 7fee1effd640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 0x7fee10040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fee2804f910 con 0x7fee30122680 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 10 client.4464.objecter ms_dispatch 0x7fee30000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:09.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 3 client.4464.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 3 client.4464.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 20 client.4464.objecter dump_active .. 0 homeless 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee2e59c640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 0x7fee10040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee2e59c640 10 monclient: get_auth_request con 0x7fee1003e550 auth_method 0 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee1effd640 10 client.4464.objecter ms_handle_connect 0x7fee1003e550 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.540+0000 7fee2e59c640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 0x7fee10040a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee240af000 tx=0x7fee24008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.544+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fee300a21e0 con 0x7fee30122680 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.544+0000 7fee1effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.544+0000 7fee1effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:09.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.712+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee30139c00 con 0x7fee1003e550 2024-09-17T16:28:09.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee1effd640 1 -- 172.21.15.105:0/967538 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34074 (secure 0 0 0) 0x7fee30139c00 con 0x7fee1003e550 2024-09-17T16:28:09.723 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:28:09.723 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:28:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 10 librados: watch_flush enter 2024-09-17T16:28:09.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 10 librados: watch_flush exit 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 20 client.4464.objecter shutdown clearing up homeless session... 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 10 client.4464.objecter successfully canceled tick 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 msgr2=0x7fee10040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 0x7fee10040a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee240af000 tx=0x7fee24008040 comp rx=0 tx=0).stop 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 10 monclient: shutdown 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 msgr2=0x7fee30122a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fee280012f0 tx=0x7fee2800b0d0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 shutdown_connections 2024-09-17T16:28:09.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fee1003e550 0x7fee10040a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee30122680 0x7fee30122a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fee30124910 0x7fee30138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 --2- 172.21.15.105:0/967538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fee18004210 0x7fee30138750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 >> 172.21.15.105:0/967538 conn(0x7fee300a8d00 msgr2=0x7fee300aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 shutdown_connections 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 -- 172.21.15.105:0/967538 wait complete. 2024-09-17T16:28:09.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.716+0000 7fee34b66640 1 librados: shutdown 2024-09-17T16:28:09.741 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T16:28:07.788912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164368,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066672,"statfs":{"total":579820584960,"available":579652272128,"internally_reserved":0,"allocated":2277376,"data_stored":1543210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001743"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452937+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:28:03.452937+0000","last_peered":"2024-09-17T16:28:03.452937+0000","last_clean":"2024-09-17T16:28:03.452937+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:28:03.452937+0000","last_undegraded":"2024-09-17T16:28:03.452937+0000","last_fullsized":"2024-09-17T16:28:03.452937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462830+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:28:03.462830+0000","last_peered":"2024-09-17T16:28:03.462830+0000","last_clean":"2024-09-17T16:28:03.462830+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:28:03.462830+0000","last_undegraded":"2024-09-17T16:28:03.462830+0000","last_fullsized":"2024-09-17T16:28:03.462830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.451997+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:28:03.451997+0000","last_peered":"2024-09-17T16:28:03.451997+0000","last_clean":"2024-09-17T16:28:03.451997+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:28:03.451997+0000","last_undegraded":"2024-09-17T16:28:03.451997+0000","last_fullsized":"2024-09-17T16:28:03.451997+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.462976+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:28:03.462976+0000","last_peered":"2024-09-17T16:28:03.462976+0000","last_clean":"2024-09-17T16:28:03.462976+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:28:03.462976+0000","last_undegraded":"2024-09-17T16:28:03.462976+0000","last_fullsized":"2024-09-17T16:28:03.462976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.461846+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:28:03.461846+0000","last_peered":"2024-09-17T16:28:03.461846+0000","last_clean":"2024-09-17T16:28:03.461846+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:28:03.461846+0000","last_undegraded":"2024-09-17T16:28:03.461846+0000","last_fullsized":"2024-09-17T16:28:03.461846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.446906+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:28:03.446906+0000","last_peered":"2024-09-17T16:28:03.446906+0000","last_clean":"2024-09-17T16:28:03.446906+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:28:03.446906+0000","last_undegraded":"2024-09-17T16:28:03.446906+0000","last_fullsized":"2024-09-17T16:28:03.446906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.452726+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:28:03.452726+0000","last_peered":"2024-09-17T16:28:03.452726+0000","last_clean":"2024-09-17T16:28:03.452726+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:28:03.452726+0000","last_undegraded":"2024-09-17T16:28:03.452726+0000","last_fullsized":"2024-09-17T16:28:03.452726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.456843+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:28:03.456843+0000","last_peered":"2024-09-17T16:28:03.456843+0000","last_clean":"2024-09-17T16:28:03.456843+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:28:03.456843+0000","last_undegraded":"2024-09-17T16:28:03.456843+0000","last_fullsized":"2024-09-17T16:28:03.456843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T16:28:03.450590+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:28:03.450590+0000","last_peered":"2024-09-17T16:28:03.450590+0000","last_clean":"2024-09-17T16:28:03.450590+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:28:03.450590+0000","last_undegraded":"2024-09-17T16:28:03.450590+0000","last_fullsized":"2024-09-17T16:28:03.450590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":650774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":60406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":60425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:28:09.742 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-17T16:28:09.742 DEBUG:tasks.mon_thrash:Yielding 2024-09-17T16:28:09.742 INFO:teuthology.run_tasks:Running task workunit... 2024-09-17T16:28:09.756 INFO:tasks.workunit:Pulling workunits from ref 1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:28:09.757 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-17T16:28:09.757 DEBUG:teuthology.orchestra.run.smithi160:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-17T16:28:09.758 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-17T16:28:09.758 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726590489, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: True, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-17T16:28:09.758 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:28:09.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:28:09.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:28:09.765 INFO:teuthology.orchestra.run.smithi160.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T16:28:09.765 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-17T16:28:09.811 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-17T16:28:09.811 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-17T16:28:09.859 INFO:tasks.workunit:timeout=3h 2024-09-17T16:28:09.859 INFO:tasks.workunit:cleanup=True 2024-09-17T16:28:09.860 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1889a165b5128787ec693450a3d1c5df5d6dc203 2024-09-17T16:28:09.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: get_monmap_and_config 2024-09-17T16:28:09.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: monmap: 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:09.866355+0000 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:09.866355+0000 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.869 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:09.870 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 Processor -- start 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 -- start start 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: init 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient: _add_conns ranks=[7,6,10,2,0,3,4,1,5,9,8] 2024-09-17T16:28:09.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc9300a68e0 0x7fc9300a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-i con 0x7fc9300a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-d con 0x7fc9300a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 0x7fc930139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-f con 0x7fc9301396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc9300a3fc0 con 0x7fc9301396a0 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc9300a63c0 con 0x7fc9300a4910 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc9300a7740 con 0x7fc9300a68e0 2024-09-17T16:28:09.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.860+0000 7fc937e03640 10 monclient(hunting): authenticate will time out at 973.131888s 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57448/0 (socket says 172.21.15.105:57448) 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 -- 172.21.15.105:0/3799233794 learned_addr learned my addr 172.21.15.105:0/3799233794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): get_auth_request con 0x7fc9300a4910 auth_method 0 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 -- 172.21.15.105:0/3799233794 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc9300a68e0 msgr2=0x7fc9300a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc935b9f640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc9300a68e0 0x7fc9300a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc9363a0640 1 --2- 172.21.15.105:0/3799233794 >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 0x7fc930139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc9300a68e0 0x7fc9300a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 -- 172.21.15.105:0/3799233794 >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 msgr2=0x7fc930139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 --2- 172.21.15.105:0/3799233794 >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 0x7fc930139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 monclient: found mon.noname-d 2024-09-17T16:28:09.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 -- 172.21.15.105:0/3799233794 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93006ebc0 con 0x7fc9300a4910 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc9363a0640 10 monclient: get_auth_request con 0x7fc9301396a0 auth_method 0 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc9363a0640 1 --2- 172.21.15.105:0/3799233794 >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 0x7fc930139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc93539e640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc924009870 tx=0x7fc92401c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c900130423c3e9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 1 -- 172.21.15.105:0/3799233794 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9240b0070 con 0x7fc9300a4910 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: dump: 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.884 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.885 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.885 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.886 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 1 -- 172.21.15.105:0/3799233794 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9240aa810 con 0x7fc9300a4910 2024-09-17T16:28:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 5 monclient: authenticate success, global_id 4407 2024-09-17T16:28:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 1 -- 172.21.15.105:0/3799233794 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9240a9300 con 0x7fc9300a4910 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 4 set_mon_vals no callback set 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc934b9d640 10 monclient: dump: 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.890 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 10 monclient: shutdown 2024-09-17T16:28:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 -- 172.21.15.105:0/3799233794 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 msgr2=0x7fc9300a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc924009870 tx=0x7fc92401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 -- 172.21.15.105:0/3799233794 shutdown_connections 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 --2- 172.21.15.105:0/3799233794 >> v2:172.21.15.160:3300/0 conn(0x7fc9301396a0 0x7fc930139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc9300a4910 0x7fc9300a7200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 --2- 172.21.15.105:0/3799233794 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc9300a68e0 0x7fc9300a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 -- 172.21.15.105:0/3799233794 >> 172.21.15.105:0/3799233794 conn(0x7fc930068ba0 msgr2=0x7fc930068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 -- 172.21.15.105:0/3799233794 shutdown_connections 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.864+0000 7fc937e03640 1 -- 172.21.15.105:0/3799233794 wait complete. 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: build_initial_monmap 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: monmap: 2024-09-17T16:28:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:09.896 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 librados: starting msgr at 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 librados: starting objecter 2024-09-17T16:28:09.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 Processor -- start 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- start start 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 librados: setting wanted keys 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 librados: calling monclient init 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: init 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _add_conns ranks=[2,0,5,4,7,8,3,1,6] 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-b con 0x7fc9300a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 0x7fc930138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-a con 0x7fc9300a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc9301396a0 0x7fc93013d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): picked mon.noname-h con 0x7fc9301396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc93006e630 con 0x7fc9300a68e0 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc9300a7af0 con 0x7fc9300a4910 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:09.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc930143320 con 0x7fc9301396a0 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient(hunting): authenticate will time out at 973.137041s 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47016/0 (socket says 172.21.15.105:47016) 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 -- 172.21.15.105:0/2832804881 learned_addr learned my addr 172.21.15.105:0/2832804881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): get_auth_request con 0x7fc9300a4910 auth_method 0 2024-09-17T16:28:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 0x7fc930138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): get_auth_request con 0x7fc9300a68e0 auth_method 0 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient(hunting): handle_auth_done global_id 4358 payload 824 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 -- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc9301396a0 msgr2=0x7fc93013d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T16:28:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc9301396a0 0x7fc93013d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 -- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 msgr2=0x7fc930138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 0x7fc930138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient: _finish_hunting 0 2024-09-17T16:28:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 monclient: found mon.noname-b 2024-09-17T16:28:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 -- 172.21.15.105:0/2832804881 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93013fcd0 con 0x7fc9300a4910 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 0x7fc930138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc935b9f640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc92800ce40 tx=0x7fc92800a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 client.?.objecter ms_handle_connect 0x7fc9300a4910 2024-09-17T16:28:09.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc92800e860 con 0x7fc9300a4910 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: dump: 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.907 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.908 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.910 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92800ea40 con 0x7fc9300a4910 2024-09-17T16:28:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc92800ebf0 con 0x7fc9300a4910 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 5 monclient: authenticate success, global_id 4358 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _renew_subs 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc930123b70 con 0x7fc9300a4910 2024-09-17T16:28:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92ffff640 4 set_mon_vals no callback set 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 monclient: dump: 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 client.4358.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _renew_subs 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc930077c10 con 0x7fc9300a4910 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 librados: init done 2024-09-17T16:28:09.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930139ae0 con 0x7fc9300a4910 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc92800e3f0 con 0x7fc9300a4910 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 0x7fc91c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:09.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fc928050350 con 0x7fc9300a4910 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 10 client.4358.objecter ms_dispatch 0x7fc930000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 3 client.4358.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 3 client.4358.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc92e7fc640 20 client.4358.objecter dump_active .. 0 homeless 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 0x7fc91c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:09.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.868+0000 7fc93539e640 10 monclient: get_auth_request con 0x7fc91c03e550 auth_method 0 2024-09-17T16:28:09.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.872+0000 7fc92e7fc640 10 client.4358.objecter ms_handle_connect 0x7fc91c03e550 2024-09-17T16:28:09.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.872+0000 7fc93539e640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 0x7fc91c040a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc9240ad000 tx=0x7fc9240a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:09.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.872+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc93013fcd0 con 0x7fc9300a4910 2024-09-17T16:28:09.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.872+0000 7fc92e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:09.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:09.872+0000 7fc92e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:09.927 INFO:tasks.workunit.client.0.smithi160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.096+0000 7fc937e03640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.096+0000 7fc937e03640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.096+0000 7fc937e03640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.096+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc93013f080 con 0x7fc9300a4910 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc92e7fc640 1 -- 172.21.15.105:0/2832804881 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7fc9280500d0 con 0x7fc9300a4910 2024-09-17T16:28:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc92e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:28:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc92e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:10.107 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":14,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":27,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 10 librados: watch_flush enter 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 10 librados: watch_flush exit 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 20 client.4358.objecter shutdown clearing up homeless session... 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 10 client.4358.objecter successfully canceled tick 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 msgr2=0x7fc91c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 0x7fc91c040a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc9240ad000 tx=0x7fc9240a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 10 monclient: shutdown 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 msgr2=0x7fc930137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc92800ce40 tx=0x7fc92800a660 comp rx=0 tx=0).stop 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 shutdown_connections 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc91c03e550 0x7fc91c040a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc9301396a0 0x7fc93013d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9300a68e0 0x7fc930138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 --2- 172.21.15.105:0/2832804881 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9300a4910 0x7fc930137fc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 >> 172.21.15.105:0/2832804881 conn(0x7fc930068ba0 msgr2=0x7fc9300a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 shutdown_connections 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 -- 172.21.15.105:0/2832804881 wait complete. 2024-09-17T16:28:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.100+0000 7fc937e03640 1 librados: shutdown 2024-09-17T16:28:10.124 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:28:10.124 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:28:10.124 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:28:10.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:28:10.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: get_monmap_and_config 2024-09-17T16:28:10.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: monmap: 2024-09-17T16:28:10.241 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.241 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:10.238204+0000 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:10.238204+0000 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:10.242 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.243 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.244 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.244 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:10.244 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:10.244 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 Processor -- start 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 -- start start 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: init 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient: _add_conns ranks=[1,6,10,2,7,9,8,4,5,0,3] 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f78dc09f8e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-d con 0x7f78dc0a46f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f78dc13b980 0x7f78dc13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-f con 0x7f78dc13b980 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78dc09f3f0 con 0x7f78dc09f8e0 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f78dc0a1a50 con 0x7f78dc13b980 2024-09-17T16:28:10.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f78dc0a1ca0 con 0x7f78dc0a46f0 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.232+0000 7f78e17a6640 10 monclient(hunting): authenticate will time out at 973.503929s 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57458/0 (socket says 172.21.15.105:57458) 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 -- 172.21.15.105:0/555794523 learned_addr learned my addr 172.21.15.105:0/555794523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): get_auth_request con 0x7f78dc0a46f0 auth_method 0 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78db7fe640 1 --2- 172.21.15.105:0/555794523 >> v2:172.21.15.160:3300/0 conn(0x7f78dc13b980 0x7f78dc13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78db7fe640 10 monclient(hunting): get_auth_request con 0x7f78dc13b980 auth_method 0 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient(hunting): handle_auth_done global_id 4416 payload 274 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 -- 172.21.15.105:0/555794523 >> v2:172.21.15.160:3300/0 conn(0x7f78dc13b980 msgr2=0x7f78dc13dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 --2- 172.21.15.105:0/555794523 >> v2:172.21.15.160:3300/0 conn(0x7f78dc13b980 0x7f78dc13dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 -- 172.21.15.105:0/555794523 >> v1:172.21.15.160:6789/0 conn(0x7f78dc09f8e0 legacy=0x7f78dc09b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 monclient: found mon.noname-d 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:10.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 -- 172.21.15.105:0/555794523 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc0a27c0 con 0x7f78dc0a46f0 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78da7fc640 1 --2- 172.21.15.105:0/555794523 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d001c240 comp rx=0 tx=0).ready entity=mon.6 client_cookie=db7220c955c80855 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 1 -- 172.21.15.105:0/555794523 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78d00b0070 con 0x7f78dc0a46f0 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: dump: 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 1 -- 172.21.15.105:0/555794523 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d00aa810 con 0x7f78dc0a46f0 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 1 -- 172.21.15.105:0/555794523 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78d00a9300 con 0x7f78dc0a46f0 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 5 monclient: authenticate success, global_id 4416 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 4 set_mon_vals no callback set 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78d9ffb640 10 monclient: dump: 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 10 monclient: shutdown 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 -- 172.21.15.105:0/555794523 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 msgr2=0x7f78dc09b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 --2- 172.21.15.105:0/555794523 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d001c240 comp rx=0 tx=0).stop 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 -- 172.21.15.105:0/555794523 shutdown_connections 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 --2- 172.21.15.105:0/555794523 >> v2:172.21.15.160:3300/0 conn(0x7f78dc13b980 0x7f78dc13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 --2- 172.21.15.105:0/555794523 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f78dc0a46f0 0x7f78dc09b760 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 -- 172.21.15.105:0/555794523 >> 172.21.15.105:0/555794523 conn(0x7f78dc0a83e0 msgr2=0x7f78dc0a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 -- 172.21.15.105:0/555794523 shutdown_connections 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.236+0000 7f78e17a6640 1 -- 172.21.15.105:0/555794523 wait complete. 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: monmap: 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:10.270 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:10.271 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 librados: starting msgr at 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 librados: starting objecter 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 Processor -- start 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- start start 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 librados: setting wanted keys 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 librados: calling monclient init 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: init 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _add_conns ranks=[8,4,2,0,5,6,7,1,3] 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 0x7f78dc136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-e con 0x7f78dc09f8e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78dc0a46f0 0x7f78dc070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-c con 0x7f78dc0a46f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:10.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): picked mon.noname-b con 0x7f78dc13b980 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc0a3170 con 0x7f78dc13b980 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f78dc0a2ff0 con 0x7f78dc0a46f0 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f78dc0a32f0 con 0x7f78dc09f8e0 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): authenticate will time out at 973.509009s 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 0x7f78dc136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78dc0a46f0 0x7f78dc070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47024/0 (socket says 172.21.15.105:47024) 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 0x7f78dc136760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34796/0 (socket says 172.21.15.105:34796) 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 -- 172.21.15.105:0/3967740612 learned_addr learned my addr 172.21.15.105:0/3967740612 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): get_auth_request con 0x7f78dc0a46f0 auth_method 0 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): get_auth_request con 0x7f78dc09f8e0 auth_method 0 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): get_auth_request con 0x7f78dc13b980 auth_method 0 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient(hunting): handle_auth_done global_id 4367 payload 824 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 msgr2=0x7f78dc136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 0x7f78dc136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78dc0a46f0 msgr2=0x7f78dc070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78dc0a46f0 0x7f78dc070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc074840 con 0x7f78dc13b980 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78dc0a46f0 0x7f78dc070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f78dc09f8e0 0x7f78dc136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78db7fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d560 tx=0x7f78d400da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 client.?.objecter ms_handle_connect 0x7f78dc13b980 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78d4012a60 con 0x7f78dc13b980 2024-09-17T16:28:10.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: dump: 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.278 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d4019070 con 0x7f78dc13b980 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78d4012c60 con 0x7f78dc13b980 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: dump: 2024-09-17T16:28:10.279 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.280 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78d97fa640 4 set_mon_vals no callback set 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 5 monclient: authenticate success, global_id 4367 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _renew_subs 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc074b20 con 0x7f78dc13b980 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 client.4367.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _renew_subs 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc074560 con 0x7f78dc13b980 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 librados: init done 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 0x7f78dc138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc072ed0 con 0x7f78dc071750 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 0x7f78dc138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient: get_auth_request con 0x7f78dc071750 auth_method 0 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f78d40125f0 con 0x7f78dc13b980 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 10 monclient(hunting): handle_auth_done global_id 4482 payload 824 2024-09-17T16:28:10.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78dc0a08b0 con 0x7f78dc071750 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f78c803e580 0x7f78c8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f78d400f040 con 0x7f78dc13b980 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 client.4367.objecter ms_dispatch 0x7f78dc000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 3 client.4367.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 3 client.4367.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78daffd640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 0x7f78dc138760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 20 client.4367.objecter dump_active .. 0 homeless 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 client.4367.objecter ms_handle_connect 0x7f78dc071750 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 client.4367.objecter resend_mon_ops 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78cc004de0 con 0x7f78dc071750 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: dump: 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.285 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f78c803e580 0x7f78c8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78da7fc640 10 monclient: get_auth_request con 0x7f78c803e580 auth_method 0 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f78cc004970 con 0x7f78dc071750 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 msgr2=0x7f78dc138760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 0x7f78dc138760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc00d520 comp rx=0 tx=0).stop 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 msgr2=0x7f78dc138760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.240+0000 7f78c37fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc071750 0x7f78dc138760 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc00d520 comp rx=0 tx=0).stop 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.244+0000 7f78c37fe640 10 client.4367.objecter ms_handle_connect 0x7f78c803e580 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.244+0000 7f78da7fc640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f78c803e580 0x7f78c8040a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78d00ad000 tx=0x7f78d00a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 0x7f78dc1390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc073320 con 0x7f78dc138ca0 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 0x7f78dc1390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient: get_auth_request con 0x7f78dc138ca0 auth_method 0 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 10 monclient(hunting): handle_auth_done global_id 4491 payload 824 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 1 -- 172.21.15.105:0/3967740612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f78dc0a0410 con 0x7f78dc138ca0 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 client.4367.objecter ms_handle_connect 0x7f78dc138ca0 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 client.4367.objecter resend_mon_ops 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78daffd640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 0x7f78dc1390b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78cc0056b0 con 0x7f78dc138ca0 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:10.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 monclient: dump: 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.289 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f78cc01e030 con 0x7f78dc138ca0 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 msgr2=0x7f78dc1390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 0x7f78dc1390b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc003af0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 msgr2=0x7f78dc1390b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78c37fe640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78dc138ca0 0x7f78dc1390b0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0048e0 tx=0x7f78cc003af0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.290 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 37907, 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:10.291 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:10.292 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 27, 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.293 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.294 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.297 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.298 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.299 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:10.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:10.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:10.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.305 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.311 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.312 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.312 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.312 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.314 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.315 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.317 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.321 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:10.322 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 librados: watch_flush enter 2024-09-17T16:28:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 librados: watch_flush exit 2024-09-17T16:28:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 20 client.4367.objecter shutdown clearing up homeless session... 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 client.4367.objecter successfully canceled tick 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f78c803e580 msgr2=0x7f78c8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f78c803e580 0x7f78c8040a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78d00ad000 tx=0x7f78d00a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 10 monclient: shutdown 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 msgr2=0x7f78dc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78db7fe640 1 -- 172.21.15.105:0/3967740612 reap_dead start 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d560 tx=0x7f78d400da20 comp rx=0 tx=0).stop 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 shutdown_connections 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 --2- 172.21.15.105:0/3967740612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78dc13b980 0x7f78dc071210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 >> 172.21.15.105:0/3967740612 conn(0x7f78dc0a83e0 msgr2=0x7f78dc09c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 shutdown_connections 2024-09-17T16:28:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 -- 172.21.15.105:0/3967740612 wait complete. 2024-09-17T16:28:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.248+0000 7f78e17a6640 1 librados: shutdown 2024-09-17T16:28:10.326 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:10.326 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:28:10.326 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 10 monclient: get_monmap_and_config 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 10 monclient: monmap: 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:10.442489+0000 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:10.442489+0000 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.445 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:10.446 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 1 Processor -- start 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 1 -- start start 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.436+0000 7fbb529a7640 10 monclient: init 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient: _add_conns ranks=[10,9,5,4,7,1,3,0,2,6,8] 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbb4c0a68e0 0x7fbb4c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-f con 0x7fbb4c0a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb4c0a4910 0x7fbb4c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-a con 0x7fbb4c0a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c139b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-h con 0x7fbb4c1396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbb4c0a3fc0 con 0x7fbb4c0a4910 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fbb4c0a63c0 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fbb4c0a7740 con 0x7fbb4c1396a0 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient(hunting): authenticate will time out at 973.708236s 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb4c0a4910 0x7fbb4c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb4c0a4910 0x7fbb4c0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53202/0 (socket says 172.21.15.105:53202) 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/3007795849 learned_addr learned my addr 172.21.15.105:0/3007795849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): get_auth_request con 0x7fbb4c0a4910 auth_method 0 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/3007795849 >> v2:172.21.15.160:3300/0 conn(0x7fbb4c0a68e0 0x7fbb4c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4bfff640 10 monclient(hunting): get_auth_request con 0x7fbb4c0a68e0 auth_method 0 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_done global_id 4500 payload 274 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb50f44640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/3007795849 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 msgr2=0x7fbb4c139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/3007795849 >> v2:172.21.15.160:3300/0 conn(0x7fbb4c0a68e0 msgr2=0x7fbb4c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/3007795849 >> v2:172.21.15.160:3300/0 conn(0x7fbb4c0a68e0 0x7fbb4c0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 monclient: found mon.noname-a 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:10.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/3007795849 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c06ebc0 con 0x7fbb4c0a4910 2024-09-17T16:28:10.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb50f44640 10 monclient: get_auth_request con 0x7fbb4c1396a0 auth_method 0 2024-09-17T16:28:10.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb50f44640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:10.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c0a7200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c074a10 tx=0x7fbb4001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354bb48ea547d6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 1 -- 172.21.15.105:0/3007795849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb400b0070 con 0x7fbb4c0a4910 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: dump: 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.464 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 1 -- 172.21.15.105:0/3007795849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb400aa810 con 0x7fbb4c0a4910 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 1 -- 172.21.15.105:0/3007795849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb400a9300 con 0x7fbb4c0a4910 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb4affd640 10 monclient: dump: 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.466 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 5 monclient: authenticate success, global_id 4500 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:10.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.440+0000 7fbb529a7640 4 set_mon_vals no callback set 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: shutdown 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/3007795849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb4c0a4910 msgr2=0x7fbb4c0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c0a7200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c074a10 tx=0x7fbb4001c3b0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/3007795849 shutdown_connections 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- 172.21.15.105:0/3007795849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c0a7200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- 172.21.15.105:0/3007795849 >> v2:172.21.15.160:3300/0 conn(0x7fbb4c0a68e0 0x7fbb4c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/3007795849 >> 172.21.15.105:0/3007795849 conn(0x7fbb4c068ba0 msgr2=0x7fbb4c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/3007795849 shutdown_connections 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/3007795849 wait complete. 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: monmap: 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.469 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 librados: starting msgr at 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 librados: starting objecter 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 Processor -- start 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- start start 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 librados: setting wanted keys 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 librados: calling monclient init 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: init 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _add_conns ranks=[1,2,5,6,0,3,8,4,7] 2024-09-17T16:28:10.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c1d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-f con 0x7fbb4c0a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 0x7fbb4c1d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-b con 0x7fbb4c0a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): picked mon.noname-h con 0x7fbb4c1396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c06e630 con 0x7fbb4c0a4910 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c0a7af0 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fbb4c143320 con 0x7fbb4c1396a0 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient(hunting): authenticate will time out at 973.714614s 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 0x7fbb4c1d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 0x7fbb4c1d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47036/0 (socket says 172.21.15.105:47036) 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/2821176213 learned_addr learned my addr 172.21.15.105:0/2821176213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): get_auth_request con 0x7fbb4c0a68e0 auth_method 0 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c1d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb50f44640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient(hunting): handle_auth_done global_id 4376 payload 824 2024-09-17T16:28:10.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 msgr2=0x7fbb4c13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb4c1396a0 0x7fbb4c13d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0a4910 msgr2=0x7fbb4c1d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c1d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c123680 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4bfff640 10 monclient: get_auth_request con 0x7fbb4c0a4910 auth_method 0 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb4c0a4910 0x7fbb4c1d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb4b7fe640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 0x7fbb4c1d1b90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097c0 tx=0x7fbb40019150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 client.?.objecter ms_handle_connect 0x7fbb4c0a68e0 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb400b9070 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: dump: 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 5 monclient: authenticate success, global_id 4376 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb400aae20 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _renew_subs 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c123960 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb400b0040 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 client.4376.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb49ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb48ff9640 10 monclient: dump: 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _renew_subs 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c0abb20 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.444+0000 7fbb529a7640 1 librados: init done 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb529a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb529a7640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 0x7fbb4c13df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c051e80 con 0x7fbb4c13db50 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 0x7fbb4c13df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient: get_auth_request con 0x7fbb4c13db50 auth_method 0 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fbb40004040 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_done global_id 4385 payload 824 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb4c122df0 con 0x7fbb4c13db50 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb3803e550 0x7fbb38040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fbb400e8af0 con 0x7fbb4c0a68e0 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 0x7fbb4c13df30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb44007db0 tx=0x7fbb4400f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 client.4376.objecter ms_dispatch 0x7fbb4c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 3 client.4376.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 3 client.4376.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 20 client.4376.objecter dump_active .. 0 homeless 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 client.4376.objecter ms_handle_connect 0x7fbb4c13db50 2024-09-17T16:28:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 client.4376.objecter resend_mon_ops 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb44003e10 con 0x7fbb4c13db50 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 monclient: dump: 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.481 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb50f44640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb3803e550 0x7fbb38040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb50f44640 10 monclient: get_auth_request con 0x7fbb3803e550 auth_method 0 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbb44019070 con 0x7fbb4c13db50 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 msgr2=0x7fbb4c13df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 0x7fbb4c13df30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb44007db0 tx=0x7fbb4400f520 comp rx=0 tx=0).stop 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 msgr2=0x7fbb4c13df30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13db50 0x7fbb4c13df30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb50f44640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb3803e550 0x7fbb38040a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b7e0 tx=0x7fbb3c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.448+0000 7fbb48ff9640 10 client.4376.objecter ms_handle_connect 0x7fbb3803e550 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb529a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb529a7640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 0x7fbb4c0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb529a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c139b10 con 0x7fbb4c13e470 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 0x7fbb4c0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient: get_auth_request con 0x7fbb4c13e470 auth_method 0 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.452+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb4bfff640 10 monclient(hunting): handle_auth_done global_id 4394 payload 824 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb4bfff640 1 -- 172.21.15.105:0/2821176213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbb4c13f880 con 0x7fbb4c13e470 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 client.4376.objecter ms_handle_connect 0x7fbb4c13e470 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 client.4376.objecter resend_mon_ops 2024-09-17T16:28:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb4bfff640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 0x7fbb4c0a0a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000f90 tx=0x7fbb44004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb440046b0 con 0x7fbb4c13e470 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 monclient: dump: 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.485 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fbb44004240 con 0x7fbb4c13e470 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 msgr2=0x7fbb4c0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 0x7fbb4c0a0a20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000f90 tx=0x7fbb44004a50 comp rx=0 tx=0).stop 2024-09-17T16:28:10.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 msgr2=0x7fbb4c0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb48ff9640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c13e470 0x7fbb4c0a0a20 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000f90 tx=0x7fbb44004a50 comp rx=0 tx=0).stop 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:10.487 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38099, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.488 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 27, 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.489 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.490 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.491 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.492 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.493 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.493 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.493 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.493 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.494 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:10.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.499 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:10.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.510 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.510 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.510 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:10.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:10.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.513 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.513 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.514 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.515 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:10.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:10.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.517 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.518 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:10.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.520 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.522 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:10.522 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 10 librados: watch_flush enter 2024-09-17T16:28:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 10 librados: watch_flush exit 2024-09-17T16:28:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 20 client.4376.objecter shutdown clearing up homeless session... 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 10 client.4376.objecter successfully canceled tick 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb3803e550 msgr2=0x7fbb38040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb3803e550 0x7fbb38040a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b7e0 tx=0x7fbb3c008040 comp rx=0 tx=0).stop 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 10 monclient: shutdown 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 msgr2=0x7fbb4c1d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 --2- 172.21.15.105:0/2821176213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb4c0a68e0 0x7fbb4c1d1b90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb400097c0 tx=0x7fbb40019150 comp rx=0 tx=0).stop 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb50f44640 1 -- 172.21.15.105:0/2821176213 reap_dead start 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb50f44640 1 -- 172.21.15.105:0/2821176213 reap_dead start 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 shutdown_connections 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 >> 172.21.15.105:0/2821176213 conn(0x7fbb4c068ba0 msgr2=0x7fbb4c0a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 shutdown_connections 2024-09-17T16:28:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 -- 172.21.15.105:0/2821176213 wait complete. 2024-09-17T16:28:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.456+0000 7fbb529a7640 1 librados: shutdown 2024-09-17T16:28:10.528 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:10.528 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:28:10.529 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: get_monmap_and_config 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: monmap: 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.642 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:10.639944+0000 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:10.639944+0000 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:10.643 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 Processor -- start 2024-09-17T16:28:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 -- start start 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: init 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient: _add_conns ranks=[0,10,2,1,7,9,4,6,5,8,3] 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f75940a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f75940a5240 0x7f759413c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): picked mon.noname-f con 0x7f75940a5240 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 0x7f759413f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): picked mon.noname-b con 0x7f759413cd70 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75940a0b90 con 0x7f75940a4750 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f75940a08b0 con 0x7f75940a5240 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f75940a0e70 con 0x7f759413cd70 2024-09-17T16:28:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759be54640 10 monclient(hunting): authenticate will time out at 973.905718s 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 0x7f759413f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7599bf0640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f75940a4750 0x7f75940a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44720/0 (socket says 172.21.15.105:44720) 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7599bf0640 1 -- 172.21.15.105:0/107828891 learned_addr learned my addr 172.21.15.105:0/107828891 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): get_auth_request con 0x7f759413cd70 auth_method 0 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 1 -- 172.21.15.105:0/107828891 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154725344 0 0) 0x7f7590001120 con 0x7f75940a4750 2024-09-17T16:28:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 10 monclient(hunting): _negotiate 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 10 monclient(hunting): my global_id is 4509 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f75993ef640 1 --2- 172.21.15.105:0/107828891 >> v2:172.21.15.160:3300/0 conn(0x7f75940a5240 0x7f759413c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f7598bee640 1 -- 172.21.15.105:0/107828891 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7580003530 con 0x7f75940a4750 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient(hunting): handle_auth_done global_id 4403 payload 274 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 -- 172.21.15.105:0/107828891 >> v2:172.21.15.160:3300/0 conn(0x7f75940a5240 msgr2=0x7f759413c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 --2- 172.21.15.105:0/107828891 >> v2:172.21.15.160:3300/0 conn(0x7f75940a5240 0x7f759413c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 -- 172.21.15.105:0/107828891 >> v1:172.21.15.105:6789/0 conn(0x7f75940a4750 legacy=0x7f75940a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.636+0000 7f759a3f1640 1 -- 172.21.15.105:0/107828891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759409fca0 con 0x7f759413cd70 2024-09-17T16:28:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759a3f1640 1 --2- 172.21.15.105:0/107828891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 0x7f759413f160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d90 tx=0x7f758c01c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d609d5f6e758aa70 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 1 -- 172.21.15.105:0/107828891 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f758c0b2070 con 0x7f759413cd70 2024-09-17T16:28:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: dump: 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.656 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 1 -- 172.21.15.105:0/107828891 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758c0a5a70 con 0x7f759413cd70 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 5 monclient: authenticate success, global_id 4403 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 1 -- 172.21.15.105:0/107828891 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f758c0a5c60 con 0x7f759413cd70 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 4 set_mon_vals no callback set 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7598bee640 10 monclient: dump: 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: shutdown 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/107828891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 msgr2=0x7f759413f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- 172.21.15.105:0/107828891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 0x7f759413f160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f758c009d90 tx=0x7f758c01c830 comp rx=0 tx=0).stop 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/107828891 shutdown_connections 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- 172.21.15.105:0/107828891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f759413cd70 0x7f759413f160 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- 172.21.15.105:0/107828891 >> v2:172.21.15.160:3300/0 conn(0x7f75940a5240 0x7f759413c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/107828891 >> 172.21.15.105:0/107828891 conn(0x7f75940a5840 msgr2=0x7f75940a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/107828891 shutdown_connections 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/107828891 wait complete. 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: monmap: 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 librados: starting msgr at 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 librados: starting objecter 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 Processor -- start 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- start start 2024-09-17T16:28:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 librados: setting wanted keys 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 librados: calling monclient init 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: init 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _add_conns ranks=[2,5,3,6,7,4,0,8,1] 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 0x7f75941d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): picked mon.noname-b con 0x7f75940a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f75940a5240 0x7f75941d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): picked mon.noname-h con 0x7f75940a5240 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759413cd70 0x7f75941d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): picked mon.noname-g con 0x7f759413cd70 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7594142040 con 0x7f75940a4750 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7594141ec0 con 0x7f759413cd70 2024-09-17T16:28:10.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f75941421c0 con 0x7f75940a5240 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): authenticate will time out at 973.910479s 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 0x7f75941d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 0x7f75941d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47064/0 (socket says 172.21.15.105:47064) 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 -- 172.21.15.105:0/2465228572 learned_addr learned my addr 172.21.15.105:0/2465228572 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): get_auth_request con 0x7f75940a4750 auth_method 0 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient(hunting): handle_auth_done global_id 4412 payload 824 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f75940a5240 msgr2=0x7f75941d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f75940a5240 0x7f75941d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759413cd70 msgr2=0x7f75941d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759413cd70 0x7f75941d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75940a3e90 con 0x7f75940a4750 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f7599bf0640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 0x7f75941d10f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f759000c700 tx=0x7f759000cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 client.?.objecter ms_handle_connect 0x7f75940a4750 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7590017070 con 0x7f75940a4750 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: dump: 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7590005d30 con 0x7f75940a4750 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 5 monclient: authenticate success, global_id 4412 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f75900074e0 con 0x7f75940a4750 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758bfff640 4 set_mon_vals no callback set 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f758a7fc640 10 monclient: dump: 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _renew_subs 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759413c850 con 0x7f75940a4750 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 client.4412.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _renew_subs 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f759409efb0 con 0x7f75940a4750 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 librados: init done 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 0x7f7594063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.640+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f759409f2f0 con 0x7f75940733b0 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 0x7f7594063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient: get_auth_request con 0x7f75940733b0 auth_method 0 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7590007680 con 0x7f75940a4750 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75940a4b30 con 0x7f75940733b0 2024-09-17T16:28:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f75993ef640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 0x7f7594063e00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f759413c320 tx=0x7f758400cb80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f757803e480 0x7f7578040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 client.4412.objecter ms_handle_connect 0x7f75940733b0 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 client.4412.objecter resend_mon_ops 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f759004e940 con 0x7f75940a4750 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 client.4412.objecter ms_dispatch 0x7f7594000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 3 client.4412.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 3 client.4412.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 20 client.4412.objecter dump_active .. 0 homeless 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7584015070 con 0x7f75940733b0 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 monclient: dump: 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.671 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7584007690 con 0x7f75940733b0 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 msgr2=0x7f7594063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f759a3f1640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f757803e480 0x7f7578040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 0x7f7594063e00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f759413c320 tx=0x7f758400cb80 comp rx=0 tx=0).stop 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 msgr2=0x7f7594063e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f75940733b0 0x7f7594063e00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f759a3f1640 10 monclient: get_auth_request con 0x7f757803e480 auth_method 0 2024-09-17T16:28:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f758a7fc640 10 client.4412.objecter ms_handle_connect 0x7f757803e480 2024-09-17T16:28:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.644+0000 7f759a3f1640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f757803e480 0x7f7578040940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f758c0af000 tx=0x7f758c004ac0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f759be54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f759be54640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 0x7f759402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f759be54640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7594051e80 con 0x7f7594064340 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 0x7f759402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient: get_auth_request con 0x7f7594064340 auth_method 0 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 1 -- 172.21.15.105:0/2465228572 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f759409eb80 con 0x7f7594064340 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f75993ef640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 0x7f759402d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f759413c320 tx=0x7f75840046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 client.4412.objecter ms_handle_connect 0x7f7594064340 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 client.4412.objecter resend_mon_ops 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f758401ac30 con 0x7f7594064340 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 monclient: dump: 2024-09-17T16:28:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f75840243f0 con 0x7f7594064340 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 msgr2=0x7f759402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 0x7f759402d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f759413c320 tx=0x7f75840046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 msgr2=0x7f759402d450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.648+0000 7f758a7fc640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7594064340 0x7f759402d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:10.678 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38287, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:10.679 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 27, 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.680 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.681 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:10.682 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:10.683 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:10.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:10.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.689 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.692 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.694 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.699 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.700 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.701 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:10.702 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 10 librados: watch_flush enter 2024-09-17T16:28:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 10 librados: watch_flush exit 2024-09-17T16:28:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 20 client.4412.objecter shutdown clearing up homeless session... 2024-09-17T16:28:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 10 client.4412.objecter successfully canceled tick 2024-09-17T16:28:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f757803e480 msgr2=0x7f7578040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f757803e480 0x7f7578040940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f758c0af000 tx=0x7f758c004ac0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 10 monclient: shutdown 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 msgr2=0x7f75941d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 --2- 172.21.15.105:0/2465228572 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f75940a4750 0x7f75941d10f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f759000c700 tx=0x7f759000cbc0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759a3f1640 1 -- 172.21.15.105:0/2465228572 reap_dead start 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759a3f1640 1 -- 172.21.15.105:0/2465228572 reap_dead start 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 shutdown_connections 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 >> 172.21.15.105:0/2465228572 conn(0x7f75940a5840 msgr2=0x7f75941336e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 shutdown_connections 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 -- 172.21.15.105:0/2465228572 wait complete. 2024-09-17T16:28:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.652+0000 7f759be54640 1 librados: shutdown 2024-09-17T16:28:10.705 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:10.705 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:28:10.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:28:10.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: get_monmap_and_config 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: monmap: 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:10.812514+0000 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:10.812514+0000 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.821 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 Processor -- start 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 -- start start 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: init 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient: _add_conns ranks=[4,2,1,6,7,8,9,0,5,3,10] 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 0x7ff4bc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-c con 0x7ff4bc0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4bc1396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc06ebc0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc0a3fc0 con 0x7ff4bc0a4910 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff4bc0a63c0 con 0x7ff4bc0a68e0 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4c1452640 10 monclient(hunting): authenticate will time out at 974.078239s 2024-09-17T16:28:10.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 0x7ff4bc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47076/0 (socket says 172.21.15.105:47076) 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 0x7ff4bc0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46356/0 (socket says 172.21.15.105:46356) 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 -- 172.21.15.105:0/2130808383 learned_addr learned my addr 172.21.15.105:0/2130808383 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): get_auth_request con 0x7ff4bc0a4910 auth_method 0 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): get_auth_request con 0x7ff4bc0a68e0 auth_method 0 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient(hunting): handle_auth_done global_id 4421 payload 274 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 -- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 msgr2=0x7ff4bc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 --2- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 0x7ff4bc0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 -- 172.21.15.105:0/2130808383 >> v1:172.21.15.160:6789/0 conn(0x7ff4bc1396a0 legacy=0x7ff4bc139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.808+0000 7ff4ba7fc640 1 -- 172.21.15.105:0/2130808383 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0a3690 con 0x7ff4bc0a4910 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4ba7fc640 1 --2- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0098a0 tx=0x7ff4ac01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463139877aac4644 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 1 -- 172.21.15.105:0/2130808383 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4ac0b0070 con 0x7ff4bc0a4910 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: dump: 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.826 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 1 -- 172.21.15.105:0/2130808383 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0aa810 con 0x7ff4bc0a4910 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 5 monclient: authenticate success, global_id 4421 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 1 -- 172.21.15.105:0/2130808383 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4ac0a9300 con 0x7ff4bc0a4910 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 4 set_mon_vals no callback set 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4b9ffb640 10 monclient: dump: 2024-09-17T16:28:10.828 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.829 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: shutdown 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 msgr2=0x7ff4bc0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0098a0 tx=0x7ff4ac01c440 comp rx=0 tx=0).stop 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- 172.21.15.105:0/2130808383 shutdown_connections 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc0a4910 0x7ff4bc0a7200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- 172.21.15.105:0/2130808383 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff4bc0a68e0 0x7ff4bc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- 172.21.15.105:0/2130808383 >> 172.21.15.105:0/2130808383 conn(0x7ff4bc068ba0 msgr2=0x7ff4bc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- 172.21.15.105:0/2130808383 shutdown_connections 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- 172.21.15.105:0/2130808383 wait complete. 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: monmap: 2024-09-17T16:28:10.830 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.831 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 librados: starting msgr at 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 librados: starting objecter 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 Processor -- start 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- start start 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 librados: setting wanted keys 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 librados: calling monclient init 2024-09-17T16:28:10.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: init 2024-09-17T16:28:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient: _add_conns ranks=[5,3,2,6,4,8,7,1,0] 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff4bc0a4910 0x7ff4bc1d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-h con 0x7ff4bc0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4bc0a68e0 0x7ff4bc1d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-g con 0x7ff4bc0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 0x7ff4bc1d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): picked mon.noname-b con 0x7ff4bc1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc1444b0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc144330 con 0x7ff4bc0a68e0 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff4bc144630 con 0x7ff4bc0a4910 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4c1452640 10 monclient(hunting): authenticate will time out at 974.083268s 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 0x7ff4bc1d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 0x7ff4bc1d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47084/0 (socket says 172.21.15.105:47084) 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 learned_addr learned my addr 172.21.15.105:0/1288633926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): get_auth_request con 0x7ff4bc1396a0 auth_method 0 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4ba7fc640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4bc0a68e0 0x7ff4bc1d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient(hunting): handle_auth_done global_id 4430 payload 824 2024-09-17T16:28:10.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff4bc0a4910 msgr2=0x7ff4bc1d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff4bc0a4910 0x7ff4bc1d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4bc0a68e0 msgr2=0x7ff4bc1d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4bc0a68e0 0x7ff4bc1d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 monclient: found mon.noname-b 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0a9f90 con 0x7ff4bc1396a0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4bb7fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 0x7ff4bc1d1fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b970 tx=0x7ff4b400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4a37fe640 10 client.?.objecter ms_handle_connect 0x7ff4bc1396a0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4a37fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.812+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b4019070 con 0x7ff4bc1396a0 2024-09-17T16:28:10.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: dump: 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.842 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 5 monclient: authenticate success, global_id 4430 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b400cdb0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 monclient: _renew_subs 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc0abb00 con 0x7ff4bc1396a0 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b40077c0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 client.4430.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: dump: 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.844 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4b97fa640 4 set_mon_vals no callback set 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 monclient: _renew_subs 2024-09-17T16:28:10.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc1229f0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 1 librados: init done 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 0x7ff4bc13d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff4bc051e80 con 0x7ff4bc13d580 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 0x7ff4bc13d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient: get_auth_request con 0x7ff4bc13d580 auth_method 0 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff4b4006d10 con 0x7ff4bc1396a0 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 10 monclient(hunting): handle_auth_done global_id 4425 payload 824 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4bc139ae0 con 0x7ff4bc13d580 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4baffd640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 0x7ff4bc13d960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e7f0 tx=0x7ff4b000ecb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff4a803e550 0x7ff4a8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 client.4430.objecter ms_handle_connect 0x7ff4bc13d580 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 client.4430.objecter resend_mon_ops 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7ff4b404efa0 con 0x7ff4bc1396a0 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 client.4430.objecter ms_dispatch 0x7ff4bc000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 3 client.4430.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 3 client.4430.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 20 client.4430.objecter dump_active .. 0 homeless 2024-09-17T16:28:10.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b0017070 con 0x7ff4bc13d580 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: dump: 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff4b000c930 con 0x7ff4bc13d580 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 msgr2=0x7ff4bc13d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4ba7fc640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff4a803e550 0x7ff4a8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 0x7ff4bc13d960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e7f0 tx=0x7ff4b000ecb0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 msgr2=0x7ff4bc13d960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13d580 0x7ff4bc13d960 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000e7f0 tx=0x7ff4b000ecb0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4ba7fc640 10 monclient: get_auth_request con 0x7ff4a803e550 auth_method 0 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4a37fe640 10 client.4430.objecter ms_handle_connect 0x7ff4a803e550 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.816+0000 7ff4ba7fc640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff4a803e550 0x7ff4a8040a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ad000 tx=0x7ff4ac0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4c1452640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4c1452640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 0x7ff4bc0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4c1452640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff4bc0746d0 con 0x7ff4bc13dea0 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 0x7ff4bc0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient: get_auth_request con 0x7ff4bc13dea0 auth_method 0 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.820+0000 7ff4baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4baffd640 10 monclient(hunting): handle_auth_done global_id 4434 payload 824 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4baffd640 1 -- 172.21.15.105:0/1288633926 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff4bc13fec0 con 0x7ff4bc13dea0 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4baffd640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 0x7ff4bc0a0a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000c10 tx=0x7ff4b00046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 client.4430.objecter ms_handle_connect 0x7ff4bc13dea0 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 client.4430.objecter resend_mon_ops 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4b0017030 con 0x7ff4bc13dea0 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 monclient: dump: 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.851 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:10.852 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff4b0005c70 con 0x7ff4bc13dea0 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 msgr2=0x7ff4bc0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 0x7ff4bc0a0a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000c10 tx=0x7ff4b00046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 msgr2=0x7ff4bc0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4a37fe640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff4bc13dea0 0x7ff4bc0a0a20 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000c10 tx=0x7ff4b00046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38463, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 28, 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:10.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:10.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:10.876 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 10 librados: watch_flush enter 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 10 librados: watch_flush exit 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 20 client.4430.objecter shutdown clearing up homeless session... 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 10 client.4430.objecter successfully canceled tick 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff4a803e550 msgr2=0x7ff4a8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff4a803e550 0x7ff4a8040a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ad000 tx=0x7ff4ac0a6000 comp rx=0 tx=0).stop 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 10 monclient: shutdown 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:10.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 msgr2=0x7ff4bc1d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 --2- 172.21.15.105:0/1288633926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4bc1396a0 0x7ff4bc1d1fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b970 tx=0x7ff4b400be30 comp rx=0 tx=0).stop 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 reap_dead start 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4bb7fe640 1 -- 172.21.15.105:0/1288633926 reap_dead start 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 shutdown_connections 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 >> 172.21.15.105:0/1288633926 conn(0x7ff4bc068ba0 msgr2=0x7ff4bc0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 shutdown_connections 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 -- 172.21.15.105:0/1288633926 wait complete. 2024-09-17T16:28:10.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.824+0000 7ff4c1452640 1 librados: shutdown 2024-09-17T16:28:10.879 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:10.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:28:10.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 10 monclient: get_monmap_and_config 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 10 monclient: build_initial_monmap 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 10 monclient: monmap: 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:10.986749+0000 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:10.986749+0000 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:10.989 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:10.990 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 1 Processor -- start 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 1 -- start start 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.980+0000 7f554193f640 10 monclient: init 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient: _add_conns ranks=[3,9,6,1,8,7,4,2,10,0,5] 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f553c0a68e0 0x7f553c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): picked mon.noname-g con 0x7f553c0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f553c0a4910 0x7f553c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): picked mon.noname-a con 0x7f553c0a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): picked mon.noname-d con 0x7f553c1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f553c0a3fc0 con 0x7f553c0a4910 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f553c0a63c0 con 0x7f553c0a68e0 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f553c0a7740 con 0x7f553c1396a0 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient(hunting): authenticate will time out at 974.252459s 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f553c0a4910 0x7f553c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57486/0 (socket says 172.21.15.105:57486) 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f553c0a4910 0x7f553c0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53214/0 (socket says 172.21.15.105:53214) 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 1 -- 172.21.15.105:0/4225618725 learned_addr learned my addr 172.21.15.105:0/4225618725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): get_auth_request con 0x7f553c1396a0 auth_method 0 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): get_auth_request con 0x7f553c0a4910 auth_method 0 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f553c0a68e0 0x7f553c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): get_auth_request con 0x7f553c0a68e0 auth_method 0 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:10.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient(hunting): handle_auth_done global_id 4518 payload 274 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 -- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 msgr2=0x7f553c139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 -- 172.21.15.105:0/4225618725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f553c0a68e0 msgr2=0x7f553c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f553c0a68e0 0x7f553c0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:10.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 monclient: found mon.noname-a 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 -- 172.21.15.105:0/4225618725 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c06ebc0 con 0x7f553c0a4910 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553b7fe640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f5533fff640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f553c0a4910 0x7f553c0a7200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55240097c0 tx=0x7f552401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94684a10440e3e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 1 -- 172.21.15.105:0/4225618725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55240b0070 con 0x7f553c0a4910 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: dump: 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:10.999 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 1 -- 172.21.15.105:0/4225618725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55240aa810 con 0x7f553c0a4910 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 1 -- 172.21.15.105:0/4225618725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55240a9300 con 0x7f553c0a4910 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 5 monclient: authenticate success, global_id 4518 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 4 set_mon_vals no callback set 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f553a7fc640 10 monclient: dump: 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.001 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 10 monclient: shutdown 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 -- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f553c0a4910 msgr2=0x7f553c0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.984+0000 7f554193f640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f553c0a4910 0x7f553c0a7200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55240097c0 tx=0x7f552401c330 comp rx=0 tx=0).stop 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/4225618725 shutdown_connections 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f553c1396a0 0x7f553c139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f553c0a4910 0x7f553c0a7200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- 172.21.15.105:0/4225618725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f553c0a68e0 0x7f553c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/4225618725 >> 172.21.15.105:0/4225618725 conn(0x7f553c068ba0 msgr2=0x7f553c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/4225618725 shutdown_connections 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/4225618725 wait complete. 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: monmap: 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.003 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 librados: starting msgr at 2024-09-17T16:28:11.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 librados: starting objecter 2024-09-17T16:28:11.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 Processor -- start 2024-09-17T16:28:11.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- start start 2024-09-17T16:28:11.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 librados: setting wanted keys 2024-09-17T16:28:11.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 librados: calling monclient init 2024-09-17T16:28:11.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: init 2024-09-17T16:28:11.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _add_conns ranks=[1,5,7,4,8,3,6,0,2] 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f553c0a4910 0x7f553c1d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): picked mon.noname-f con 0x7f553c0a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 0x7f553c1d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): picked mon.noname-h con 0x7f553c0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f553c1396a0 0x7f553c13d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): picked mon.noname-i con 0x7f553c1396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f553c06e630 con 0x7f553c0a4910 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f553c0a7af0 con 0x7f553c0a68e0 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f553c143320 con 0x7f553c1396a0 2024-09-17T16:28:11.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient(hunting): authenticate will time out at 974.257713s 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 0x7f553c1d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f553c0a4910 0x7f553c1d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f553c1396a0 0x7f553c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 0x7f553c1d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:33066/0 (socket says 172.21.15.105:33066) 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 -- 172.21.15.105:0/1739492380 learned_addr learned my addr 172.21.15.105:0/1739492380 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): get_auth_request con 0x7f553c0a68e0 auth_method 0 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): get_auth_request con 0x7f553c1396a0 auth_method 0 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): get_auth_request con 0x7f553c0a4910 auth_method 0 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f553b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient(hunting): handle_auth_done global_id 4343 payload 824 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f553c1396a0 msgr2=0x7f553c13d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f553c1396a0 0x7f553c13d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f553c0a4910 msgr2=0x7f553c1d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f553c0a4910 0x7f553c1d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 monclient: found mon.noname-h 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0aa1a0 con 0x7f553c0a68e0 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5533fff640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 0x7f553c1d19e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f552401cbd0 tx=0x7f55240190d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 client.?.objecter ms_handle_connect 0x7f553c0a68e0 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55240b0040 con 0x7f553c0a68e0 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: dump: 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.016 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 5 monclient: authenticate success, global_id 4343 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _renew_subs 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c123710 con 0x7f553c0a68e0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55240b9070 con 0x7f553c0a68e0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 client.4343.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _renew_subs 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c0a9de0 con 0x7f553c0a68e0 2024-09-17T16:28:11.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f554193f640 1 librados: init done 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5539ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55240aa7e0 con 0x7f553c0a68e0 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.988+0000 7f5538ff9640 10 monclient: dump: 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.019 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f554193f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f554193f640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 0x7f553c13e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f553c051e80 con 0x7f553c13dc80 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 0x7f553c13e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient: get_auth_request con 0x7f553c13dc80 auth_method 0 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 10 monclient(hunting): handle_auth_done global_id 4418 payload 824 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f553c139b50 con 0x7f553c13dc80 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 client.4343.objecter ms_handle_connect 0x7f553c13dc80 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 client.4343.objecter resend_mon_ops 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553affd640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 0x7f553c13e060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c011dc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f55240aac80 con 0x7f553c0a68e0 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f552003e580 0x7f5520040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f55240e9400 con 0x7f553c0a68e0 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 client.4343.objecter ms_dispatch 0x7f553c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 3 client.4343.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 3 client.4343.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 20 client.4343.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f552c019070 con 0x7f553c13dc80 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 monclient: dump: 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.022 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f552c004300 con 0x7f553c13dc80 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 msgr2=0x7f553c13e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 0x7f553c13e060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c011dc0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 msgr2=0x7f553c13e060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13dc80 0x7f553c13e060 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c011dc0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553b7fe640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f552003e580 0x7f5520040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553b7fe640 10 monclient: get_auth_request con 0x7f552003e580 auth_method 0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f553b7fe640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f552003e580 0x7f5520040a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55340072e0 tx=0x7f553400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.992+0000 7f5538ff9640 10 client.4343.objecter ms_handle_connect 0x7f552003e580 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f554193f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f554193f640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 0x7f553c0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f554193f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f553c13efd0 con 0x7f553c13e5a0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 0x7f553c0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient: get_auth_request con 0x7f553c13e5a0 auth_method 0 2024-09-17T16:28:11.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 10 monclient(hunting): handle_auth_done global_id 4427 payload 824 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:10.996+0000 7f553affd640 1 -- 172.21.15.105:0/1739492380 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f553c0746d0 con 0x7f553c13e5a0 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 client.4343.objecter ms_handle_connect 0x7f553c13e5a0 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 client.4343.objecter resend_mon_ops 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f553affd640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 0x7f553c0a0a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f552c019040 con 0x7f553c13e5a0 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:11.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 monclient: dump: 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.026 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38643, 2024-09-17T16:28:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 28, 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:11.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.030 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:11.031 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.032 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.034 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.035 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.037 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:11.039 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.043 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.045 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.049 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.049 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:11.049 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f552c003c40 con 0x7f553c13e5a0 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 msgr2=0x7f553c0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 0x7f553c0a0a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 msgr2=0x7f553c0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f5538ff9640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f553c13e5a0 0x7f553c0a0a20 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f552c011900 tx=0x7f552c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 10 librados: watch_flush enter 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 10 librados: watch_flush exit 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 20 client.4343.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 10 client.4343.objecter successfully canceled tick 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f552003e580 msgr2=0x7f5520040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f552003e580 0x7f5520040a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55340072e0 tx=0x7f553400b040 comp rx=0 tx=0).stop 2024-09-17T16:28:11.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 10 monclient: shutdown 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 msgr2=0x7f553c1d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 --2- 172.21.15.105:0/1739492380 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f553c0a68e0 0x7f553c1d19e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f552401cbd0 tx=0x7f55240190d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f553b7fe640 1 -- 172.21.15.105:0/1739492380 reap_dead start 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f553b7fe640 1 -- 172.21.15.105:0/1739492380 reap_dead start 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 shutdown_connections 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 >> 172.21.15.105:0/1739492380 conn(0x7f553c068ba0 msgr2=0x7f553c0a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 shutdown_connections 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 -- 172.21.15.105:0/1739492380 wait complete. 2024-09-17T16:28:11.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.000+0000 7f554193f640 1 librados: shutdown 2024-09-17T16:28:11.052 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:11.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:28:11.053 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:28:11.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: get_monmap_and_config 2024-09-17T16:28:11.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: monmap: 2024-09-17T16:28:11.166 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:11.163991+0000 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:11.163991+0000 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.167 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.168 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.168 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:11.168 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:11.168 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 Processor -- start 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 -- start start 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: init 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient: _add_conns ranks=[7,0,1,3,8,6,10,2,5,9,4] 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf00a6a30 0x7f8bf00a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-i con 0x7f8bf00a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bf00a4ab0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8bf00a7350 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf00a4080 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf013a610 con 0x7f8bf00a7350 2024-09-17T16:28:11.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8bf00a7c40 con 0x7f8bf00a6a30 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 10 monclient(hunting): authenticate will time out at 974.429918s 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8bf00a4ab0 0x7f8bf01397f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44724/0 (socket says 172.21.15.105:44724) 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 -- 172.21.15.105:0/1864764216 learned_addr learned my addr 172.21.15.105:0/1864764216 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf00a6a30 0x7f8bf00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): get_auth_request con 0x7f8bf00a6a30 auth_method 0 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906084937 0 0) 0x7f8bf00a4080 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): _negotiate 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): my global_id is 4527 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bd8003530 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 265316977 0 0) 0x7f8bd8003530 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf00a6a30 msgr2=0x7f8bf00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf00a6a30 0x7f8bf00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 >> v1:172.21.15.160:6789/0 conn(0x7f8bf00a7350 legacy=0x7f8bf0139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:28:11.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf00a37e0 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f8be0001970 con 0x7f8bf00a4ab0 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: dump: 2024-09-17T16:28:11.182 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.183 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 >> v1:172.21.15.105:6789/0 conn(0x7f8bf00a4ab0 legacy=0x7f8bf01397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient: _add_conns ranks=[5,8,6,1,2,0,7,3,4] 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bd8004240 0x7f8bd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): picked mon.h con 0x7f8bd8004240 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 0x7f8bd8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): picked mon.e con 0x7f8bd8004da0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8bd8009c60 0x7f8bd800c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): picked mon.d con 0x7f8bd8009c60 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8bf00a7c40 con 0x7f8bd8004240 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8bd800c9d0 con 0x7f8bd8009c60 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8bd800cb80 con 0x7f8bd8004da0 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beeffd640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 0x7f8bd8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8bd8009c60 0x7f8bd800c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): get_auth_request con 0x7f8bd8004da0 auth_method 0 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:11.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): get_auth_request con 0x7f8bd8009c60 auth_method 0 2024-09-17T16:28:11.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bd8004240 0x7f8bd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): get_auth_request con 0x7f8bd8004240 auth_method 0 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8beffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:11.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_done global_id 4527 payload 293 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 -- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8bd8009c60 msgr2=0x7f8bd800c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8bd8009c60 0x7f8bd800c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 -- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bd8004240 msgr2=0x7f8bd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bd8004240 0x7f8bd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 monclient: found mon.e 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 1 -- 172.21.15.105:0/1864764216 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd8003530 con 0x7f8bd8004da0 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 5 monclient: authenticate success, global_id 4527 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf644c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8bd8009c60 0x7f8bd800c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.160+0000 7f8bf49e9640 1 -- 172.21.15.105:0/1864764216 reap_dead start 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 0x7f8bd8005220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00a7e50 tx=0x7f8be001c950 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3fbd5ae0523deb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be0007b70 con 0x7f8bd8004da0 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: dump: 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.189 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be00a9020 con 0x7f8bd8004da0 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 1 -- 172.21.15.105:0/1864764216 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be0005e10 con 0x7f8bd8004da0 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 4 set_mon_vals no callback set 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8beeffd640 10 monclient: dump: 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.191 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: shutdown 2024-09-17T16:28:11.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 msgr2=0x7f8bd8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 0x7f8bd8005220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00a7e50 tx=0x7f8be001c950 comp rx=0 tx=0).stop 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- 172.21.15.105:0/1864764216 shutdown_connections 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 --2- 172.21.15.105:0/1864764216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bd8004da0 0x7f8bd8005220 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- 172.21.15.105:0/1864764216 >> 172.21.15.105:0/1864764216 conn(0x7f8bf00a8d00 msgr2=0x7f8bf00a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- 172.21.15.105:0/1864764216 shutdown_connections 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- 172.21.15.105:0/1864764216 wait complete. 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: monmap: 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.193 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 librados: starting msgr at 2024-09-17T16:28:11.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 librados: starting objecter 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 Processor -- start 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 -- start start 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 librados: setting wanted keys 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 librados: calling monclient init 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: init 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 10 monclient: _add_conns ranks=[7,8,1,3,6,0,2,5,4] 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.164+0000 7f8bf644c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bd8004da0 0x7f8bf01386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-i con 0x7f8bd8004da0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 0x7f8bf0122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-e con 0x7f8bf0138be0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0122bc0 0x7f8bf0123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): picked mon.noname-f con 0x7f8bf0122bc0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:11.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf00a7760 con 0x7f8bf0122bc0 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8bf006b5b0 con 0x7f8bd8004da0 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8bf006e710 con 0x7f8bf0138be0 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): authenticate will time out at 974.436073s 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 0x7f8bf0122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 0x7f8bf0122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34846/0 (socket says 172.21.15.105:34846) 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 -- 172.21.15.105:0/4255523538 learned_addr learned my addr 172.21.15.105:0/4255523538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): get_auth_request con 0x7f8bf0138be0 auth_method 0 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient(hunting): handle_auth_done global_id 4436 payload 824 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bd8004da0 0x7f8bf01386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bd8004da0 msgr2=0x7f8bf01386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bd8004da0 0x7f8bf01386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0122bc0 msgr2=0x7f8bf0123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0122bc0 0x7f8bf0123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 monclient: found mon.noname-e 2024-09-17T16:28:11.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf0123d00 con 0x7f8bf0138be0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 0x7f8bf0122680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8be001b050 tx=0x7f8bd8005840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.?.objecter ms_handle_connect 0x7f8bf0138be0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be0001970 con 0x7f8bf0138be0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: dump: 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.198 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.199 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be0001090 con 0x7f8bf0138be0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 5 monclient: authenticate success, global_id 4436 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient: _renew_subs 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf0139040 con 0x7f8bf0138be0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be00014c0 con 0x7f8bf0138be0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:11.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 client.4436.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: dump: 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.201 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient: _renew_subs 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bee7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01257c0 con 0x7f8bf0138be0 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 librados: init done 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 0x7f8bf009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0125e00 con 0x7f8bf0123580 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8be0007070 con 0x7f8bf0138be0 2024-09-17T16:28:11.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 0x7f8bf009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bdc03e550 0x7f8bdc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient: get_auth_request con 0x7f8bf0123580 auth_method 0 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f8be00aa030 con 0x7f8bf0138be0 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.4436.objecter ms_dispatch 0x7f8bf0000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 3 client.4436.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 3 client.4436.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 20 client.4436.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf49e9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bdc03e550 0x7f8bdc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf49e9640 10 monclient: get_auth_request con 0x7f8bdc03e550 auth_method 0 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 10 monclient(hunting): handle_auth_done global_id 4456 payload 824 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bf013a000 con 0x7f8bf0123580 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.4436.objecter ms_handle_connect 0x7f8bf0123580 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.4436.objecter resend_mon_ops 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8beffff640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 0x7f8bf009da00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4012d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 client.4436.objecter ms_handle_connect 0x7f8bdc03e550 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8bf49e9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bdc03e550 0x7f8bdc040a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80039c0 tx=0x7f8be8009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be4015450 con 0x7f8bf0123580 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:28:11.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.168+0000 7f8becff9640 10 monclient: dump: 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.205 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8be4010ab0 con 0x7f8bf0123580 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 msgr2=0x7f8bf009da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 0x7f8bf009da00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4012d60 comp rx=0 tx=0).stop 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 msgr2=0x7f8bf009da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.172+0000 7f8becff9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf0123580 0x7f8bf009da00 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4012d60 comp rx=0 tx=0).stop 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8bf644c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8bf644c640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 0x7f8bf009e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8bf644c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf01c61f0 con 0x7f8bf009df40 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 0x7f8bf009e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient: get_auth_request con 0x7f8bf009df40 auth_method 0 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 10 monclient(hunting): handle_auth_done global_id 4465 payload 824 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 1 -- 172.21.15.105:0/4255523538 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8bf0125a00 con 0x7f8bf009df40 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8beffff640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 0x7f8bf009e350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8becff9640 10 client.4436.objecter ms_handle_connect 0x7f8bf009df40 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.176+0000 7f8becff9640 10 client.4436.objecter resend_mon_ops 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8be4015c90 con 0x7f8bf009df40 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 10 monclient: dump: 2024-09-17T16:28:11.209 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.210 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f8be402d450 con 0x7f8bf009df40 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 msgr2=0x7f8bf009e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 0x7f8bf009e350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4004740 comp rx=0 tx=0).stop 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 msgr2=0x7f8bf009e350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8becff9640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bf009df40 0x7f8bf009e350 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8be40128a0 tx=0x7f8be4004740 comp rx=0 tx=0).stop 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.211 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38783, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.212 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 28, 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.213 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.214 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:11.215 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.216 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.217 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:11.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:11.220 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.223 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.224 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.224 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.226 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.229 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.231 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:11.236 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:11.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:11.238 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 10 librados: watch_flush enter 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 10 librados: watch_flush exit 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 20 client.4436.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 10 client.4436.objecter successfully canceled tick 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bdc03e550 msgr2=0x7f8bdc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bdc03e550 0x7f8bdc040a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd80039c0 tx=0x7f8be8009040 comp rx=0 tx=0).stop 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 10 monclient: shutdown 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 msgr2=0x7f8bf0122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 --2- 172.21.15.105:0/4255523538 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf0138be0 0x7f8bf0122680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8be001b050 tx=0x7f8bd8005840 comp rx=0 tx=0).stop 2024-09-17T16:28:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf49e9640 1 -- 172.21.15.105:0/4255523538 reap_dead start 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf49e9640 1 -- 172.21.15.105:0/4255523538 reap_dead start 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 shutdown_connections 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 >> 172.21.15.105:0/4255523538 conn(0x7f8bf00a8d00 msgr2=0x7f8bf00aa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 shutdown_connections 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 -- 172.21.15.105:0/4255523538 wait complete. 2024-09-17T16:28:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.180+0000 7f8bf644c640 1 librados: shutdown 2024-09-17T16:28:11.241 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:11.241 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:28:11.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: get_monmap_and_config 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: monmap: 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:11.352758+0000 2024-09-17T16:28:11.355 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:11.352758+0000 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:11.356 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:11.357 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 Processor -- start 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 -- start start 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: init 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient: _add_conns ranks=[9,8,2,3,1,4,7,0,10,5,6] 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-a con 0x7f61380a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 0x7f61380a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-e con 0x7f61380a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-b con 0x7f61381396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:11.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f61380a3fc0 con 0x7f61380a68e0 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f61380a63c0 con 0x7f61381396a0 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f61380a7740 con 0x7f61380a4910 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613ed57640 10 monclient(hunting): authenticate will time out at 974.618552s 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 0x7f61380a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47086/0 (socket says 172.21.15.105:47086) 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53224/0 (socket says 172.21.15.105:53224) 2024-09-17T16:28:11.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 -- 172.21.15.105:0/3319987607 learned_addr learned my addr 172.21.15.105:0/3319987607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): get_auth_request con 0x7f61381396a0 auth_method 0 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): get_auth_request con 0x7f61380a68e0 auth_method 0 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): get_auth_request con 0x7f61380a4910 auth_method 0 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient(hunting): handle_auth_done global_id 4439 payload 274 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 -- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 msgr2=0x7f61380a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 0x7f61380a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 -- 172.21.15.105:0/3319987607 >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 msgr2=0x7f61380a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 --2- 172.21.15.105:0/3319987607 >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 monclient: found mon.noname-b 2024-09-17T16:28:11.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613d2f4640 1 -- 172.21.15.105:0/3319987607 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613806ebc0 con 0x7f61381396a0 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f6137fff640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 0x7f61380a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.348+0000 7f613caf3640 1 --2- 172.21.15.105:0/3319987607 >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613d2f4640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6130009e50 tx=0x7f613001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1674e580cf3bb2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 1 -- 172.21.15.105:0/3319987607 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61300090d0 con 0x7f61381396a0 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: dump: 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.371 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 1 -- 172.21.15.105:0/3319987607 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61300a5b90 con 0x7f61381396a0 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 1 -- 172.21.15.105:0/3319987607 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61300a5d80 con 0x7f61381396a0 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 5 monclient: authenticate success, global_id 4439 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 4 set_mon_vals no callback set 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f61377fe640 10 monclient: dump: 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.373 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: shutdown 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 msgr2=0x7f6138139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6130009e50 tx=0x7f613001c8c0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- 172.21.15.105:0/3319987607 shutdown_connections 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61381396a0 0x7f6138139b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- 172.21.15.105:0/3319987607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61380a4910 0x7f61380a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- 172.21.15.105:0/3319987607 >> v2:172.21.15.105:3300/0 conn(0x7f61380a68e0 0x7f61380a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- 172.21.15.105:0/3319987607 >> 172.21.15.105:0/3319987607 conn(0x7f6138068ba0 msgr2=0x7f6138068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- 172.21.15.105:0/3319987607 shutdown_connections 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- 172.21.15.105:0/3319987607 wait complete. 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: monmap: 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.375 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 librados: starting msgr at 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 librados: starting objecter 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 Processor -- start 2024-09-17T16:28:11.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- start start 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 librados: setting wanted keys 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 librados: calling monclient init 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: init 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient: _add_conns ranks=[6,5,7,4,2,0,1,3,8] 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 0x7f61381d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-d con 0x7f61380a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61380a68e0 0x7f61381d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-h con 0x7f61380a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61381396a0 0x7f613813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): picked mon.noname-i con 0x7f61381396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f613806e630 con 0x7f61380a68e0 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f61380a7af0 con 0x7f61380a4910 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6138143320 con 0x7f61381396a0 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613ed57640 10 monclient(hunting): authenticate will time out at 974.623770s 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 0x7f61381d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 0x7f61381d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57494/0 (socket says 172.21.15.105:57494) 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 1 -- 172.21.15.105:0/1916152077 learned_addr learned my addr 172.21.15.105:0/1916152077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): get_auth_request con 0x7f61380a4910 auth_method 0 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.352+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 10 monclient(hunting): handle_auth_done global_id 4452 payload 824 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61381396a0 msgr2=0x7f613813d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61380a68e0 0x7f61381d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613d2f4640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61381396a0 0x7f613813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61381396a0 0x7f613813d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61380a68e0 msgr2=0x7f61381d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61380a68e0 0x7f61381d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 monclient: found mon.noname-d 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6138123930 con 0x7f61380a4910 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613caf3640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 0x7f61381d1500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f612800ec10 tx=0x7f612800c520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.?.objecter ms_handle_connect 0x7f61380a4910 2024-09-17T16:28:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6128017070 con 0x7f61380a4910 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: dump: 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612800cc30 con 0x7f61380a4910 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 5 monclient: authenticate success, global_id 4452 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 monclient: _renew_subs 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6138123c10 con 0x7f61380a4910 2024-09-17T16:28:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f612800cde0 con 0x7f61380a4910 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6136ffd640 4 set_mon_vals no callback set 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 client.4452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: dump: 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.382 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 monclient: _renew_subs 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f61380ab9d0 con 0x7f61380a4910 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 1 librados: init done 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f612801a070 con 0x7f61380a4910 2024-09-17T16:28:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 0x7f613802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f613813f9b0 con 0x7f613813e270 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f612403e580 0x7f6124040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f612800f040 con 0x7f61380a4910 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.4452.objecter ms_dispatch 0x7f6138000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 3 client.4452.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 3 client.4452.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 20 client.4452.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 0x7f613802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient: get_auth_request con 0x7f613813e270 auth_method 0 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613d2f4640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f612403e580 0x7f6124040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613d2f4640 10 monclient: get_auth_request con 0x7f612403e580 auth_method 0 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 10 monclient(hunting): handle_auth_done global_id 4332 payload 824 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f613813f2f0 con 0x7f613813e270 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.4452.objecter ms_handle_connect 0x7f613813e270 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.4452.objecter resend_mon_ops 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f6137fff640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 0x7f613802d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f613813d450 tx=0x7f612c010c20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 client.4452.objecter ms_handle_connect 0x7f612403e580 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f613d2f4640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f612403e580 0x7f6124040a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61300ae000 tx=0x7f61300a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f612c00f040 con 0x7f613813e270 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: dump: 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.386 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f612c01c400 con 0x7f613813e270 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 msgr2=0x7f613802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 0x7f613802d450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f613813d450 tx=0x7f612c010c20 comp rx=0 tx=0).stop 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 msgr2=0x7f613802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.356+0000 7f61357fa640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e270 0x7f613802d450 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f613813d450 tx=0x7f612c010c20 comp rx=0 tx=0).stop 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f613ed57640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f613ed57640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 0x7f613802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f613ed57640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f61381226e0 con 0x7f613813e650 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 0x7f613802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient: get_auth_request con 0x7f613813e650 auth_method 0 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 10 monclient(hunting): handle_auth_done global_id 4341 payload 824 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 1 -- 172.21.15.105:0/1916152077 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6138139b50 con 0x7f613813e650 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f6137fff640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 0x7f613802d990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f612c01b000 tx=0x7f612c0046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 10 client.4452.objecter ms_handle_connect 0x7f613813e650 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 10 client.4452.objecter resend_mon_ops 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f612c01ccb0 con 0x7f613813e650 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.364+0000 7f61357fa640 10 monclient: dump: 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f612c01c840 con 0x7f613813e650 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 msgr2=0x7f613802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 0x7f613802d990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f612c01b000 tx=0x7f612c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 msgr2=0x7f613802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f61357fa640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f613813e650 0x7f613802d990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.392 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 38991, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.393 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 28, 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.394 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.395 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:11.396 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.397 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.399 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:11.403 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.405 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.406 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:11.407 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.408 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.410 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.411 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.412 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.413 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:11.414 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 10 librados: watch_flush enter 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 10 librados: watch_flush exit 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 20 client.4452.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 10 client.4452.objecter successfully canceled tick 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f612403e580 msgr2=0x7f6124040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f612403e580 0x7f6124040a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61300ae000 tx=0x7f61300a7070 comp rx=0 tx=0).stop 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 10 monclient: shutdown 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 msgr2=0x7f61381d1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 --2- 172.21.15.105:0/1916152077 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61380a4910 0x7f61381d1500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f612800ec10 tx=0x7f612800c520 comp rx=0 tx=0).stop 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613d2f4640 1 -- 172.21.15.105:0/1916152077 reap_dead start 2024-09-17T16:28:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613d2f4640 1 -- 172.21.15.105:0/1916152077 reap_dead start 2024-09-17T16:28:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 shutdown_connections 2024-09-17T16:28:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 >> 172.21.15.105:0/1916152077 conn(0x7f6138068ba0 msgr2=0x7f61381403f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 shutdown_connections 2024-09-17T16:28:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 -- 172.21.15.105:0/1916152077 wait complete. 2024-09-17T16:28:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.368+0000 7f613ed57640 1 librados: shutdown 2024-09-17T16:28:11.417 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:11.417 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:28:11.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:28:11.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: get_monmap_and_config 2024-09-17T16:28:11.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: monmap: 2024-09-17T16:28:11.538 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.538 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:11.536047+0000 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:11.536047+0000 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.539 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:11.540 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:11.540 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 Processor -- start 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 -- start start 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: init 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient: _add_conns ranks=[2,1,9,0,3,10,8,5,7,4,6] 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-b con 0x7f92000a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f92000a4ab0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-a con 0x7f92000a7350 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92000a4080 con 0x7f92000a4ab0 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f92000a7c40 con 0x7f92000a7350 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f92000a37e0 con 0x7f92000a6a30 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f92069d5640 10 monclient(hunting): authenticate will time out at 974.801818s 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47094/0 (socket says 172.21.15.105:47094) 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53230/0 (socket says 172.21.15.105:53230) 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 -- 172.21.15.105:0/869178159 learned_addr learned my addr 172.21.15.105:0/869178159 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): get_auth_request con 0x7f92000a6a30 auth_method 0 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): get_auth_request con 0x7f92000a7350 auth_method 0 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient(hunting): handle_auth_done global_id 4448 payload 274 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 -- 172.21.15.105:0/869178159 >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 msgr2=0x7f9200139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 --2- 172.21.15.105:0/869178159 >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 -- 172.21.15.105:0/869178159 >> v1:172.21.15.160:6789/0 conn(0x7f92000a4ab0 legacy=0x7f92001397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 monclient: found mon.noname-b 2024-09-17T16:28:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f91fffff640 1 -- 172.21.15.105:0/869178159 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92000a3ac0 con 0x7f92000a6a30 2024-09-17T16:28:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.532+0000 7f9204f72640 1 --2- 172.21.15.105:0/869178159 >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 1 --2- 172.21.15.105:0/869178159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009e50 tx=0x7f91f801c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68afcac5b7956137 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 1 -- 172.21.15.105:0/869178159 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91f80090d0 con 0x7f92000a6a30 2024-09-17T16:28:11.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:11.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: dump: 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 1 -- 172.21.15.105:0/869178159 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f80a5b50 con 0x7f92000a6a30 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 5 monclient: authenticate success, global_id 4448 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 1 -- 172.21.15.105:0/869178159 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91f80a5d40 con 0x7f92000a6a30 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:11.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91feffd640 10 monclient: dump: 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 4 set_mon_vals no callback set 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: shutdown 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/869178159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 msgr2=0x7f92000a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- 172.21.15.105:0/869178159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009e50 tx=0x7f91f801c8c0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/869178159 shutdown_connections 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- 172.21.15.105:0/869178159 >> v2:172.21.15.105:3300/0 conn(0x7f92000a7350 0x7f9200139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- 172.21.15.105:0/869178159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92000a6a30 0x7f92000a6e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/869178159 >> 172.21.15.105:0/869178159 conn(0x7f92000a8d00 msgr2=0x7f92000a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/869178159 shutdown_connections 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/869178159 wait complete. 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: monmap: 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.557 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 librados: starting msgr at 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 librados: starting objecter 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 Processor -- start 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- start start 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 librados: setting wanted keys 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 librados: calling monclient init 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: init 2024-09-17T16:28:11.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _add_conns ranks=[5,6,3,8,0,7,1,4,2] 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f92000a4ab0 0x7f92001388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-h con 0x7f92000a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 0x7f9200138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-d con 0x7f92000a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92000a7350 0x7f920013e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): picked mon.noname-g con 0x7f92000a7350 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f920013a440 con 0x7f92000a7350 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f920006e770 con 0x7f92000a4ab0 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f920013a5c0 con 0x7f92000a6a30 2024-09-17T16:28:11.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient(hunting): authenticate will time out at 974.806745s 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 0x7f9200138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 0x7f9200138e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57496/0 (socket says 172.21.15.105:57496) 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 -- 172.21.15.105:0/3407720620 learned_addr learned my addr 172.21.15.105:0/3407720620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request con 0x7f92000a6a30 auth_method 0 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f9204f72640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92000a7350 0x7f920013e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f92000a4ab0 0x7f92001388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f9204f72640 10 monclient(hunting): get_auth_request con 0x7f92000a7350 auth_method 0 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f9204f72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f9204f72640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f9204f72640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 10 monclient(hunting): get_auth_request con 0x7f92000a4ab0 auth_method 0 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_done global_id 4461 payload 824 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f92000a4ab0 msgr2=0x7f92001388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f92000a4ab0 0x7f92001388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92000a7350 msgr2=0x7f920013e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92000a7350 0x7f920013e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92001231e0 con 0x7f92000a6a30 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91ff7fe640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 0x7f9200138e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c410 tx=0x7f91e800c8d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 client.?.objecter ms_handle_connect 0x7f92000a6a30 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91e8011a00 con 0x7f92000a6a30 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: dump: 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.563 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 5 monclient: authenticate success, global_id 4461 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e8017070 con 0x7f92000a6a30 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _renew_subs 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92001234c0 con 0x7f92000a6a30 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91e8011c60 con 0x7f92000a6a30 2024-09-17T16:28:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 client.4461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fe7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f91fcff9640 10 monclient: dump: 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _renew_subs 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f92000a24f0 con 0x7f92000a6a30 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.536+0000 7f92069d5640 1 librados: init done 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f92069d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f92069d5640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 0x7f920013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9200051e80 con 0x7f920013e610 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f91e8011590 con 0x7f92000a6a30 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f91e003e550 0x7f91e0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f91e804f3f0 con 0x7f92000a6a30 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 client.4461.objecter ms_dispatch 0x7f9200000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 0x7f920013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 3 client.4461.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 3 client.4461.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 20 client.4461.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient: get_auth_request con 0x7f920013e610 auth_method 0 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f9204f72640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f91e003e550 0x7f91e0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f9204f72640 10 monclient: get_auth_request con 0x7f91e003e550 auth_method 0 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 10 monclient(hunting): handle_auth_done global_id 4361 payload 824 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9200122bd0 con 0x7f920013e610 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f9204f72640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f91e003e550 0x7f91e0040a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f920006e240 tx=0x7f91f0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 client.4461.objecter ms_handle_connect 0x7f91e003e550 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 client.4461.objecter ms_handle_connect 0x7f920013e610 2024-09-17T16:28:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 client.4461.objecter resend_mon_ops 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fffff640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 0x7f920013e9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f920006e6e0 tx=0x7f91f8019660 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91f80bd070 con 0x7f920013e610 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 monclient: dump: 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f91f80a56e0 con 0x7f920013e610 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 msgr2=0x7f920013e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 0x7f920013e9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f920006e6e0 tx=0x7f91f8019660 comp rx=0 tx=0).stop 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 msgr2=0x7f920013e9f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.540+0000 7f91fcff9640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013e610 0x7f920013e9f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 0x7f920013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f920006a200 con 0x7f920013ef30 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 0x7f920013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient: get_auth_request con 0x7f920013ef30 auth_method 0 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 1 -- 172.21.15.105:0/3407720620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92000a6e10 con 0x7f920013ef30 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fffff640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 0x7f920013a800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000f90 tx=0x7f91f80a7000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 client.4461.objecter ms_handle_connect 0x7f920013ef30 2024-09-17T16:28:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 client.4461.objecter resend_mon_ops 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91f80046a0 con 0x7f920013ef30 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 monclient: dump: 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f91f8004230 con 0x7f920013ef30 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 msgr2=0x7f920013a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 0x7f920013a800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000f90 tx=0x7f91f80a7000 comp rx=0 tx=0).stop 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 msgr2=0x7f920013a800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f91fcff9640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f920013ef30 0x7f920013a800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 39151, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:11.576 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 28, 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:11.577 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:11.578 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.579 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:11.580 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.581 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.582 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.586 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:11.589 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.590 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.591 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.593 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.597 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.598 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:11.599 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.599 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.599 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:11.599 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 librados: watch_flush enter 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 librados: watch_flush exit 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 20 client.4461.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 client.4461.objecter successfully canceled tick 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f91e003e550 msgr2=0x7f91e0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f91e003e550 0x7f91e0040a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f920006e240 tx=0x7f91f0008040 comp rx=0 tx=0).stop 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 10 monclient: shutdown 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 msgr2=0x7f9200138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 --2- 172.21.15.105:0/3407720620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f92000a6a30 0x7f9200138e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c410 tx=0x7f91e800c8d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f9204f72640 1 -- 172.21.15.105:0/3407720620 reap_dead start 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f9204f72640 1 -- 172.21.15.105:0/3407720620 reap_dead start 2024-09-17T16:28:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 shutdown_connections 2024-09-17T16:28:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 >> 172.21.15.105:0/3407720620 conn(0x7f92000a8d00 msgr2=0x7f9200141260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 shutdown_connections 2024-09-17T16:28:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 -- 172.21.15.105:0/3407720620 wait complete. 2024-09-17T16:28:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.548+0000 7f92069d5640 1 librados: shutdown 2024-09-17T16:28:11.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:11.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:28:11.602 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:28:11.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: get_monmap_and_config 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: monmap: 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:11.708329+0000 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:11.708329+0000 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.711 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:11.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 Processor -- start 2024-09-17T16:28:11.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 -- start start 2024-09-17T16:28:11.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: init 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient: _add_conns ranks=[4,6,5,3,9,1,7,8,10,2,0] 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-c con 0x7f9ee80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-d con 0x7f9ee80a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9ee81396a0 0x7f9ee8139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-h con 0x7f9ee81396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9ee80a3fc0 con 0x7f9ee80a68e0 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9ee80a63c0 con 0x7f9ee81396a0 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9ee80a7740 con 0x7f9ee80a4910 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9eee7f0640 10 monclient(hunting): authenticate will time out at 974.973955s 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:57508/0 (socket says 172.21.15.105:57508) 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46376/0 (socket says 172.21.15.105:46376) 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 -- 172.21.15.105:0/1385873168 learned_addr learned my addr 172.21.15.105:0/1385873168 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): get_auth_request con 0x7f9ee80a4910 auth_method 0 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request con 0x7f9ee80a68e0 auth_method 0 2024-09-17T16:28:11.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_done global_id 4470 payload 274 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 -- 172.21.15.105:0/1385873168 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9ee81396a0 msgr2=0x7f9ee8139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9ee81396a0 0x7f9ee8139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 -- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 msgr2=0x7f9ee80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee77fe640 1 -- 172.21.15.105:0/1385873168 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee806ec10 con 0x7f9ee80a4910 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.704+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0098d0 tx=0x7f9edc01cea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=5f6deab3d4e15c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 1 -- 172.21.15.105:0/1385873168 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9edc0b0070 con 0x7f9ee80a4910 2024-09-17T16:28:11.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: dump: 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.719 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 1 -- 172.21.15.105:0/1385873168 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc0aa810 con 0x7f9ee80a4910 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 5 monclient: authenticate success, global_id 4470 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 1 -- 172.21.15.105:0/1385873168 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9edc0a9300 con 0x7f9ee80a4910 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:11.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee6ffd640 10 monclient: dump: 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.721 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 4 set_mon_vals no callback set 2024-09-17T16:28:11.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: shutdown 2024-09-17T16:28:11.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 msgr2=0x7f9ee80a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0098d0 tx=0x7f9edc01cea0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/1385873168 shutdown_connections 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9ee81396a0 0x7f9ee8139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee80a4910 0x7f9ee80a7200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/1385873168 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/1385873168 >> 172.21.15.105:0/1385873168 conn(0x7f9ee8068ba0 msgr2=0x7f9ee8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/1385873168 shutdown_connections 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/1385873168 wait complete. 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: monmap: 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 librados: starting msgr at 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 librados: starting objecter 2024-09-17T16:28:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 Processor -- start 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- start start 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 librados: setting wanted keys 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 librados: calling monclient init 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: init 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _add_conns ranks=[8,4,2,1,0,7,6,5,3] 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 0x7f9ee8137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-e con 0x7f9ee80a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee8138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-c con 0x7f9ee80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee81396a0 0x7f9ee813d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): picked mon.noname-b con 0x7f9ee81396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee8143320 con 0x7f9ee81396a0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9ee806e630 con 0x7f9ee80a68e0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f9ee81434a0 con 0x7f9ee80a4910 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient(hunting): authenticate will time out at 974.978737s 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 0x7f9ee8137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee8138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 0x7f9ee8137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34858/0 (socket says 172.21.15.105:34858) 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee8138500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46384/0 (socket says 172.21.15.105:46384) 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee81396a0 0x7f9ee813d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 1 -- 172.21.15.105:0/3565402157 learned_addr learned my addr 172.21.15.105:0/3565402157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request con 0x7f9ee80a4910 auth_method 0 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): get_auth_request con 0x7f9ee80a68e0 auth_method 0 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): get_auth_request con 0x7f9ee81396a0 auth_method 0 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient(hunting): handle_auth_done global_id 4457 payload 824 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 msgr2=0x7f9ee8137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 0x7f9ee8137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 msgr2=0x7f9ee8138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee8138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 monclient: found mon.noname-b 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee813fd20 con 0x7f9ee81396a0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee80a68e0 0x7f9ee8138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee80a4910 0x7f9ee8137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eecd8d640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee81396a0 0x7f9ee813d6f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000bdf0 tx=0x7f9ee000bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 client.?.objecter ms_handle_connect 0x7f9ee81396a0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ee0019070 con 0x7f9ee81396a0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: dump: 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.738 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee000cc70 con 0x7f9ee81396a0 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ee000ce60 con 0x7f9ee81396a0 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee4ff9640 10 monclient: dump: 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.740 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 5 monclient: authenticate success, global_id 4457 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9ee67fc640 4 set_mon_vals no callback set 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _renew_subs 2024-09-17T16:28:11.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee8123b70 con 0x7f9ee81396a0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 client.4457.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _renew_subs 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee8077c80 con 0x7f9ee81396a0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.708+0000 7f9eee7f0640 1 librados: init done 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9eee7f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 0x7f9ee813e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9ee8051e80 con 0x7f9ee813dc30 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9ee001c360 con 0x7f9ee81396a0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ed403e480 0x7f9ed4040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f9ee004f5e0 con 0x7f9ee81396a0 2024-09-17T16:28:11.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 client.4457.objecter ms_dispatch 0x7f9ee8000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 3 client.4457.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 3 client.4457.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 0x7f9ee813e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 20 client.4457.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient: get_auth_request con 0x7f9ee813dc30 auth_method 0 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ed403e480 0x7f9ed4040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee77fe640 10 monclient: get_auth_request con 0x7f9ed403e480 auth_method 0 2024-09-17T16:28:11.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_done global_id 4372 payload 824 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ee80aa120 con 0x7f9ee813dc30 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 client.4457.objecter ms_handle_connect 0x7f9ee813dc30 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 client.4457.objecter resend_mon_ops 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 0x7f9ee813e010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009ef0 tx=0x7f9ed800f610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 client.4457.objecter ms_handle_connect 0x7f9ed403e480 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee77fe640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ed403e480 0x7f9ed4040940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0ad000 tx=0x7f9edc004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed800fe50 con 0x7f9ee813dc30 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 monclient: dump: 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.744 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ed800fe50 con 0x7f9ee813dc30 2024-09-17T16:28:11.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 msgr2=0x7f9ee813e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 0x7f9ee813e010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009ef0 tx=0x7f9ed800f610 comp rx=0 tx=0).stop 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 msgr2=0x7f9ee813e010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.712+0000 7f9ee4ff9640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813dc30 0x7f9ee813e010 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009ef0 tx=0x7f9ed800f610 comp rx=0 tx=0).stop 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 0x7f9ee80a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9ee8139ae0 con 0x7f9ee813e550 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 0x7f9ee80a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient: get_auth_request con 0x7f9ee813e550 auth_method 0 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 10 monclient(hunting): handle_auth_done global_id 4381 payload 824 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 1 -- 172.21.15.105:0/3565402157 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ee813f0f0 con 0x7f9ee813e550 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee7fff640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 0x7f9ee80a0a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c10 tx=0x7f9ed80046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 client.4457.objecter ms_handle_connect 0x7f9ee813e550 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 client.4457.objecter resend_mon_ops 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed8022aa0 con 0x7f9ee813e550 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 monclient: dump: 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.747 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.748 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f9ed8022630 con 0x7f9ee813e550 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 msgr2=0x7f9ee80a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 0x7f9ee80a0a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c10 tx=0x7f9ed80046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 msgr2=0x7f9ee80a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9ee4ff9640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9ee813e550 0x7f9ee80a0a20 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c10 tx=0x7f9ed80046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.749 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 14, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 39343, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:28:11.750 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 29, 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.751 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.752 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:11.753 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:11.754 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:11.755 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.758 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:11.759 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:11.761 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:11.769 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:11.770 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 librados: watch_flush enter 2024-09-17T16:28:11.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 librados: watch_flush exit 2024-09-17T16:28:11.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 20 client.4457.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 client.4457.objecter successfully canceled tick 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ed403e480 msgr2=0x7f9ed4040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ed403e480 0x7f9ed4040940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0ad000 tx=0x7f9edc004740 comp rx=0 tx=0).stop 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 10 monclient: shutdown 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee81396a0 msgr2=0x7f9ee813d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 --2- 172.21.15.105:0/3565402157 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee81396a0 0x7f9ee813d6f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000bdf0 tx=0x7f9ee000bee0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eecd8d640 1 -- 172.21.15.105:0/3565402157 reap_dead start 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eecd8d640 1 -- 172.21.15.105:0/3565402157 reap_dead start 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 shutdown_connections 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 >> 172.21.15.105:0/3565402157 conn(0x7f9ee8068ba0 msgr2=0x7f9ee80a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 shutdown_connections 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 -- 172.21.15.105:0/3565402157 wait complete. 2024-09-17T16:28:11.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.720+0000 7f9eee7f0640 1 librados: shutdown 2024-09-17T16:28:11.774 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:28:11.774 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'i'] 2024-09-17T16:28:11.774 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['f', 'h'] 2024-09-17T16:28:11.774 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-17T16:28:11.774 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a store 2024-09-17T16:28:11.774 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a sync_force --yes-i-really-mean-it 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: get_monmap_and_config 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: monmap: 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:11.893720+0000 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:11.893720+0000 2024-09-17T16:28:11.896 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.898 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:11.898 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:11.899 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 Processor -- start 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 -- start start 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: init 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient: _add_conns ranks=[6,0,2,3,10,5,4,1,8,7,9] 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc42c0a68e0 0x7fc42c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-d con 0x7fc42c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc42c0a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:11.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 0x7fc42c139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-b con 0x7fc42c0a7200 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc42c06ebc0 con 0x7fc42c0a4910 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0a3fc0 con 0x7fc42c0a7200 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc42c0a63c0 con 0x7fc42c0a68e0 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc43242f640 10 monclient(hunting): authenticate will time out at 975.159392s 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc42c0a4910 0x7fc42c1396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44736/0 (socket says 172.21.15.105:44736) 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42b7fe640 1 -- 172.21.15.105:0/116943216 learned_addr learned my addr 172.21.15.105:0/116943216 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 0x7fc42c139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc42c0a68e0 0x7fc42c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): get_auth_request con 0x7fc42c0a7200 auth_method 0 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): get_auth_request con 0x7fc42c0a68e0 auth_method 0 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc4309cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 1 -- 172.21.15.105:0/116943216 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3528129744 0 0) 0x7fc42c06ebc0 con 0x7fc42c0a4910 2024-09-17T16:28:11.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 10 monclient(hunting): _negotiate 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 10 monclient(hunting): my global_id is 4554 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.888+0000 7fc42affd640 1 -- 172.21.15.105:0/116943216 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc414003530 con 0x7fc42c0a4910 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 10 monclient(hunting): handle_auth_done global_id 4466 payload 274 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 -- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc42c0a68e0 msgr2=0x7fc42c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc42c0a68e0 0x7fc42c0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 -- 172.21.15.105:0/116943216 >> v1:172.21.15.105:6789/0 conn(0x7fc42c0a4910 legacy=0x7fc42c1396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 monclient: found mon.noname-b 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 -- 172.21.15.105:0/116943216 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c0a3690 con 0x7fc42c0a7200 2024-09-17T16:28:11.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc4309cc640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 0x7fc42c139db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc424009dc0 tx=0x7fc42401c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f0a7264b1ac0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 1 -- 172.21.15.105:0/116943216 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4240b2070 con 0x7fc42c0a7200 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: dump: 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.911 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 1 -- 172.21.15.105:0/116943216 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4240a5b30 con 0x7fc42c0a7200 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 1 -- 172.21.15.105:0/116943216 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4240a5d60 con 0x7fc42c0a7200 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 5 monclient: authenticate success, global_id 4466 2024-09-17T16:28:11.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 4 set_mon_vals no callback set 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc42affd640 10 monclient: dump: 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.913 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 10 monclient: shutdown 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 -- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 msgr2=0x7fc42c139db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 0x7fc42c139db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc424009dc0 tx=0x7fc42401c840 comp rx=0 tx=0).stop 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 -- 172.21.15.105:0/116943216 shutdown_connections 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc42c0a7200 0x7fc42c139db0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 --2- 172.21.15.105:0/116943216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc42c0a68e0 0x7fc42c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 -- 172.21.15.105:0/116943216 >> 172.21.15.105:0/116943216 conn(0x7fc42c068ba0 msgr2=0x7fc42c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 -- 172.21.15.105:0/116943216 shutdown_connections 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 -- 172.21.15.105:0/116943216 wait complete. 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 10 monclient: build_initial_monmap 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 10 monclient: monmap: 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 librados: starting msgr at 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 librados: starting objecter 2024-09-17T16:28:11.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.892+0000 7fc43242f640 1 Processor -- start 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- start start 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 librados: setting wanted keys 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 librados: calling monclient init 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: init 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _add_conns ranks=[1,0,7,2,6,3,5,8,4] 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc42c0a4910 0x7fc42c13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-f con 0x7fc42c0a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 0x7fc42c13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-a con 0x7fc42c0a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc42c0a7200 0x7fc42c13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): picked mon.noname-i con 0x7fc42c0a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c13a2f0 con 0x7fc42c0a68e0 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c06e630 con 0x7fc42c0a4910 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc42c13a470 con 0x7fc42c0a7200 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): authenticate will time out at 975.164478s 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 0x7fc42c13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 0x7fc42c13e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53240/0 (socket says 172.21.15.105:53240) 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 -- 172.21.15.105:0/1917687266 learned_addr learned my addr 172.21.15.105:0/1917687266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): get_auth_request con 0x7fc42c0a68e0 auth_method 0 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient(hunting): handle_auth_done global_id 4563 payload 824 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc4309cc640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc42c0a7200 0x7fc42c13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc42c0a7200 msgr2=0x7fc42c13e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc42c0a4910 0x7fc42c13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc42c0a7200 0x7fc42c13e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc42c0a4910 msgr2=0x7fc42c13df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc42c0a4910 0x7fc42c13df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 monclient: found mon.noname-a 2024-09-17T16:28:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c123650 con 0x7fc42c0a68e0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42b7fe640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 0x7fc42c13e480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc420001ec0 tx=0x7fc42000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.?.objecter ms_handle_connect 0x7fc42c0a68e0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc420017070 con 0x7fc42c0a68e0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: dump: 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc420007070 con 0x7fc42c0a68e0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc420007710 con 0x7fc42c0a68e0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 5 monclient: authenticate success, global_id 4563 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _renew_subs 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c0a2430 con 0x7fc42c0a68e0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42a7fc640 4 set_mon_vals no callback set 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:11.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: dump: 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 client.4563.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _renew_subs 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c06ddd0 con 0x7fc42c0a68e0 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 librados: init done 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 0x7fc42c13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c051e80 con 0x7fc42c13a8c0 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 0x7fc42c13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient: get_auth_request con 0x7fc42c13a8c0 auth_method 0 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc4200072a0 con 0x7fc42c0a68e0 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc41803e550 0x7fc418040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 10 monclient(hunting): handle_auth_done global_id 4572 payload 824 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fc42004e960 con 0x7fc42c0a68e0 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.4563.objecter ms_dispatch 0x7fc42c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 3 client.4563.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 3 client.4563.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc42c0a6cc0 con 0x7fc42c13a8c0 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 20 client.4563.objecter dump_active .. 0 homeless 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.4563.objecter ms_handle_connect 0x7fc42c13a8c0 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.4563.objecter resend_mon_ops 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc42bfff640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 0x7fc42c13aca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0107c0 tx=0x7fc41c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc41c00f040 con 0x7fc42c13a8c0 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: dump: 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc4309cc640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc41803e550 0x7fc418040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc4309cc640 10 monclient: get_auth_request con 0x7fc41803e550 auth_method 0 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc41c01c400 con 0x7fc42c13a8c0 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 msgr2=0x7fc42c13aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 0x7fc42c13aca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0107c0 tx=0x7fc41c010c80 comp rx=0 tx=0).stop 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 msgr2=0x7fc42c13aca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13a8c0 0x7fc42c13aca0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0107c0 tx=0x7fc41c010c80 comp rx=0 tx=0).stop 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc428ff9640 10 client.4563.objecter ms_handle_connect 0x7fc41803e550 2024-09-17T16:28:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.896+0000 7fc4309cc640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc41803e550 0x7fc418040a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4240af000 tx=0x7fc4240a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 0x7fc42c1db110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0746d0 con 0x7fc42c13b1e0 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 0x7fc42c1db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient: get_auth_request con 0x7fc42c13b1e0 auth_method 0 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 10 monclient(hunting): handle_auth_done global_id 4581 payload 824 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 1 -- 172.21.15.105:0/1917687266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fc42c139fc0 con 0x7fc42c13b1e0 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc42bfff640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 0x7fc42c1db110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c10 tx=0x7fc41c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 client.4563.objecter ms_handle_connect 0x7fc42c13b1e0 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 client.4563.objecter resend_mon_ops 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc41c00fea0 con 0x7fc42c13b1e0 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 monclient: dump: 2024-09-17T16:28:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:11.932 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fc41c025070 con 0x7fc42c13b1e0 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 msgr2=0x7fc42c1db110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 0x7fc42c1db110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc41c000c10 tx=0x7fc41c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 msgr2=0x7fc42c1db110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc428ff9640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c13b1e0 0x7fc42c1db110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:11.933 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:11.934 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:28:11.934 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:28:11.934 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:11.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 librados: watch_flush enter 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 librados: watch_flush exit 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 20 client.4563.objecter shutdown clearing up homeless session... 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 client.4563.objecter successfully canceled tick 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc41803e550 msgr2=0x7fc418040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc41803e550 0x7fc418040a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc4240af000 tx=0x7fc4240a7070 comp rx=0 tx=0).stop 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 10 monclient: shutdown 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 msgr2=0x7fc42c13e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 --2- 172.21.15.105:0/1917687266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc42c0a68e0 0x7fc42c13e480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc420001ec0 tx=0x7fc42000c960 comp rx=0 tx=0).stop 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc4309cc640 1 -- 172.21.15.105:0/1917687266 reap_dead start 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc4309cc640 1 -- 172.21.15.105:0/1917687266 reap_dead start 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 shutdown_connections 2024-09-17T16:28:11.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 >> 172.21.15.105:0/1917687266 conn(0x7fc42c068ba0 msgr2=0x7fc42c0aa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 shutdown_connections 2024-09-17T16:28:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 -- 172.21.15.105:0/1917687266 wait complete. 2024-09-17T16:28:11.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:11.904+0000 7fc43242f640 1 librados: shutdown 2024-09-17T16:28:11.937 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-17T16:28:11.937 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T16:28:11.937 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:28:11.948 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T16:28:11.948 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-17T16:28:11.948 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-17T16:28:11.948 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-17T16:28:11.948 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:28:11.968 INFO:tasks.ceph.mon.i:Stopped 2024-09-17T16:28:11.969 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-17T16:28:11.969 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-17T16:28:11.969 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-17T16:28:11.969 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-17T16:28:11.969 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:28:26.970 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-17T16:28:26.971 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-17T16:28:26.971 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-17T16:28:26.971 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-17T16:28:26.972 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-17T16:28:26.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:28:27.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: get_monmap_and_config 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: build_initial_monmap 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: monmap: 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:27.100329+0000 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:27.100329+0000 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:27.103 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:27.104 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 Processor -- start 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 -- start start 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: init 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient: _add_conns ranks=[0,1,7,4,2,10,6,5,8,9,3] 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84300a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f84300a4910 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-i con 0x7f84300a73d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843006ebc0 con 0x7f84300a68e0 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84300a3690 con 0x7f84300a4910 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:27.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f84300a3fc0 con 0x7f84300a73d0 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f84357ad640 10 monclient(hunting): authenticate will time out at 990.366026s 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f842effd640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f84300a68e0 legacy=0x7f84300a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f842f7fe640 1 -- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 msgr2=0x7f8430142220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f842e7fc640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f84300a4910 0x7f843013eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:42666/0 (socket says 172.21.15.105:42666) 2024-09-17T16:28:27.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.096+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 learned_addr learned my addr 172.21.15.105:0/628098182 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:27.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.300+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> v1:172.21.15.105:6789/0 conn(0x7f84300a68e0 legacy=0x7f84300a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:27.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.300+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 msgr2=0x7f8430142220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:27.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.300+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:27.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.704+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> v1:172.21.15.105:6789/0 conn(0x7f84300a68e0 legacy=0x7f84300a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:27.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.704+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 msgr2=0x7f8430142220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:27.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:27.704+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:28.508+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> v1:172.21.15.105:6789/0 conn(0x7f84300a68e0 legacy=0x7f84300a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:28.508+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 msgr2=0x7f8430142220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:28.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:28.508+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:30.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): tick 2024-09-17T16:28:30.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:28:30.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:28:30.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:28:30.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 msgr2=0x7f8430142220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f84300a73d0 0x7f8430142220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> v1:172.21.15.160:6789/0 conn(0x7f84300a4910 legacy=0x7f843013eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> v1:172.21.15.105:6789/0 conn(0x7f84300a68e0 legacy=0x7f84300a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient: _add_conns ranks=[8,10,4,2,5,6,3,9] 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-e con 0x7f841c003830 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-f con 0x7f841c006280 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-c con 0x7f841c008dc0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f84300a3fc0 con 0x7f841c006280 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:30.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f841c00b9a0 con 0x7f841c008dc0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f841c00bba0 con 0x7f841c003830 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842cff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003830 auth_method 0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c006280 auth_method 0 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c008dc0 auth_method 0 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:30.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:30.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c006280 auth_method 0 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003830 auth_method 0 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c008dc0 auth_method 0 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:30.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:30.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.304+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c008dc0 auth_method 0 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c006280 auth_method 0 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003830 auth_method 0 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:30.708+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c006280 auth_method 0 2024-09-17T16:28:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c008dc0 auth_method 0 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003830 auth_method 0 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:31.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:31.512+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): tick 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 msgr2=0x7f841c005c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f841c003830 0x7f841c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f841c008dc0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:33.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.160:3300/0 conn(0x7f841c006280 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient: _add_conns ranks=[6,5,3,2,9] 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 reap_dead start 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-d con 0x7f841c00c1d0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-h con 0x7f841c00eba0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-g con 0x7f841c0116b0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f84300a3fc0 con 0x7f841c0116b0 2024-09-17T16:28:33.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f841c00b9a0 con 0x7f841c00eba0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f841c00bba0 con 0x7f841c00c1d0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842cff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 reap_dead start 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c00c1d0 auth_method 0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c0116b0 auth_method 0 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c00eba0 auth_method 0 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:33.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.100+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c0116b0 auth_method 0 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c00c1d0 auth_method 0 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c00eba0 auth_method 0 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:33.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.300+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c00c1d0 auth_method 0 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c0116b0 auth_method 0 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c00eba0 auth_method 0 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:33.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:33.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:33.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:33.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:33.704+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:34.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:34.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:34.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:34.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f841c00eba0 auth_method 0 2024-09-17T16:28:34.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:34.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:34.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f841c0116b0 auth_method 0 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c00c1d0 auth_method 0 2024-09-17T16:28:34.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:34.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:34.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:34.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:34.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:34.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842f7fe640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:34.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:28:34.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:34.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:34.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:34.508+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): tick 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 msgr2=0x7f841c00e5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f841c00c1d0 0x7f841c00e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 msgr2=0x7f841c010f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f841c00eba0 0x7f841c010f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 msgr2=0x7f841c013b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f841c0116b0 0x7f841c013b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient: _add_conns ranks=[2,9] 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-b con 0x7f841c003ed0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): picked mon.noname-a con 0x7f841c004350 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f84300a3fc0 con 0x7f841c004350 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f841c00b9a0 con 0x7f841c003ed0 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842cff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003ed0 auth_method 0 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.096+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:36.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:36.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:36.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003ed0 auth_method 0 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.300+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:28:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003ed0 auth_method 0 2024-09-17T16:28:36.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:28:36.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:36.700+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:28:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842e7fc640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842e7fc640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f841c003ed0 auth_method 0 2024-09-17T16:28:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient(hunting): handle_auth_done global_id 4475 payload 274 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 msgr2=0x7f841c008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> v2:172.21.15.105:3300/0 conn(0x7f841c004350 0x7f841c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 1 monclient: found mon.noname-b 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:37.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.500+0000 7f842effd640 1 -- 172.21.15.105:0/628098182 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84300a3970 con 0x7f841c003ed0 2024-09-17T16:28:37.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842effd640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8424003990 tx=0x7f842401e750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1fc811fa9143bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 1 -- 172.21.15.105:0/628098182 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84240b2070 con 0x7f841c003ed0 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: dump: 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.516 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 1 -- 172.21.15.105:0/628098182 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84240ac810 con 0x7f841c003ed0 2024-09-17T16:28:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 1 -- 172.21.15.105:0/628098182 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84240ace30 con 0x7f841c003ed0 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 5 monclient: authenticate success, global_id 4475 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 4 set_mon_vals no callback set 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842dffb640 10 monclient: dump: 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.518 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.519 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 10 monclient: shutdown 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 1 -- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 msgr2=0x7f841c00b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 1 --2- 172.21.15.105:0/628098182 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f841c003ed0 0x7f841c00b260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8424003990 tx=0x7f842401e750 comp rx=0 tx=0).stop 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f842f7fe640 1 -- 172.21.15.105:0/628098182 reap_dead start 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 1 -- 172.21.15.105:0/628098182 shutdown_connections 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 1 -- 172.21.15.105:0/628098182 >> 172.21.15.105:0/628098182 conn(0x7f8430068ba0 msgr2=0x7f8430068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.508+0000 7f84357ad640 1 -- 172.21.15.105:0/628098182 shutdown_connections 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 -- 172.21.15.105:0/628098182 wait complete. 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient: build_initial_monmap 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient: monmap: 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 librados: starting msgr at 2024-09-17T16:28:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 librados: starting objecter 2024-09-17T16:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 Processor -- start 2024-09-17T16:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 -- start start 2024-09-17T16:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 librados: setting wanted keys 2024-09-17T16:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 librados: calling monclient init 2024-09-17T16:28:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient: init 2024-09-17T16:28:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient: _add_conns ranks=[3,2,6,0,5,4,1,7,8] 2024-09-17T16:28:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-g con 0x7f8430138680 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-b con 0x7f8430124860 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8430122a70 0x7f8430122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient(hunting): picked mon.noname-d con 0x7f8430122a70 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.512+0000 7f84357ad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f843013a030 con 0x7f8430124860 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8430139eb0 con 0x7f8430138680 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f843013a1b0 con 0x7f8430122a70 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient(hunting): authenticate will time out at 1000.784052s 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8430122a70 0x7f8430122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:52158/0 (socket says 172.21.15.105:52158) 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:48756/0 (socket says 172.21.15.105:48756) 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 -- 172.21.15.105:0/1113202301 learned_addr learned my addr 172.21.15.105:0/1113202301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): get_auth_request con 0x7f8430138680 auth_method 0 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): get_auth_request con 0x7f8430124860 auth_method 0 2024-09-17T16:28:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): get_auth_request con 0x7f8430122a70 auth_method 0 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient(hunting): handle_auth_done global_id 4359 payload 824 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 -- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8430122a70 msgr2=0x7f8430122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8430122a70 0x7f8430122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 -- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 msgr2=0x7f8430122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 monclient: found mon.noname-g 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 -- 172.21.15.105:0/1113202301 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843006df10 con 0x7f8430138680 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842effd640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f842401b360 tx=0x7f842401b4f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 client.?.objecter ms_handle_connect 0x7f8430138680 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84240b2040 con 0x7f8430138680 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: dump: 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84240ace30 con 0x7f8430138680 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84240ad870 con 0x7f8430138680 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 monclient: dump: 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 5 monclient: authenticate success, global_id 4359 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _renew_subs 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84300a25e0 con 0x7f8430138680 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842cff9640 4 set_mon_vals no callback set 2024-09-17T16:28:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 client.4359.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _renew_subs 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8430126060 con 0x7f8430138680 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 librados: init done 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843006ebc0 con 0x7f8430138680 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f84240ad400 con 0x7f8430138680 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 0x7f841c0427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f84240eafe0 con 0x7f8430138680 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 10 client.4359.objecter ms_dispatch 0x7f8430000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 3 client.4359.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 3 client.4359.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f84177fe640 20 client.4359.objecter dump_active .. 0 homeless 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 0x7f841c0427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.516+0000 7f842e7fc640 10 monclient: get_auth_request con 0x7f841c040310 auth_method 0 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.520+0000 7f84177fe640 10 client.4359.objecter ms_handle_connect 0x7f841c040310 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.520+0000 7f842e7fc640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 0x7f841c0427d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8420005fd0 tx=0x7f8420009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.528+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8424013040 con 0x7f8430138680 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.528+0000 7f84177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.528+0000 7f84177fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.772+0000 7f84357ad640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:28:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.772+0000 7f84357ad640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:28:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.772+0000 7f84357ad640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:28:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.772+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f84300008e0 con 0x7f8430138680 2024-09-17T16:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84177fe640 1 -- 172.21.15.105:0/1113202301 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3200 (secure 0 0 0) 0x7f84240e9b20 con 0x7f8430138680 2024-09-17T16:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84177fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:28:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84177fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:37.782 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":22,"quorum":[1,2,3,4,5,6,8],"quorum_names":["f","b","g","c","h","d","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 10 librados: watch_flush enter 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 10 librados: watch_flush exit 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 20 client.4359.objecter shutdown clearing up homeless session... 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 10 client.4359.objecter successfully canceled tick 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 msgr2=0x7f841c0427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 0x7f841c0427d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8420005fd0 tx=0x7f8420009040 comp rx=0 tx=0).stop 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 10 monclient: shutdown 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 msgr2=0x7f8430138a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f842401b360 tx=0x7f842401b4f0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 shutdown_connections 2024-09-17T16:28:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f841c040310 0x7f841c0427d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8430122a70 0x7f8430122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8430124860 0x7f8430122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 --2- 172.21.15.105:0/1113202301 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8430138680 0x7f8430138a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 >> 172.21.15.105:0/1113202301 conn(0x7f8430068ba0 msgr2=0x7f84301c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 shutdown_connections 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 -- 172.21.15.105:0/1113202301 wait complete. 2024-09-17T16:28:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.776+0000 7f84357ad640 1 librados: shutdown 2024-09-17T16:28:37.802 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-17T16:28:37.802 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-17T16:28:37.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:28:37.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: get_monmap_and_config 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: build_initial_monmap 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: monmap: 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:37.916417+0000 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:37.916417+0000 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:37.919 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:37.920 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 Processor -- start 2024-09-17T16:28:37.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 -- start start 2024-09-17T16:28:37.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: init 2024-09-17T16:28:37.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient: _add_conns ranks=[0,5,1,6,7,9,2,4,8,3,10] 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2f080a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 0x7f2f0813eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-h con 0x7f2f080a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2f080a73d0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0806ebc0 con 0x7f2f080a68e0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f080a3690 con 0x7f2f080a73d0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f2f080a3fc0 con 0x7f2f080a4910 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0eaa3640 10 monclient(hunting): authenticate will time out at 1001.182828s 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0c83f640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f2f080a68e0 legacy=0x7f2f080a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 0x7f2f0813eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0d040640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2f080a73d0 0x7f2f08141040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:40154/0 (socket says 172.21.15.105:40154) 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f0d040640 1 -- 172.21.15.105:0/1196596743 learned_addr learned my addr 172.21.15.105:0/1196596743 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): get_auth_request con 0x7f2f080a4910 auth_method 0 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient(hunting): handle_auth_done global_id 4379 payload 274 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f077fe640 1 -- 172.21.15.105:0/1196596743 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899964668 0 0) 0x7f2f080a3690 con 0x7f2f080a73d0 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 1 -- 172.21.15.105:0/1196596743 >> v1:172.21.15.160:6789/0 conn(0x7f2f080a73d0 legacy=0x7f2f08141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 1 -- 172.21.15.105:0/1196596743 >> v1:172.21.15.105:6789/0 conn(0x7f2f080a68e0 legacy=0x7f2f080a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient: _finish_hunting 0 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 1 monclient: found mon.noname-h 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f07fff640 1 -- 172.21.15.105:0/1196596743 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f080a3970 con 0x7f2f080a4910 2024-09-17T16:28:37.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.912+0000 7f2f077fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:28:37.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f07fff640 1 --2- 172.21.15.105:0/1196596743 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 0x7f2f0813eaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097f0 tx=0x7f2ef001c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5b4f55787dacc297 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 1 -- 172.21.15.105:0/1196596743 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef00b0070 con 0x7f2f080a4910 2024-09-17T16:28:37.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:28:37.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: dump: 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.933 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 5 monclient: authenticate success, global_id 4379 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 1 -- 172.21.15.105:0/1196596743 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef00aa810 con 0x7f2f080a4910 2024-09-17T16:28:37.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 4 set_mon_vals no callback set 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 1 -- 172.21.15.105:0/1196596743 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef00aae30 con 0x7f2f080a4910 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f077fe640 10 monclient: dump: 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.935 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: shutdown 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/1196596743 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 msgr2=0x7f2f0813eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/1196596743 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 0x7f2f0813eaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097f0 tx=0x7f2ef001c3a0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/1196596743 shutdown_connections 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/1196596743 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2f080a4910 0x7f2f0813eaf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/1196596743 >> 172.21.15.105:0/1196596743 conn(0x7f2f08068ba0 msgr2=0x7f2f08068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/1196596743 shutdown_connections 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/1196596743 wait complete. 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: build_initial_monmap 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: monmap: 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:37.937 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 librados: starting msgr at 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 librados: starting objecter 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 Processor -- start 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- start start 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 librados: setting wanted keys 2024-09-17T16:28:37.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 librados: calling monclient init 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: init 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient: _add_conns ranks=[6,3,2,4,5,8,7,1,0] 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f080a4910 0x7f2f081d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-d con 0x7f2f080a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f080a68e0 0x7f2f081d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-g con 0x7f2f080a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 0x7f2f081d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient(hunting): picked mon.noname-b con 0x7f2f080a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.916+0000 7f2f0eaa3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08145310 con 0x7f2f080a73d0 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08145190 con 0x7f2f080a68e0 2024-09-17T16:28:37.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2f08145490 con 0x7f2f080a4910 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient(hunting): authenticate will time out at 1001.188019s 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f080a4910 0x7f2f081d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 0x7f2f081d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 0x7f2f081d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:44016/0 (socket says 172.21.15.105:44016) 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): get_auth_request con 0x7f2f080a4910 auth_method 0 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 -- 172.21.15.105:0/3766720171 learned_addr learned my addr 172.21.15.105:0/3766720171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): get_auth_request con 0x7f2f080a73d0 auth_method 0 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f080a68e0 0x7f2f081d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): get_auth_request con 0x7f2f080a68e0 auth_method 0 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient(hunting): handle_auth_done global_id 4497 payload 824 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f080a68e0 msgr2=0x7f2f081d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f080a68e0 0x7f2f081d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 msgr2=0x7f2f081d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 0x7f2f081d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient: _finish_hunting 0 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 monclient: found mon.noname-d 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f081290f0 con 0x7f2f080a4910 2024-09-17T16:28:37.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f080a73d0 0x7f2f081d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0c83f640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f080a4910 0x7f2f081d1390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800bdd0 tx=0x7f2ef8009f70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.?.objecter ms_handle_connect 0x7f2f080a4910 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef800ccf0 con 0x7f2f080a4910 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: dump: 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.943 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: _finish_auth 0 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: _check_auth_tickets 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8010070 con 0x7f2f080a4910 2024-09-17T16:28:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 5 monclient: authenticate success, global_id 4497 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient: _renew_subs 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0812a380 con 0x7f2f080a4910 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef8015d50 con 0x7f2f080a4910 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f06ffd640 4 set_mon_vals no callback set 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 client.4497.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: dump: 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.946 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient: _renew_subs 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0806de40 con 0x7f2f080a4910 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 librados: init done 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 0x7f2f08126590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08051e80 con 0x7f2f081261b0 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 0x7f2f08126590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient: get_auth_request con 0x7f2f081261b0 auth_method 0 2024-09-17T16:28:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 10 monclient(hunting): handle_auth_done global_id 4502 payload 824 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2ef8015470 con 0x7f2f080a4910 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f080a2010 con 0x7f2f081261b0 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f07fff640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 0x7f2f08126590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f08074a10 tx=0x7f2ef0019140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2ee803e550 0x7f2ee8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.4497.objecter ms_handle_connect 0x7f2f081261b0 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.4497.objecter resend_mon_ops 2024-09-17T16:28:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f2ef804fc40 con 0x7f2f080a4910 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.4497.objecter ms_dispatch 0x7f2f08000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 3 client.4497.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 3 client.4497.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 20 client.4497.objecter dump_active .. 0 homeless 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef00b0040 con 0x7f2f081261b0 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: dump: 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.949 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2ee803e550 0x7f2ee8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 10 monclient: get_auth_request con 0x7f2ee803e550 auth_method 0 2024-09-17T16:28:37.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2ef00036e0 con 0x7f2f081261b0 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 msgr2=0x7f2f08126590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 0x7f2f08126590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f08074a10 tx=0x7f2ef0019140 comp rx=0 tx=0).stop 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 msgr2=0x7f2f08126590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f081261b0 0x7f2f08126590 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f08074a10 tx=0x7f2ef0019140 comp rx=0 tx=0).stop 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f057fa640 10 client.4497.objecter ms_handle_connect 0x7f2ee803e550 2024-09-17T16:28:37.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.920+0000 7f2f0d040640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2ee803e550 0x7f2ee8040a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007c50 tx=0x7f2f00007ba0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 0x7f2f08122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2f080a6cc0 con 0x7f2f08126ad0 2024-09-17T16:28:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 0x7f2f08122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient: get_auth_request con 0x7f2f08126ad0 auth_method 0 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 10 monclient(hunting): handle_auth_done global_id 4511 payload 824 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 1 -- 172.21.15.105:0/3766720171 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f08127c00 con 0x7f2f08126ad0 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 client.4497.objecter ms_handle_connect 0x7f2f08126ad0 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f07fff640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 0x7f2f08122530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001aa50 tx=0x7f2ef00a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:37.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 client.4497.objecter resend_mon_ops 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ef00b0030 con 0x7f2f08126ad0 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 monclient: dump: 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:37.955 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9264 (secure 0 0 0) 0x7f2ef00b5b90 con 0x7f2f08126ad0 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 msgr2=0x7f2f08122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 0x7f2f08122530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001aa50 tx=0x7f2ef00a8000 comp rx=0 tx=0).stop 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 msgr2=0x7f2f08122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:37.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f057fa640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f08126ad0 0x7f2f08122530 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001aa50 tx=0x7f2ef00a8000 comp rx=0 tx=0).stop 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 65571, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:37.957 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:37.958 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:37.959 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:37.960 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:37.961 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.963 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:37.965 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:37.967 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:37.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:37.973 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:37.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.977 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:37.978 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:37.979 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:37.979 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:37.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:37.979 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:37.979 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:37.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 librados: watch_flush enter 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 librados: watch_flush exit 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 20 client.4497.objecter shutdown clearing up homeless session... 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 client.4497.objecter successfully canceled tick 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2ee803e550 msgr2=0x7f2ee8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2ee803e550 0x7f2ee8040a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007c50 tx=0x7f2f00007ba0 comp rx=0 tx=0).stop 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 10 monclient: shutdown 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f080a4910 msgr2=0x7f2f081d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:37.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 --2- 172.21.15.105:0/3766720171 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f080a4910 0x7f2f081d1390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800bdd0 tx=0x7f2ef8009f70 comp rx=0 tx=0).stop 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0d040640 1 -- 172.21.15.105:0/3766720171 reap_dead start 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0d040640 1 -- 172.21.15.105:0/3766720171 reap_dead start 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 shutdown_connections 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 >> 172.21.15.105:0/3766720171 conn(0x7f2f08068ba0 msgr2=0x7f2f0809fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 shutdown_connections 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 -- 172.21.15.105:0/3766720171 wait complete. 2024-09-17T16:28:37.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:37.928+0000 7f2f0eaa3640 1 librados: shutdown 2024-09-17T16:28:37.982 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:37.983 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-17T16:28:37.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:28:38.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: get_monmap_and_config 2024-09-17T16:28:38.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: monmap: 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:38.096421+0000 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:38.096421+0000 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.099 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.100 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.100 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:38.100 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:38.100 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 Processor -- start 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 -- start start 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: init 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient: _add_conns ranks=[6,2,1,7,3,10,0,5,8,4,9] 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f020c0a68e0 0x7f020c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-d con 0x7f020c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-b con 0x7f020c0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f020c1396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c06ebc0 con 0x7f020c1396a0 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f020c0a3fc0 con 0x7f020c0a4910 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f020c0a63c0 con 0x7f020c0a68e0 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0210f72640 10 monclient(hunting): authenticate will time out at 1001.362345s 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:44054/0 (socket says 172.21.15.105:44054) 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 -- 172.21.15.105:0/2352151687 learned_addr learned my addr 172.21.15.105:0/2352151687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): get_auth_request con 0x7f020c0a4910 auth_method 0 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f020c0a68e0 0x7f020c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): get_auth_request con 0x7f020c0a68e0 auth_method 0 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient(hunting): handle_auth_done global_id 4520 payload 274 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 -- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f020c0a68e0 msgr2=0x7f020c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f020c0a68e0 0x7f020c0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 -- 172.21.15.105:0/2352151687 >> v1:172.21.15.160:6789/0 conn(0x7f020c1396a0 legacy=0x7f020c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 monclient: found mon.noname-b 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:38.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.092+0000 7f0209d9b640 1 -- 172.21.15.105:0/2352151687 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0a3690 con 0x7f020c0a4910 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.096+0000 7f0209d9b640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0098a0 tx=0x7f01fc01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8c4f1471067cdd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 1 -- 172.21.15.105:0/2352151687 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0b0070 con 0x7f020c0a4910 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: dump: 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.106 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 1 -- 172.21.15.105:0/2352151687 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0aa810 con 0x7f020c0a4910 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 1 -- 172.21.15.105:0/2352151687 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0a9300 con 0x7f020c0a4910 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020959a640 10 monclient: dump: 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 5 monclient: authenticate success, global_id 4520 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 4 set_mon_vals no callback set 2024-09-17T16:28:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: shutdown 2024-09-17T16:28:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 msgr2=0x7f020c0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0098a0 tx=0x7f01fc01c440 comp rx=0 tx=0).stop 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/2352151687 shutdown_connections 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c0a4910 0x7f020c0a7200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- 172.21.15.105:0/2352151687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f020c0a68e0 0x7f020c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/2352151687 >> 172.21.15.105:0/2352151687 conn(0x7f020c068ba0 msgr2=0x7f020c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/2352151687 shutdown_connections 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/2352151687 wait complete. 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: monmap: 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 librados: starting msgr at 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 librados: starting objecter 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 Processor -- start 2024-09-17T16:28:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- start start 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 librados: setting wanted keys 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 librados: calling monclient init 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: init 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _add_conns ranks=[4,7,8,2,5,6,3,1,0] 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 0x7f020c1d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-c con 0x7f020c0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f020c0a68e0 0x7f020c1d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-i con 0x7f020c0a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f020c1396a0 0x7f020c1d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): picked mon.noname-e con 0x7f020c1396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f020c1444b0 con 0x7f020c0a4910 2024-09-17T16:28:38.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f020c144330 con 0x7f020c0a68e0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f020c144630 con 0x7f020c1396a0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient(hunting): authenticate will time out at 1001.370696s 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 0x7f020c1d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 0x7f020c1d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:54552/0 (socket says 172.21.15.105:54552) 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f020c1396a0 0x7f020c1d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 -- 172.21.15.105:0/1564981798 learned_addr learned my addr 172.21.15.105:0/1564981798 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): get_auth_request con 0x7f020c0a4910 auth_method 0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0209d9b640 1 -- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f020c0a68e0 msgr2=0x7f020c1d1a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:38.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): get_auth_request con 0x7f020c1396a0 auth_method 0 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0209d9b640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f020c0a68e0 0x7f020c1d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient(hunting): handle_auth_done global_id 4396 payload 824 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f020c1396a0 msgr2=0x7f020c1d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f020c1396a0 0x7f020c1d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f020c0a68e0 msgr2=0x7f020c1d1a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f020c0a68e0 0x7f020c1d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 monclient: found mon.noname-c 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0ab970 con 0x7f020c0a4910 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020ad9d640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f020c1396a0 0x7f020c1d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f020a59c640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 0x7f020c1d1540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020000e930 tx=0x7f020000edf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 client.?.objecter ms_handle_connect 0x7f020c0a4910 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0200017070 con 0x7f020c0a4910 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:28:38.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: dump: 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.122 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0200013470 con 0x7f020c0a4910 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 5 monclient: authenticate success, global_id 4396 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _renew_subs 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c140330 con 0x7f020c0a4910 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0200013a90 con 0x7f020c0a4910 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01faffd640 10 monclient: dump: 2024-09-17T16:28:38.123 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.124 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f01fbfff640 4 set_mon_vals no callback set 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 client.4396.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _renew_subs 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c13fc30 con 0x7f020c0a4910 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.100+0000 7f0210f72640 1 librados: init done 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0210f72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0210f72640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 0x7f020c13d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f020c051e80 con 0x7f020c140900 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 0x7f020c13d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient: get_auth_request con 0x7f020c140900 auth_method 0 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f020001a070 con 0x7f020c0a4910 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 10 monclient(hunting): handle_auth_done global_id 4405 payload 824 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f020c138950 con 0x7f020c140900 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01ec03e550 0x7f01ec040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f020000f040 con 0x7f020c0a4910 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 client.4396.objecter ms_dispatch 0x7f020c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 3 client.4396.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:38.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 3 client.4396.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f0209d9b640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 0x7f020c13d390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f020c074620 tx=0x7f01fc01ac80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 20 client.4396.objecter dump_active .. 0 homeless 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 client.4396.objecter ms_handle_connect 0x7f020c140900 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 client.4396.objecter resend_mon_ops 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f020ad9d640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01ec03e550 0x7f01ec040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f020ad9d640 10 monclient: get_auth_request con 0x7f01ec03e550 auth_method 0 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0ba070 con 0x7f020c140900 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 monclient: dump: 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.127 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01fc0a9300 con 0x7f020c140900 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 msgr2=0x7f020c13d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 0x7f020c13d390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f020c074620 tx=0x7f01fc01ac80 comp rx=0 tx=0).stop 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 msgr2=0x7f020c13d390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c140900 0x7f020c13d390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f01faffd640 10 client.4396.objecter ms_handle_connect 0x7f01ec03e550 2024-09-17T16:28:38.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.104+0000 7f020ad9d640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01ec03e550 0x7f01ec040a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f020c13f190 tx=0x7f0204009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0210f72640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0210f72640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 0x7f020c13dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0210f72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f020c13ee40 con 0x7f020c13d8d0 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 0x7f020c13dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient: get_auth_request con 0x7f020c13d8d0 auth_method 0 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.108+0000 7f0209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0209d9b640 10 monclient(hunting): handle_auth_done global_id 4414 payload 824 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0209d9b640 1 -- 172.21.15.105:0/1564981798 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f020c13e1f0 con 0x7f020c13d8d0 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0209d9b640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 0x7f020c13dcb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc0191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 client.4396.objecter ms_handle_connect 0x7f020c13d8d0 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 client.4396.objecter resend_mon_ops 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0ba050 con 0x7f020c13d8d0 2024-09-17T16:28:38.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 monclient: dump: 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.133 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9431 (secure 0 0 0) 0x7f01fc0bb040 con 0x7f020c13d8d0 2024-09-17T16:28:38.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 msgr2=0x7f020c13dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 0x7f020c13dcb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc0191e0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 msgr2=0x7f020c13dcb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f01faffd640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c13d8d0 0x7f020c13dcb0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0097c0 tx=0x7f01fc0191e0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:38.135 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 65751, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:38.136 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:38.137 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:38.138 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:38.139 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.140 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.142 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:38.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.148 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.153 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.155 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.156 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:28:38.157 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.157 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.157 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:38.157 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 10 librados: watch_flush enter 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 10 librados: watch_flush exit 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 20 client.4396.objecter shutdown clearing up homeless session... 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 10 client.4396.objecter successfully canceled tick 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01ec03e550 msgr2=0x7f01ec040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01ec03e550 0x7f01ec040a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f020c13f190 tx=0x7f0204009040 comp rx=0 tx=0).stop 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 10 monclient: shutdown 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 msgr2=0x7f020c1d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 --2- 172.21.15.105:0/1564981798 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f020c0a4910 0x7f020c1d1540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020000e930 tx=0x7f020000edf0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f020ad9d640 1 -- 172.21.15.105:0/1564981798 reap_dead start 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f020ad9d640 1 -- 172.21.15.105:0/1564981798 reap_dead start 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 shutdown_connections 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 >> 172.21.15.105:0/1564981798 conn(0x7f020c068ba0 msgr2=0x7f020c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 shutdown_connections 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 -- 172.21.15.105:0/1564981798 wait complete. 2024-09-17T16:28:38.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.112+0000 7f0210f72640 1 librados: shutdown 2024-09-17T16:28:38.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:38.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-17T16:28:38.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:28:38.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 10 monclient: get_monmap_and_config 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 10 monclient: monmap: 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:38.274585+0000 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:38.274585+0000 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.277 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:38.278 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 1 Processor -- start 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 1 -- start start 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.268+0000 7f38958e3640 10 monclient: init 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient: _add_conns ranks=[8,0,7,3,1,10,6,9,4,2,5] 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-e con 0x7f38900a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38900a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 0x7f3890139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-i con 0x7f38900a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389006ebc0 con 0x7f38900a4910 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f38900a3fc0 con 0x7f38900a7200 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f38900a63c0 con 0x7f38900a68e0 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient(hunting): authenticate will time out at 1001.540573s 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388e7fc640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f38900a4910 legacy=0x7f38901396a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388f7fe640 1 -- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 msgr2=0x7f3890139db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:28:38.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 0x7f3890139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41534/0 (socket says 172.21.15.105:41534) 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 -- 172.21.15.105:0/459439166 learned_addr learned my addr 172.21.15.105:0/459439166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): get_auth_request con 0x7f38900a68e0 auth_method 0 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient(hunting): handle_auth_done global_id 4472 payload 274 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 -- 172.21.15.105:0/459439166 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 msgr2=0x7f3890139db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 --2- 172.21.15.105:0/459439166 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 0x7f3890139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 -- 172.21.15.105:0/459439166 >> v1:172.21.15.105:6789/0 conn(0x7f38900a4910 legacy=0x7f38901396a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:38.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 monclient: found mon.noname-e 2024-09-17T16:28:38.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:38.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 -- 172.21.15.105:0/459439166 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38900a3690 con 0x7f38900a68e0 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388effd640 1 --2- 172.21.15.105:0/459439166 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f387801c430 comp rx=0 tx=0).ready entity=mon.8 client_cookie=95103820636b3f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 1 -- 172.21.15.105:0/459439166 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38780b0070 con 0x7f38900a68e0 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: dump: 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 5 monclient: authenticate success, global_id 4472 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 1 -- 172.21.15.105:0/459439166 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38780044a0 con 0x7f38900a68e0 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 4 set_mon_vals no callback set 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 1 -- 172.21.15.105:0/459439166 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3878003a90 con 0x7f38900a68e0 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f388dffb640 10 monclient: dump: 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 10 monclient: shutdown 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- 172.21.15.105:0/459439166 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 msgr2=0x7f38900a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 --2- 172.21.15.105:0/459439166 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f387801c430 comp rx=0 tx=0).stop 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- 172.21.15.105:0/459439166 shutdown_connections 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 --2- 172.21.15.105:0/459439166 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f38900a7200 0x7f3890139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 --2- 172.21.15.105:0/459439166 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f38900a68e0 0x7f38900a6cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.272+0000 7f38958e3640 1 -- 172.21.15.105:0/459439166 >> 172.21.15.105:0/459439166 conn(0x7f3890068ba0 msgr2=0x7f3890068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- 172.21.15.105:0/459439166 shutdown_connections 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- 172.21.15.105:0/459439166 wait complete. 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: monmap: 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:38.294 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 librados: starting msgr at 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 librados: starting objecter 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 Processor -- start 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- start start 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 librados: setting wanted keys 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 librados: calling monclient init 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: init 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _add_conns ranks=[6,3,0,2,5,4,8,1,7] 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 0x7f38901d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-d con 0x7f38900a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38900a68e0 0x7f38901d1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-g con 0x7f38900a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38900a7200 0x7f389013e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): picked mon.noname-a con 0x7f38900a7200 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f389013a2f0 con 0x7f38900a7200 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f389006e630 con 0x7f38900a68e0 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f389013a470 con 0x7f38900a4910 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): authenticate will time out at 1001.545544s 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388f7fe640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38900a7200 msgr2=0x7f389013e130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38900a7200 0x7f389013e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 0x7f38901d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 0x7f38901d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58058/0 (socket says 172.21.15.105:58058) 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 -- 172.21.15.105:0/3314075616 learned_addr learned my addr 172.21.15.105:0/3314075616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): get_auth_request con 0x7f38900a4910 auth_method 0 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38900a68e0 0x7f38901d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): get_auth_request con 0x7f38900a68e0 auth_method 0 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient(hunting): handle_auth_done global_id 4515 payload 824 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38900a68e0 msgr2=0x7f38901d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38900a68e0 0x7f38901d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38900a7200 msgr2=0x7f389013e130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38900a7200 0x7f389013e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 monclient: found mon.noname-d 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901238c0 con 0x7f38900a4910 2024-09-17T16:28:38.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388effd640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 0x7f38901d13f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f387801c260 tx=0x7f38780191d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 client.?.objecter ms_handle_connect 0x7f38900a4910 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38780bb070 con 0x7f38900a4910 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: dump: 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3878004110 con 0x7f38900a4910 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 5 monclient: authenticate success, global_id 4515 2024-09-17T16:28:38.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _renew_subs 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38900a26b0 con 0x7f38900a4910 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38780b0040 con 0x7f38900a4910 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388d7fa640 4 set_mon_vals no callback set 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 client.4515.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: dump: 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _renew_subs 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f389006de60 con 0x7f38900a4910 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 librados: init done 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 0x7f389013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f38901407f0 con 0x7f3890123260 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 0x7f389013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient: get_auth_request con 0x7f3890123260 auth_method 0 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f38780042c0 con 0x7f38900a4910 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 10 monclient(hunting): handle_auth_done global_id 4524 payload 824 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38901229f0 con 0x7f3890123260 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f387003e550 0x7f3870040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f38780e90f0 con 0x7f38900a4910 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 client.4515.objecter ms_dispatch 0x7f3890000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 3 client.4515.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 3 client.4515.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388e7fc640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 0x7f389013e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ed0 tx=0x7f3880007660 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 20 client.4515.objecter dump_active .. 0 homeless 2024-09-17T16:28:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 client.4515.objecter ms_handle_connect 0x7f3890123260 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 client.4515.objecter resend_mon_ops 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f388001a070 con 0x7f3890123260 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388f7fe640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f387003e550 0x7f3870040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f3887fff640 10 monclient: dump: 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.276+0000 7f388f7fe640 10 monclient: get_auth_request con 0x7f387003e550 auth_method 0 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f38800052e0 con 0x7f3890123260 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 msgr2=0x7f389013e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 0x7f389013e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ed0 tx=0x7f3880007660 comp rx=0 tx=0).stop 2024-09-17T16:28:38.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 msgr2=0x7f389013e670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3890123260 0x7f389013e670 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3880009ed0 tx=0x7f3880007660 comp rx=0 tx=0).stop 2024-09-17T16:28:38.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f3887fff640 10 client.4515.objecter ms_handle_connect 0x7f387003e550 2024-09-17T16:28:38.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.280+0000 7f388f7fe640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f387003e550 0x7f3870040a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38880038f0 tx=0x7f3888006c00 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 0x7f389013efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3890051e80 con 0x7f389013ebb0 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 0x7f389013efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient: get_auth_request con 0x7f389013ebb0 auth_method 0 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 10 monclient(hunting): handle_auth_done global_id 4533 payload 824 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 1 -- 172.21.15.105:0/3314075616 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f389013f500 con 0x7f389013ebb0 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 client.4515.objecter ms_handle_connect 0x7f389013ebb0 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388e7fc640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 0x7f389013efc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3880017000 tx=0x7f3880005d30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 client.4515.objecter resend_mon_ops 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f388001a050 con 0x7f389013ebb0 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 monclient: dump: 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.311 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.312 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9264 (secure 0 0 0) 0x7f388001b040 con 0x7f389013ebb0 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 msgr2=0x7f389013efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 0x7f389013efc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3880017000 tx=0x7f3880005d30 comp rx=0 tx=0).stop 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 msgr2=0x7f389013efc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f3887fff640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f389013ebb0 0x7f389013efc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 65923, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:28:38.314 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:38.317 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.318 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.321 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.323 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.325 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.326 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.328 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:38.330 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:38.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.334 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:38.335 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 librados: watch_flush enter 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 librados: watch_flush exit 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 20 client.4515.objecter shutdown clearing up homeless session... 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 client.4515.objecter successfully canceled tick 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f387003e550 msgr2=0x7f3870040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f387003e550 0x7f3870040a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38880038f0 tx=0x7f3888006c00 comp rx=0 tx=0).stop 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 10 monclient: shutdown 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 msgr2=0x7f38901d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 --2- 172.21.15.105:0/3314075616 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f38900a4910 0x7f38901d13f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f387801c260 tx=0x7f38780191d0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388f7fe640 1 -- 172.21.15.105:0/3314075616 reap_dead start 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f388f7fe640 1 -- 172.21.15.105:0/3314075616 reap_dead start 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 shutdown_connections 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 >> 172.21.15.105:0/3314075616 conn(0x7f3890068ba0 msgr2=0x7f389009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 shutdown_connections 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 -- 172.21.15.105:0/3314075616 wait complete. 2024-09-17T16:28:38.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.284+0000 7f38958e3640 1 librados: shutdown 2024-09-17T16:28:38.338 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:38.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-17T16:28:38.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:28:38.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: get_monmap_and_config 2024-09-17T16:28:38.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: monmap: 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:38.460542+0000 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:38.460542+0000 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.463 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:38.464 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 Processor -- start 2024-09-17T16:28:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 -- start start 2024-09-17T16:28:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: init 2024-09-17T16:28:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient: _add_conns ranks=[9,0,2,5,10,1,7,8,4,6,3] 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 0x7f5d780a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-a con 0x7f5d780a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5d780a6cc0 addr v1:172.21.15.105:6789/0 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-b con 0x7f5d7813eaf0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d780a7b80 con 0x7f5d780a6cc0 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5d780a2ea0 con 0x7f5d780a68e0 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5d78069ab0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7e9b7640 10 monclient(hunting): authenticate will time out at 1001.726811s 2024-09-17T16:28:38.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d6f7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f5d780a6cc0 legacy=0x7f5d780a70a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d77fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 msgr2=0x7f5d780a48a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d77fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 0x7f5d780a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:44064/0 (socket says 172.21.15.105:44064) 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3777351412 learned_addr learned my addr 172.21.15.105:0/3777351412 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): get_auth_request con 0x7f5d7813eaf0 auth_method 0 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_done global_id 4529 payload 274 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3777351412 >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 msgr2=0x7f5d780a48a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 --2- 172.21.15.105:0/3777351412 >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 0x7f5d780a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3777351412 >> v1:172.21.15.105:6789/0 conn(0x7f5d780a6cc0 legacy=0x7f5d780a70a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 monclient: found mon.noname-b 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:28:38.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.456+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3777351412 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780a3620 con 0x7f5d7813eaf0 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7cf54640 1 --2- 172.21.15.105:0/3777351412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009e50 tx=0x7f5d7001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e83113de3e8186 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 1 -- 172.21.15.105:0/3777351412 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d700090d0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: dump: 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.476 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 1 -- 172.21.15.105:0/3777351412 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d700a59c0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 1 -- 172.21.15.105:0/3777351412 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d700a5bb0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d777fe640 10 monclient: dump: 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.478 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.480 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 5 monclient: authenticate success, global_id 4529 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 4 set_mon_vals no callback set 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 10 monclient: shutdown 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3777351412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 msgr2=0x7f5d78140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3777351412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009e50 tx=0x7f5d7001c910 comp rx=0 tx=0).stop 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3777351412 shutdown_connections 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3777351412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d7813eaf0 0x7f5d78140ee0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3777351412 >> v2:172.21.15.105:3300/0 conn(0x7f5d780a68e0 0x7f5d780a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3777351412 >> 172.21.15.105:0/3777351412 conn(0x7f5d780a96e0 msgr2=0x7f5d78068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3777351412 shutdown_connections 2024-09-17T16:28:38.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.460+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3777351412 wait complete. 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: monmap: 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.482 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 librados: starting msgr at 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 librados: starting objecter 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 Processor -- start 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- start start 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 librados: setting wanted keys 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 librados: calling monclient init 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: init 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _add_conns ranks=[1,5,3,4,7,8,2,0,6] 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 0x7f5d7813c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-f con 0x7f5d780a6cc0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-h con 0x7f5d7813eaf0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 0x7f5d78137030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): picked mon.noname-g con 0x7f5d78136bb0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78143fd0 con 0x7f5d780a6cc0 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f5d78143e50 con 0x7f5d78136bb0 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5d78144150 con 0x7f5d7813eaf0 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): authenticate will time out at 1001.733359s 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 0x7f5d78137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 0x7f5d7813c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 0x7f5d78137030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:51614/0 (socket says 172.21.15.105:51614) 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 0x7f5d7813c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:50296/0 (socket says 172.21.15.105:50296) 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:33072/0 (socket says 172.21.15.105:33072) 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3620018372 learned_addr learned my addr 172.21.15.105:0/3620018372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): get_auth_request con 0x7f5d78136bb0 auth_method 0 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): get_auth_request con 0x7f5d7813eaf0 auth_method 0 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): get_auth_request con 0x7f5d780a6cc0 auth_method 0 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient(hunting): handle_auth_done global_id 4388 payload 824 2024-09-17T16:28:38.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 msgr2=0x7f5d78137030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 0x7f5d78137030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 msgr2=0x7f5d7813c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 0x7f5d7813c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 monclient: found mon.noname-h 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7813c3a0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d780a6cc0 0x7f5d7813c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7cf54640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d78136bb0 0x7f5d78137030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d6f7fe640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000b460 tx=0x7f5d6000b920 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 client.?.objecter ms_handle_connect 0x7f5d7813eaf0 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:38.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d60009240 con 0x7f5d7813eaf0 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: dump: 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.488 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60015070 con 0x7f5d7813eaf0 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d60003bd0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:38.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d76ffd640 4 set_mon_vals no callback set 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 10 monclient: dump: 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.490 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 5 monclient: authenticate success, global_id 4388 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _renew_subs 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7813d880 con 0x7f5d7813eaf0 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 client.4388.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _renew_subs 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7813c0c0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 librados: init done 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 0x7f5d78138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5d780a5680 con 0x7f5d780a96e0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 0x7f5d78138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient: get_auth_request con 0x7f5d780a96e0 auth_method 0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 10 monclient(hunting): handle_auth_done global_id 4481 payload 824 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5d6000f070 con 0x7f5d7813eaf0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.464+0000 7f5d77fff640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d780008e0 con 0x7f5d780a96e0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d77fff640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 0x7f5d78138630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68009f90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5d5803e550 0x7f5d58040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 client.4388.objecter ms_handle_connect 0x7f5d780a96e0 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 client.4388.objecter resend_mon_ops 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f5d6004efc0 con 0x7f5d7813eaf0 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 client.4388.objecter ms_dispatch 0x7f5d78000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 3 client.4388.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 3 client.4388.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 20 client.4388.objecter dump_active .. 0 homeless 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d68016070 con 0x7f5d780a96e0 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 monclient: dump: 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d7cf54640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5d5803e550 0x7f5d58040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d7cf54640 10 monclient: get_auth_request con 0x7f5d5803e550 auth_method 0 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5d6800ea90 con 0x7f5d780a96e0 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 msgr2=0x7f5d78138630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 0x7f5d78138630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68009f90 comp rx=0 tx=0).stop 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 msgr2=0x7f5d78138630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d780a96e0 0x7f5d78138630 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68009f90 comp rx=0 tx=0).stop 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d757fa640 10 client.4388.objecter ms_handle_connect 0x7f5d5803e550 2024-09-17T16:28:38.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.468+0000 7f5d7cf54640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5d5803e550 0x7f5d58040a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af040 tx=0x7f5d700a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d7e9b7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 0x7f5d78138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d7e9b7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5d78051e80 con 0x7f5d78138b70 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 0x7f5d78138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient: get_auth_request con 0x7f5d78138b70 auth_method 0 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 10 monclient(hunting): handle_auth_done global_id 4490 payload 824 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 1 -- 172.21.15.105:0/3620018372 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5d781394c0 con 0x7f5d78138b70 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 client.4388.objecter ms_handle_connect 0x7f5d78138b70 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 client.4388.objecter resend_mon_ops 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d77fff640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 0x7f5d78138f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68005980 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d68016040 con 0x7f5d78138b70 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 monclient: dump: 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.499 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9431 (secure 0 0 0) 0x7f5d680059f0 con 0x7f5d78138b70 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 msgr2=0x7f5d78138f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 0x7f5d78138f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68005980 comp rx=0 tx=0).stop 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 msgr2=0x7f5d78138f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.472+0000 7f5d757fa640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5d78138b70 0x7f5d78138f80 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005280 tx=0x7f5d68005980 comp rx=0 tx=0).stop 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 66115, 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:38.501 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.502 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.503 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:38.504 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.505 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:38.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.507 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.509 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.514 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:38.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.517 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:38.520 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:38.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:38.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:38.523 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 10 librados: watch_flush enter 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 10 librados: watch_flush exit 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 20 client.4388.objecter shutdown clearing up homeless session... 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 10 client.4388.objecter successfully canceled tick 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5d5803e550 msgr2=0x7f5d58040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5d5803e550 0x7f5d58040a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d700af040 tx=0x7f5d700a7070 comp rx=0 tx=0).stop 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 10 monclient: shutdown 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 msgr2=0x7f5d78136670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000b460 tx=0x7f5d6000b920 comp rx=0 tx=0).stop 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7cf54640 1 -- 172.21.15.105:0/3620018372 reap_dead start 2024-09-17T16:28:38.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 shutdown_connections 2024-09-17T16:28:38.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 --2- 172.21.15.105:0/3620018372 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5d7813eaf0 0x7f5d78136670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 >> 172.21.15.105:0/3620018372 conn(0x7f5d780a68e0 msgr2=0x7f5d780a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 shutdown_connections 2024-09-17T16:28:38.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 -- 172.21.15.105:0/3620018372 wait complete. 2024-09-17T16:28:38.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.476+0000 7f5d7e9b7640 1 librados: shutdown 2024-09-17T16:28:38.526 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:38.526 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-17T16:28:38.526 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:28:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: get_monmap_and_config 2024-09-17T16:28:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: monmap: 2024-09-17T16:28:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:38.640262+0000 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:38.640262+0000 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:38.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 Processor -- start 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 -- start start 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: init 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient: _add_conns ranks=[5,8,9,0,4,6,7,10,1,2,3] 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 0x7f8b980a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-h con 0x7f8b980a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 0x7f8b980a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-e con 0x7f8b980a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 0x7f8b98139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-a con 0x7f8b981396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8b980a3fc0 con 0x7f8b981396a0 2024-09-17T16:28:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8b980a63c0 con 0x7f8b980a68e0 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8b980a7740 con 0x7f8b980a4910 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9e292640 10 monclient(hunting): authenticate will time out at 1001.905941s 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9c82f640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 msgr2=0x7f8b98139ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b9c82f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 0x7f8b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 0x7f8b980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 0x7f8b980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 0x7f8b980a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:33076/0 (socket says 172.21.15.105:33076) 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 1 -- 172.21.15.105:0/2723906077 learned_addr learned my addr 172.21.15.105:0/2723906077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): get_auth_request con 0x7f8b980a68e0 auth_method 0 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): get_auth_request con 0x7f8b980a4910 auth_method 0 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient(hunting): handle_auth_done global_id 4499 payload 274 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 -- 172.21.15.105:0/2723906077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 msgr2=0x7f8b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:38.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 --2- 172.21.15.105:0/2723906077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 0x7f8b980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 -- 172.21.15.105:0/2723906077 >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 msgr2=0x7f8b98139ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 --2- 172.21.15.105:0/2723906077 >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 0x7f8b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 monclient: found mon.noname-e 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:28:38.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.636+0000 7f8b977fe640 1 -- 172.21.15.105:0/2723906077 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9806ebc0 con 0x7f8b980a4910 2024-09-17T16:28:38.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- 172.21.15.105:0/2723906077 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 0x7f8b980a7200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b800097c0 tx=0x7f8b8001c350 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c3b2e9c51f5abb2c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 1 -- 172.21.15.105:0/2723906077 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b800b0070 con 0x7f8b980a4910 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: dump: 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.657 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 5 monclient: authenticate success, global_id 4499 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 1 -- 172.21.15.105:0/2723906077 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b800aa810 con 0x7f8b980a4910 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 4 set_mon_vals no callback set 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 1 -- 172.21.15.105:0/2723906077 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b800aaa40 con 0x7f8b980a4910 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b96ffd640 10 monclient: dump: 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.659 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.660 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: shutdown 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- 172.21.15.105:0/2723906077 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 msgr2=0x7f8b980a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- 172.21.15.105:0/2723906077 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 0x7f8b980a7200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b800097c0 tx=0x7f8b8001c350 comp rx=0 tx=0).stop 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- 172.21.15.105:0/2723906077 shutdown_connections 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- 172.21.15.105:0/2723906077 >> v2:172.21.15.105:3300/0 conn(0x7f8b981396a0 0x7f8b98139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- 172.21.15.105:0/2723906077 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b980a4910 0x7f8b980a7200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- 172.21.15.105:0/2723906077 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b980a68e0 0x7f8b980a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- 172.21.15.105:0/2723906077 >> 172.21.15.105:0/2723906077 conn(0x7f8b98068ba0 msgr2=0x7f8b98068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- 172.21.15.105:0/2723906077 shutdown_connections 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- 172.21.15.105:0/2723906077 wait complete. 2024-09-17T16:28:38.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: monmap: 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.662 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 librados: starting msgr at 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 librados: starting objecter 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 Processor -- start 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- start start 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 librados: setting wanted keys 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 librados: calling monclient init 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: init 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient: _add_conns ranks=[0,6,3,5,8,1,7,2,4] 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b980a4910 0x7f8b98137a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-a con 0x7f8b980a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 0x7f8b98137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-d con 0x7f8b980a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b981396a0 0x7f8b98138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): picked mon.noname-g con 0x7f8b981396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9806e630 con 0x7f8b980a4910 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8b980a7af0 con 0x7f8b981396a0 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b97fff640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b980a4910 msgr2=0x7f8b98137a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8b98143320 con 0x7f8b980a68e0 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9e292640 10 monclient(hunting): authenticate will time out at 1001.911200s 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b980a4910 0x7f8b98137a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 0x7f8b98137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 0x7f8b98137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58080/0 (socket says 172.21.15.105:58080) 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 -- 172.21.15.105:0/1398777939 learned_addr learned my addr 172.21.15.105:0/1398777939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): get_auth_request con 0x7f8b980a68e0 auth_method 0 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b981396a0 0x7f8b98138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): get_auth_request con 0x7f8b981396a0 auth_method 0 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b9c82f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient(hunting): handle_auth_done global_id 4542 payload 824 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b981396a0 msgr2=0x7f8b98138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b981396a0 0x7f8b98138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b980a4910 msgr2=0x7f8b98137a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b980a4910 0x7f8b98137a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9813f740 con 0x7f8b980a68e0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b977fe640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 0x7f8b98137fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ae040 tx=0x7f8b800190f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b94ff9640 10 client.?.objecter ms_handle_connect 0x7f8b980a68e0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.640+0000 7f8b94ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b800b0040 con 0x7f8b980a68e0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: dump: 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.667 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 5 monclient: authenticate success, global_id 4542 2024-09-17T16:28:38.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 monclient: _renew_subs 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b800044b0 con 0x7f8b980a68e0 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b98123820 con 0x7f8b980a68e0 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b800aacb0 con 0x7f8b980a68e0 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b967fc640 4 set_mon_vals no callback set 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 client.4542.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: dump: 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.669 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 monclient: _renew_subs 2024-09-17T16:28:38.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b980aa090 con 0x7f8b980a68e0 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 1 librados: init done 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 0x7f8b980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98051e80 con 0x7f8b980a0a20 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8b80004040 con 0x7f8b980a68e0 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 0x7f8b980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient: get_auth_request con 0x7f8b980a0a20 auth_method 0 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b7803e550 0x7f8b78040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 10 monclient(hunting): handle_auth_done global_id 4501 payload 824 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7f8b800b9070 con 0x7f8b980a68e0 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 client.4542.objecter ms_dispatch 0x7f8b98000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 3 client.4542.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 3 client.4542.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b98139ae0 con 0x7f8b980a0a20 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 20 client.4542.objecter dump_active .. 0 homeless 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9c82f640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b7803e550 0x7f8b78040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9c82f640 10 monclient: get_auth_request con 0x7f8b7803e550 auth_method 0 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b97fff640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 0x7f8b980a0e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b8800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 client.4542.objecter ms_handle_connect 0x7f8b980a0a20 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 client.4542.objecter resend_mon_ops 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b88017070 con 0x7f8b980a0a20 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: dump: 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.673 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 client.4542.objecter ms_handle_connect 0x7f8b7803e550 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b9c82f640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b7803e550 0x7f8b78040a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b90003cc0 tx=0x7f8b9000d040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8b8800c980 con 0x7f8b980a0a20 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 msgr2=0x7f8b980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 0x7f8b980a0e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b8800ec90 comp rx=0 tx=0).stop 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 msgr2=0x7f8b980a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.644+0000 7f8b94ff9640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a0a20 0x7f8b980a0e00 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b8800ec90 comp rx=0 tx=0).stop 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 0x7f8b9813d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98122970 con 0x7f8b980a1340 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 0x7f8b9813d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient: get_auth_request con 0x7f8b980a1340 auth_method 0 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 10 monclient(hunting): handle_auth_done global_id 4510 payload 824 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 1 -- 172.21.15.105:0/1398777939 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8b98123d40 con 0x7f8b980a1340 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 client.4542.objecter ms_handle_connect 0x7f8b980a1340 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 client.4542.objecter resend_mon_ops 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b97fff640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 0x7f8b9813d6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b88004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b88017070 con 0x7f8b980a1340 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 monclient: dump: 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.679 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9431 (secure 0 0 0) 0x7f8b88004320 con 0x7f8b980a1340 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 msgr2=0x7f8b9813d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 0x7f8b9813d6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b88004740 comp rx=0 tx=0).stop 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 msgr2=0x7f8b9813d6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b94ff9640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b980a1340 0x7f8b9813d6a0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006c90 tx=0x7f8b88004740 comp rx=0 tx=0).stop 2024-09-17T16:28:38.680 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 66259, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:38.681 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.682 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.683 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.683 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.683 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.683 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.683 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.684 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:38.685 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:38.687 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.691 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.695 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.696 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.698 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.702 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.704 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:38.704 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:38.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 librados: watch_flush enter 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 librados: watch_flush exit 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 20 client.4542.objecter shutdown clearing up homeless session... 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 client.4542.objecter successfully canceled tick 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b7803e550 msgr2=0x7f8b78040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b7803e550 0x7f8b78040a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b90003cc0 tx=0x7f8b9000d040 comp rx=0 tx=0).stop 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 10 monclient: shutdown 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 msgr2=0x7f8b98137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 --2- 172.21.15.105:0/1398777939 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b980a68e0 0x7f8b98137fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ae040 tx=0x7f8b800190f0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9c82f640 1 -- 172.21.15.105:0/1398777939 reap_dead start 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9c82f640 1 -- 172.21.15.105:0/1398777939 reap_dead start 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 shutdown_connections 2024-09-17T16:28:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 >> 172.21.15.105:0/1398777939 conn(0x7f8b98068ba0 msgr2=0x7f8b980a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 shutdown_connections 2024-09-17T16:28:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 -- 172.21.15.105:0/1398777939 wait complete. 2024-09-17T16:28:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.652+0000 7f8b9e292640 1 librados: shutdown 2024-09-17T16:28:38.707 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:38.707 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-17T16:28:38.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:28:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: get_monmap_and_config 2024-09-17T16:28:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: monmap: 2024-09-17T16:28:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:38.820286+0000 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:38.820286+0000 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:38.824 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 Processor -- start 2024-09-17T16:28:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 -- start start 2024-09-17T16:28:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: init 2024-09-17T16:28:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient: _add_conns ranks=[3,6,10,2,7,8,5,0,4,9,1] 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c0a6a30 0x7ff58c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-g con 0x7ff58c0a6a30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-d con 0x7ff58c0a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 0x7ff58c139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-f con 0x7ff58c1397f0 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ff58c0a7c40 con 0x7ff58c1397f0 2024-09-17T16:28:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0a7890 con 0x7ff58c0a6a30 2024-09-17T16:28:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff58c0a4080 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff591ac6640 10 monclient(hunting): authenticate will time out at 1002.085999s 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58084/0 (socket says 172.21.15.105:58084) 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 -- 172.21.15.105:0/4161006369 learned_addr learned my addr 172.21.15.105:0/4161006369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): get_auth_request con 0x7ff58c0a4ab0 auth_method 0 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c0a6a30 0x7ff58c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): get_auth_request con 0x7ff58c0a6a30 auth_method 0 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 1 --2- 172.21.15.105:0/4161006369 >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 0x7ff58c139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 10 monclient(hunting): get_auth_request con 0x7ff58c1397f0 auth_method 0 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient(hunting): handle_auth_done global_id 4551 payload 274 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 -- 172.21.15.105:0/4161006369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c0a6a30 msgr2=0x7ff58c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c0a6a30 0x7ff58c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 -- 172.21.15.105:0/4161006369 >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 msgr2=0x7ff58c139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 --2- 172.21.15.105:0/4161006369 >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 0x7ff58c139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 monclient: found mon.noname-d 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58affd640 1 -- 172.21.15.105:0/4161006369 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0a37e0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.816+0000 7ff58bfff640 1 --2- 172.21.15.105:0/4161006369 >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 0x7ff58c139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:28:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58affd640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff580009870 tx=0x7ff58001c480 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f1f1787f00799c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 1 -- 172.21.15.105:0/4161006369 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5800b1070 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: dump: 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 1 -- 172.21.15.105:0/4161006369 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5800a8050 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 1 -- 172.21.15.105:0/4161006369 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5800abcc0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 5 monclient: authenticate success, global_id 4551 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 4 set_mon_vals no callback set 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58a7fc640 10 monclient: dump: 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: shutdown 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/4161006369 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 msgr2=0x7ff58c0a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff580009870 tx=0x7ff58001c480 comp rx=0 tx=0).stop 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/4161006369 shutdown_connections 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- 172.21.15.105:0/4161006369 >> v2:172.21.15.160:3300/0 conn(0x7ff58c1397f0 0x7ff58c139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c0a7350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- 172.21.15.105:0/4161006369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c0a6a30 0x7ff58c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/4161006369 >> 172.21.15.105:0/4161006369 conn(0x7ff58c0a8d00 msgr2=0x7ff58c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/4161006369 shutdown_connections 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/4161006369 wait complete. 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: build_initial_monmap 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: monmap: 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:38.845 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 librados: starting msgr at 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 librados: starting objecter 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 Processor -- start 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- start start 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 librados: setting wanted keys 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 librados: calling monclient init 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: init 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _add_conns ranks=[6,0,2,8,5,4,3,7,1] 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c138180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-d con 0x7ff58c0a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff58c0a6a30 0x7ff58c1386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-a con 0x7ff58c0a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 0x7ff58c13d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): picked mon.noname-b con 0x7ff58c1397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06e770 con 0x7ff58c0a6a30 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c13a5a0 con 0x7ff58c1397f0 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff58c143260 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient(hunting): authenticate will time out at 1002.090983s 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58affd640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff58c0a6a30 msgr2=0x7ff58c1386c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff58c0a6a30 0x7ff58c1386c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c138180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 0x7ff58c13d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c138180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58086/0 (socket says 172.21.15.105:58086) 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 0x7ff58c13d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:44074/0 (socket says 172.21.15.105:44074) 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 -- 172.21.15.105:0/3312054178 learned_addr learned my addr 172.21.15.105:0/3312054178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): get_auth_request con 0x7ff58c0a4ab0 auth_method 0 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): get_auth_request con 0x7ff58c1397f0 auth_method 0 2024-09-17T16:28:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient(hunting): handle_auth_done global_id 4560 payload 824 2024-09-17T16:28:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 msgr2=0x7ff58c13d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 0x7ff58c13d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff58c0a6a30 msgr2=0x7ff58c1386c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff58c0a6a30 0x7ff58c1386c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 monclient: found mon.noname-d 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c123cc0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58bfff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff58c1397f0 0x7ff58c13d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff58b7fe640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c138180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9b0 tx=0x7ff57c00ee70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 client.?.objecter ms_handle_connect 0x7ff58c0a4ab0 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff57c017070 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: dump: 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.852 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.853 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: _finish_auth 0 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: _check_auth_tickets 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c00ce10 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:38.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff57c0138e0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 5 monclient: authenticate success, global_id 4560 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _renew_subs 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c123fa0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff589ffb640 4 set_mon_vals no callback set 2024-09-17T16:28:38.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff573fff640 10 monclient: dump: 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.857 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 client.4560.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _renew_subs 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c0abdf0 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.820+0000 7ff591ac6640 1 librados: init done 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff591ac6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff57c01a070 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff591ac6640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 0x7ff58c13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c051e80 con 0x7ff58c13dc90 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 0x7ff58c13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient: get_auth_request con 0x7ff58c13dc90 auth_method 0 2024-09-17T16:28:38.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff57803e580 0x7ff578040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7ff57c00f040 con 0x7ff58c0a4ab0 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 client.4560.objecter ms_dispatch 0x7ff58c000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 3 client.4560.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 3 client.4560.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 20 client.4560.objecter dump_active .. 0 homeless 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58bfff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff57803e580 0x7ff578040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58bfff640 10 monclient: get_auth_request con 0x7ff57803e580 auth_method 0 2024-09-17T16:28:38.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 10 monclient(hunting): handle_auth_done global_id 4413 payload 824 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff58c13fb80 con 0x7ff58c13dc90 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 client.4560.objecter ms_handle_connect 0x7ff58c13dc90 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 client.4560.objecter resend_mon_ops 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58affd640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 0x7ff58c13e070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06eab0 tx=0x7ff580019220 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5800b1040 con 0x7ff58c13dc90 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 monclient: dump: 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.864 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 client.4560.objecter ms_handle_connect 0x7ff57803e580 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff58bfff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff57803e580 0x7ff578040a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff58c140550 tx=0x7ff584009700 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff5800a8040 con 0x7ff58c13dc90 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:38.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 msgr2=0x7ff58c13e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 0x7ff58c13e070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff58c06eab0 tx=0x7ff580019220 comp rx=0 tx=0).stop 2024-09-17T16:28:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 msgr2=0x7ff58c13e070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.824+0000 7ff573fff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13dc90 0x7ff58c13e070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 0x7ff58c0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c06a200 con 0x7ff58c13e5b0 2024-09-17T16:28:38.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 0x7ff58c0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient: get_auth_request con 0x7ff58c13e5b0 auth_method 0 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 10 monclient(hunting): handle_auth_done global_id 4422 payload 824 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 1 -- 172.21.15.105:0/3312054178 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff58c139c50 con 0x7ff58c13e5b0 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 client.4560.objecter ms_handle_connect 0x7ff58c13e5b0 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58affd640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 0x7ff58c0a0a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5800097c0 tx=0x7ff580009240 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 client.4560.objecter resend_mon_ops 2024-09-17T16:28:38.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff580003ca0 con 0x7ff58c13e5b0 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 monclient: dump: 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:38.870 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9433 (secure 0 0 0) 0x7ff580003830 con 0x7ff58c13e5b0 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 msgr2=0x7ff58c0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 0x7ff58c0a0a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5800097c0 tx=0x7ff580009240 comp rx=0 tx=0).stop 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 msgr2=0x7ff58c0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:38.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff573fff640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff58c13e5b0 0x7ff58c0a0a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 66463, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:38.872 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.873 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.874 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.875 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:38.876 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:38.877 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:38.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.881 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.883 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.886 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.886 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:38.886 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:38.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.888 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:38.889 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.893 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:38.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:38.893 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.894 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:38.895 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:38.896 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:38.898 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:38.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:38.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.901 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.902 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:38.903 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 librados: watch_flush enter 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 librados: watch_flush exit 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 20 client.4560.objecter shutdown clearing up homeless session... 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 client.4560.objecter successfully canceled tick 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff57803e580 msgr2=0x7ff578040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff57803e580 0x7ff578040a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff58c140550 tx=0x7ff584009700 comp rx=0 tx=0).stop 2024-09-17T16:28:38.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 10 monclient: shutdown 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 msgr2=0x7ff58c138180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 --2- 172.21.15.105:0/3312054178 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff58c0a4ab0 0x7ff58c138180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9b0 tx=0x7ff57c00ee70 comp rx=0 tx=0).stop 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58bfff640 1 -- 172.21.15.105:0/3312054178 reap_dead start 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff58bfff640 1 -- 172.21.15.105:0/3312054178 reap_dead start 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 shutdown_connections 2024-09-17T16:28:38.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 >> 172.21.15.105:0/3312054178 conn(0x7ff58c0a8d00 msgr2=0x7ff58c0a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:38.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 shutdown_connections 2024-09-17T16:28:38.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 -- 172.21.15.105:0/3312054178 wait complete. 2024-09-17T16:28:38.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:38.832+0000 7ff591ac6640 1 librados: shutdown 2024-09-17T16:28:38.908 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:38.909 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-17T16:28:38.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:28:39.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: get_monmap_and_config 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: build_initial_monmap 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: monmap: 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:28:39.032479+0000 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:28:39.032479+0000 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:28:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:28:39.036 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:28:39.037 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 Processor -- start 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 -- start start 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: init 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient: _add_conns ranks=[1,10,5,0,8,9,7,2,3,6,4] 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe5c00a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:28:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-f con 0x7fe5c00a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:28:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 0x7fe5c0141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-h con 0x7fe5c00a73d0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c006ebc0 con 0x7fe5c00a68e0 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c00a3fc0 con 0x7fe5c00a4910 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5c00a63c0 con 0x7fe5c00a73d0 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c82b8640 10 monclient(hunting): authenticate will time out at 1002.298692s 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6054640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe5c00a68e0 0x7fe5c00a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:40164/0 (socket says 172.21.15.105:40164) 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:50342/0 (socket says 172.21.15.105:50342) 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6054640 1 -- 172.21.15.105:0/1228745893 learned_addr learned my addr 172.21.15.105:0/1228745893 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:28:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): get_auth_request con 0x7fe5c00a4910 auth_method 0 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 --2- 172.21.15.105:0/1228745893 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 0x7fe5c0141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): get_auth_request con 0x7fe5c00a73d0 auth_method 0 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 1 -- 172.21.15.105:0/1228745893 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138948048 0 0) 0x7fe5c006ebc0 con 0x7fe5c00a68e0 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 10 monclient(hunting): _negotiate 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 10 monclient(hunting): my global_id is 4537 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5052640 1 -- 172.21.15.105:0/1228745893 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5ac003530 con 0x7fe5c00a68e0 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient(hunting): handle_auth_done global_id 4406 payload 274 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 -- 172.21.15.105:0/1228745893 >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 msgr2=0x7fe5c013eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 --2- 172.21.15.105:0/1228745893 >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 -- 172.21.15.105:0/1228745893 >> v1:172.21.15.160:6789/0 conn(0x7fe5c00a68e0 legacy=0x7fe5c00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient: _finish_hunting 0 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 monclient: found mon.noname-h 2024-09-17T16:28:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c6855640 1 -- 172.21.15.105:0/1228745893 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c00a3690 con 0x7fe5c00a73d0 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.028+0000 7fe5c5853640 1 --2- 172.21.15.105:0/1228745893 >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c6855640 1 --2- 172.21.15.105:0/1228745893 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 0x7fe5c0141040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009d90 tx=0x7fe5b801c800 comp rx=0 tx=0).ready entity=mon.5 client_cookie=312c4337dd5637 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 1 -- 172.21.15.105:0/1228745893 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5b80b2070 con 0x7fe5c00a73d0 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: dump: 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.053 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: _finish_auth 0 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: _check_auth_tickets 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 1 -- 172.21.15.105:0/1228745893 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b80a5ac0 con 0x7fe5c00a73d0 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 5 monclient: authenticate success, global_id 4406 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: get_monmap_and_config success 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 1 -- 172.21.15.105:0/1228745893 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5b80a5cb0 con 0x7fe5c00a73d0 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 4 set_mon_vals no callback set 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c5052640 10 monclient: dump: 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: shutdown 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- 172.21.15.105:0/1228745893 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 msgr2=0x7fe5c0141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/1228745893 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 0x7fe5c0141040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009d90 tx=0x7fe5b801c800 comp rx=0 tx=0).stop 2024-09-17T16:28:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- 172.21.15.105:0/1228745893 shutdown_connections 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/1228745893 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c00a73d0 0x7fe5c0141040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/1228745893 >> v2:172.21.15.160:3300/0 conn(0x7fe5c00a4910 0x7fe5c013eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- 172.21.15.105:0/1228745893 >> 172.21.15.105:0/1228745893 conn(0x7fe5c0068ba0 msgr2=0x7fe5c0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- 172.21.15.105:0/1228745893 shutdown_connections 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- 172.21.15.105:0/1228745893 wait complete. 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: build_initial_monmap 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: monmap: 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:28:39.058 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 librados: starting msgr at 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 librados: starting objecter 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 Processor -- start 2024-09-17T16:28:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- start start 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 librados: setting wanted keys 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 librados: calling monclient init 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: init 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: _reopen_session rank -1 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient: _add_conns ranks=[6,8,0,1,3,4,5,2,7] 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-d con 0x7fe5c00a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe5c00a68e0 0x7fe5c0126830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-e con 0x7fe5c00a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c00a73d0 0x7fe5c0126d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): picked mon.noname-a con 0x7fe5c00a73d0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c0143f20 con 0x7fe5c00a73d0 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe5c0143da0 con 0x7fe5c00a4910 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe5c01440a0 con 0x7fe5c00a68e0 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): _renew_subs 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c82b8640 10 monclient(hunting): authenticate will time out at 1002.303884s 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c6855640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c00a73d0 msgr2=0x7fe5c0126d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.032+0000 7fe5c6855640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c00a73d0 0x7fe5c0126d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe5c00a68e0 0x7fe5c0126830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58096/0 (socket says 172.21.15.105:58096) 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 -- 172.21.15.105:0/186512334 learned_addr learned my addr 172.21.15.105:0/186512334 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:28:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): get_auth_request con 0x7fe5c00a4910 auth_method 0 2024-09-17T16:28:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): get_auth_request con 0x7fe5c00a68e0 auth_method 0 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient(hunting): handle_auth_done global_id 4569 payload 824 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe5c00a68e0 msgr2=0x7fe5c0126830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe5c00a68e0 0x7fe5c0126830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c00a73d0 msgr2=0x7fe5c0126d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5c00a73d0 0x7fe5c0126d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient: _finish_hunting 0 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 monclient: found mon.noname-d 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:28:39.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c0129200 con 0x7fe5c00a4910 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6054640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c4d0 tx=0x7fe5bc00c990 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.?.objecter ms_handle_connect 0x7fe5c00a4910 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5bc017070 con 0x7fe5c00a4910 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: dump: 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:28:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 5 monclient: authenticate success, global_id 4569 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc007070 con 0x7fe5c00a4910 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 monclient: _renew_subs 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c012a570 con 0x7fe5c00a4910 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5bc0076d0 con 0x7fe5c00a4910 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c4851640 4 set_mon_vals no callback set 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 client.4569.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: dump: 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 monclient: _renew_subs 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c012a070 con 0x7fe5c00a4910 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 1 librados: init done 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 0x7fe5c01d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5c0129cf0 con 0x7fe5c01d0fb0 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe5bc007260 con 0x7fe5c00a4910 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 0x7fe5c01d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient: get_auth_request con 0x7fe5c01d0fb0 auth_method 0 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5a403e550 0x7fe5a4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x7fe5bc04e930 con 0x7fe5c00a4910 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.4569.objecter ms_dispatch 0x7fe5c0000ba0 osd_map(20..20 src has 1..20) 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 3 client.4569.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 3 client.4569.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 10 monclient(hunting): handle_auth_done global_id 4415 payload 824 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5c013fe10 con 0x7fe5c01d0fb0 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 20 client.4569.objecter dump_active .. 0 homeless 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6855640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5a403e550 0x7fe5a4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6855640 10 monclient: get_auth_request con 0x7fe5a403e550 auth_method 0 2024-09-17T16:28:39.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.4569.objecter ms_handle_connect 0x7fe5c01d0fb0 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.4569.objecter resend_mon_ops 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c5853640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 0x7fe5c01d1390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00119a0 tx=0x7fe5b0011e60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5b000fc20 con 0x7fe5c01d0fb0 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: dump: 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.071 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 client.4569.objecter ms_handle_connect 0x7fe5a403e550 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5c6855640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5a403e550 0x7fe5a4040a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80af000 tx=0x7fe5b80a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe5b000fc20 con 0x7fe5c01d0fb0 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 msgr2=0x7fe5c01d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 0x7fe5c01d1390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00119a0 tx=0x7fe5b0011e60 comp rx=0 tx=0).stop 2024-09-17T16:28:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 msgr2=0x7fe5c01d1390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:39.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.036+0000 7fe5b6ffd640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d0fb0 0x7fe5c01d1390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 0x7fe5c0122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5c0051e80 con 0x7fe5c01d18d0 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 0x7fe5c0122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient: get_auth_request con 0x7fe5c01d18d0 auth_method 0 2024-09-17T16:28:39.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 10 monclient(hunting): handle_auth_done global_id 4424 payload 824 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 1 -- 172.21.15.105:0/186512334 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe5c006de80 con 0x7fe5c01d18d0 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c5853640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 0x7fe5c0122530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00119a0 tx=0x7fe5b00046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 client.4569.objecter ms_handle_connect 0x7fe5c01d18d0 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 client.4569.objecter resend_mon_ops 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5b000fea0 con 0x7fe5c01d18d0 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:28:39.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 monclient: dump: 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:28:39.077 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9431 (secure 0 0 0) 0x7fe5b001d400 con 0x7fe5c01d18d0 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 msgr2=0x7fe5c0122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 0x7fe5c0122530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00119a0 tx=0x7fe5b00046d0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 msgr2=0x7fe5c0122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5b6ffd640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5c01d18d0 0x7fe5c0122530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:28:39.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 22, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 66651, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:28:39.079 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:39.080 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:39.081 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:39.082 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:28:39.083 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:28:39.084 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.085 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:28:39.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.086 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:28:39.086 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:28:39.086 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.090 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:28:39.091 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:28:39.091 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.091 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:28:39.095 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.096 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:28:39.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.099 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:28:39.100 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:28:39.102 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:28:39.103 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:28:39.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:28:39.105 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 librados: watch_flush enter 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 librados: watch_flush exit 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 20 client.4569.objecter shutdown clearing up homeless session... 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 client.4569.objecter successfully canceled tick 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5a403e550 msgr2=0x7fe5a4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5a403e550 0x7fe5a4040a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80af000 tx=0x7fe5b80a7070 comp rx=0 tx=0).stop 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 10 monclient: shutdown 2024-09-17T16:28:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 msgr2=0x7fe5c01262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c4d0 tx=0x7fe5bc00c990 comp rx=0 tx=0).stop 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c6855640 1 -- 172.21.15.105:0/186512334 reap_dead start 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 shutdown_connections 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 --2- 172.21.15.105:0/186512334 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe5c00a4910 0x7fe5c01262f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 >> 172.21.15.105:0/186512334 conn(0x7fe5c0068ba0 msgr2=0x7fe5c009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 shutdown_connections 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 -- 172.21.15.105:0/186512334 wait complete. 2024-09-17T16:28:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:28:39.044+0000 7fe5c82b8640 1 librados: shutdown 2024-09-17T16:28:39.108 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:28:39.109 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:28:59.653 INFO:tasks.workunit.client.0.smithi160.stderr:Note: switching to '1889a165b5128787ec693450a3d1c5df5d6dc203'. 2024-09-17T16:28:59.653 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.653 INFO:tasks.workunit.client.0.smithi160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:state without impacting any branches by switching back to a branch. 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: git switch -c 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:Or undo this operation with: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: git switch - 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-17T16:28:59.654 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-09-17T16:28:59.655 INFO:tasks.workunit.client.0.smithi160.stderr:HEAD is now at 1889a165b51 Merge branch 'wip-matanb-cleanup-snapmap' of https://github.com/Matan-B/ceph into wip-bharath10-testing-2024-09-17-1348 2024-09-17T16:28:59.659 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-17T16:28:59.666 INFO:tasks.workunit.client.0.smithi160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-17T16:28:59.669 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-17T16:28:59.669 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-17T16:29:00.389 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-17T16:29:00.452 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-17T16:29:00.499 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-17T16:29:00.500 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-17T16:29:00.500 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-17T16:29:00.541 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-17T16:29:00.545 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-09-17T16:29:00.545 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-17T16:29:00.593 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-17T16:29:00.594 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-17T16:29:00.594 DEBUG:teuthology.orchestra.run.smithi160:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1889a165b5128787ec693450a3d1c5df5d6dc203 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ parallel=1 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' '' = --serial ']' 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ crimson=0 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' '' = --crimson ']' 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ color= 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -t 1 ']' 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-17T16:29:00.647 INFO:tasks.workunit.client.0.smithi160.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-17T16:29:00.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ declare -A pids 2024-09-17T16:29:00.648 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_aio 2024-09-17T16:29:00.649 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_aio' 2024-09-17T16:29:00.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_aio 2024-09-17T16:29:00.649 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.650 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_aio 2024-09-17T16:29:00.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29343 2024-09-17T16:29:00.651 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_aio on pid 29343' 2024-09-17T16:29:00.651 INFO:tasks.workunit.client.0.smithi160.stdout:test api_aio on pid 29343 2024-09-17T16:29:00.652 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29343 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_aio_pp 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_aio_pp' 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_aio_pp 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.653 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_aio: /' 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_aio_pp 2024-09-17T16:29:00.654 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29351 2024-09-17T16:29:00.655 INFO:tasks.workunit.client.0.smithi160.stdout:test api_aio_pp on pid 29351 2024-09-17T16:29:00.656 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_aio_pp on pid 29351' 2024-09-17T16:29:00.656 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29351 2024-09-17T16:29:00.656 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.656 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_io 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_io' 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_io 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.657 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_io 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29359 2024-09-17T16:29:00.658 INFO:tasks.workunit.client.0.smithi160.stdout:test api_io on pid 29359 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_io on pid 29359' 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29359 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_io_pp 2024-09-17T16:29:00.660 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_io_pp' 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_io_pp 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_io.log 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_io: /' 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_io_pp 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29367 2024-09-17T16:29:00.661 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_io_pp on pid 29367' 2024-09-17T16:29:00.662 INFO:tasks.workunit.client.0.smithi160.stdout:test api_io_pp on pid 29367 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29367 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_asio 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_asio' 2024-09-17T16:29:00.663 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_asio 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_asio 2024-09-17T16:29:00.664 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29375 2024-09-17T16:29:00.665 INFO:tasks.workunit.client.0.smithi160.stdout:test api_asio on pid 29375 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_asio on pid 29375' 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29375 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-17T16:29:00.666 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_list 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_list' 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_list 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_asio: /' 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_list 2024-09-17T16:29:00.667 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29383 2024-09-17T16:29:00.668 INFO:tasks.workunit.client.0.smithi160.stdout:test api_list on pid 29383 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_list on pid 29383' 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29383 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_lock 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_lock' 2024-09-17T16:29:00.669 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_lock 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_list.log 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_list: /' 2024-09-17T16:29:00.670 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.671 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_lock 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stdout:test api_lock on pid 29391 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29391 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_lock on pid 29391' 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29391 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.672 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.673 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-17T16:29:00.673 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.673 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.673 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-09-17T16:29:00.674 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-17T16:29:00.686 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_lock_pp 2024-09-17T16:29:00.687 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_lock: /' 2024-09-17T16:29:00.687 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_lock_pp' 2024-09-17T16:29:00.687 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.689 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_lock_pp 2024-09-17T16:29:00.690 INFO:tasks.workunit.client.0.smithi160.stdout:test api_lock_pp on pid 29407 2024-09-17T16:29:00.690 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_lock_pp 2024-09-17T16:29:00.690 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29407 2024-09-17T16:29:00.690 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_lock_pp on pid 29407' 2024-09-17T16:29:00.690 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29407 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_misc 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_misc' 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_misc 2024-09-17T16:29:00.691 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-17T16:29:00.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.693 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-17T16:29:00.694 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.694 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-09-17T16:29:00.694 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-17T16:29:00.696 INFO:tasks.workunit.client.0.smithi160.stdout:test api_misc on pid 29430 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_misc 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29430 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_misc on pid 29430' 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29430 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-17T16:29:00.697 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_misc_pp 2024-09-17T16:29:00.698 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_misc_pp' 2024-09-17T16:29:00.699 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.699 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.699 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.699 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.699 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.701 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_misc: /' 2024-09-17T16:29:00.701 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_misc_pp 2024-09-17T16:29:00.702 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_misc_pp 2024-09-17T16:29:00.702 INFO:tasks.workunit.client.0.smithi160.stdout:test api_misc_pp on pid 29459 2024-09-17T16:29:00.702 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29459 2024-09-17T16:29:00.702 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_misc_pp on pid 29459' 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29459 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_tier_pp 2024-09-17T16:29:00.703 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_tier_pp' 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-17T16:29:00.704 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-17T16:29:00.705 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-09-17T16:29:00.705 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-17T16:29:00.705 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.707 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_tier_pp 2024-09-17T16:29:00.708 INFO:tasks.workunit.client.0.smithi160.stdout:test api_tier_pp on pid 29481 2024-09-17T16:29:00.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_tier_pp 2024-09-17T16:29:00.708 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29481 2024-09-17T16:29:00.709 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_tier_pp on pid 29481' 2024-09-17T16:29:00.709 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29481 2024-09-17T16:29:00.709 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.709 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.713 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-17T16:29:00.713 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_pool 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_pool' 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] Global test environment set-up. 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: get_monmap_and_config 2024-09-17T16:29:00.714 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: build_initial_monmap 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: monmap: 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: epoch 0 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: last_changed 0.000000 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: created 0.000000 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: min_mon_release 0 (unknown) 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: election_strategy: 1 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:29:00.715 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 Processor -- start 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 -- start start 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: init 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: _reopen_session rank -1 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient: _add_conns ranks=[2,3,5,7,1,0,8,6,4] 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5608cba96a00 0x5608cba956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-b con 0x5608cba96a00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5608cba96de0 0x5608cbb52470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-g con 0x5608cba96de0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:29:00.716 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-h con 0x5608cba971c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x5608cbac7170 con 0x5608cba96a00 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x5608cb9ea9f0 con 0x5608cba96de0 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x5608cbabfb40 con 0x5608cba971c0 2024-09-17T16:29:00.717 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): _renew_subs 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.708+0000 7fae627bd880 10 monclient(hunting): authenticate will time out at 1018.979700s 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.160:56476/0 (socket says 172.21.15.160:56476) 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 -- 172.21.15.160:0/1129132689 learned_addr learned my addr 172.21.15.160:0/1129132689 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): get_auth_request con 0x5608cba971c0 auth_method 0 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5608cba96de0 0x5608cbb52470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): get_auth_request con 0x5608cba96de0 auth_method 0 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:29:00.718 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient(hunting): handle_auth_done global_id 4451 payload 274 2024-09-17T16:29:00.719 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 -- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5608cba96de0 msgr2=0x5608cbb52470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_pool 2024-09-17T16:29:00.720 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5608cba96de0 0x5608cbb52470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 -- 172.21.15.160:0/1129132689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5608cba96a00 msgr2=0x5608cba956c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5608cba96a00 0x5608cba956c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient: _finish_hunting 0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 monclient: found mon.noname-h 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 -- 172.21.15.160:0/1129132689 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608cb9e9a10 con 0x5608cba971c0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.712+0000 7fae60ac8640 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae5800a4b0 tx=0x7fae5800a970 comp rx=0 tx=0).ready entity=mon.5 client_cookie=70c840ff132702a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 1 -- 172.21.15.160:0/1129132689 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae58015070 con 0x5608cba971c0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:29:00.721 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: dump: 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: epoch 1 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: min_mon_release 19 (squid) 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: election_strategy: 3 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:29:00.722 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: _finish_auth 0 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: _check_auth_tickets 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 5 monclient: authenticate success, global_id 4451 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:29:00.723 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 1 -- 172.21.15.160:0/1129132689 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5800e980 con 0x5608cba971c0 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5eac4640 10 monclient: handle_config config(0 keys) 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: get_monmap_and_config success 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 4 set_mon_vals no callback set 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: shutdown 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 msgr2=0x5608cba93c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:00.724 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae5800a4b0 tx=0x7fae5800a970 comp rx=0 tx=0).stop 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- 172.21.15.160:0/1129132689 shutdown_connections 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x5608cba971c0 0x5608cba93c10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5608cba96de0 0x5608cbb52470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- 172.21.15.160:0/1129132689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5608cba96a00 0x5608cba956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- 172.21.15.160:0/1129132689 >> 172.21.15.160:0/1129132689 conn(0x5608cbac6af0 msgr2=0x5608cba93500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- 172.21.15.160:0/1129132689 shutdown_connections 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- 172.21.15.160:0/1129132689 wait complete. 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: build_initial_monmap 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: monmap: 2024-09-17T16:29:00.725 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: epoch 0 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: last_changed 0.000000 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: created 0.000000 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: min_mon_release 0 (unknown) 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: election_strategy: 1 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:29:00.726 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:29:00.727 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:29:00.727 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.727 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_pool 2024-09-17T16:29:00.727 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 librados: sttest api_pool on pid 29515 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29515 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_pool on pid 29515' 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29515 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-17T16:29:00.728 INFO:tasks.workunit.client.0.smithi160.stdout:arting msgr at 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 librados: starting objecter 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 Processor -- start 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- start start 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 librados: setting wanted keys 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 librados: calling monclient init 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: init 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: _reopen_session rank -1 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient: _add_conns ranks=[4,8,1,3,5,0,6,2,7] 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 0x5608cbbf5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-c con 0x5608cba96a00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x5608cba96de0 0x5608cbbf55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.729 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-e con 0x5608cba96de0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5608cba971c0 0x5608cbb581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): picked mon.noname-f con 0x5608cba971c0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5608cba927e0 con 0x5608cba971c0 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x5608cbb55ca0 con 0x5608cba96a00 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): start opening mon connection 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x5608cbac6240 con 0x5608cba96de0 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): _renew_subs 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae627bd880 10 monclient(hunting): authenticate will time out at 1018.987283s 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 0x5608cbbf5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x5608cba96de0 0x5608cbbf55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 0x5608cbbf5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.160:45008/0 (socket says 172.21.15.160:45008) 2024-09-17T16:29:00.730 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 1 -- 172.21.15.160:0/3113679633 learned_addr learned my addr 172.21.15.160:0/3113679633 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): get_auth_request con 0x5608cba96a00 auth_method 0 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): get_auth_request con 0x5608cba96de0 auth_method 0 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae60ac8640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5608cba971c0 0x5608cbb581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae602c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:29:00.731 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.716+0000 7fae5fac6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): get_auth_request con 0x5608cba971c0 auth_method 0 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae60ac8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 -- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x5608cba96de0 msgr2=0x5608cbbf55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:00.732 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x5608cba96de0 0x5608cbbf55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.733 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.733 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.733 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.733 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 -- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5608cba971c0 msgr2=0x5608cbb581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5608cba971c0 0x5608cbb581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 10 monclient: _finish_hunting 0 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 monclient: found mon.noname-c 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608cb9c3c00 con 0x5608cba96a00 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae602c7640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 0x5608cbbf5070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c10 tx=0x7fae500180f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:29:00.734 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 client.?.objecter ms_handle_connect 0x5608cba96a00 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 client.?.objecter resend_mon_ops 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae500ba020 con 0x5608cba96a00 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: dump: 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: epoch 1 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: min_mon_release 19 (squid) 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: election_strategy: 3 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:29:00.735 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: _finish_auth 0 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: _check_auth_tickets 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:29:00.736 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 5 monclient: authenticate success, global_id 4450 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 monclient: _renew_subs 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608cbb5a1a0 con 0x5608cba96a00 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae500b0040 con 0x5608cba96a00 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: handle_config config(0 keys) 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae500af070 con 0x5608cba96a00 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 10 monclient: dump: 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: epoch 1 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:29:00.737 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: min_mon_release 19 (squid) 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: election_strategy: 3 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 monclient: _renew_subs 2024-09-17T16:29:00.738 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x5608cbb5b2e0 con 0x5608cba96a00 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5eac4640 4 set_mon_vals no callback set 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 1 librados: init done 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 librados: wait_for_osdmap waiting 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fae500bd020 con 0x5608cba96a00 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae4c03cb10 0x7fae4c03efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.739 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 4374+0+0 (secure 0 0 0) 0x5608cbabf5d0 con 0x5608cba96a00 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_snapshots 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_snapshots' 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_pool: /' 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_snapshots 2024-09-17T16:29:00.740 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.741 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_snapshots 2024-09-17T16:29:00.741 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7faetest api_snapshots on pid 29557 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29557 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_snapshots on pid 29557' 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29557 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.742 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.743 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_snapshots_pp 2024-09-17T16:29:00.743 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_snapshots_pp' 2024-09-17T16:29:00.743 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.743 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.744 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-09-17T16:29:00.744 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.747 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_snapshots_pp 2024-09-17T16:29:00.748 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_snapshots_pp 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stdout:test api_snapshots_pp on pid 29591 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29591 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_snapshots_pp on pid 29591' 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29591 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.749 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-17T16:29:00.750 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-17T16:29:00.752 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_stat 2024-09-17T16:29:00.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_stat' 2024-09-17T16:29:00.753 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-09-17T16:29:00.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_stat 2024-09-17T16:29:00.754 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.756 INFO:tasks.workunit.client.0.smithi160.stdout:test api_stat on pid 29607 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_stat 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29607 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_stat on pid 29607' 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29607 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.757 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_stat_pp 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_stat_pp' 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.758 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-17T16:29:00.759 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-17T16:29:00.759 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_stat: /' 2024-09-17T16:29:00.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_stat_pp 2024-09-17T16:29:00.760 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.762 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-09-17T16:29:00.762 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_stat_pp 2024-09-17T16:29:00.762 INFO:tasks.workunit.client.0.smithi160.stdout:test api_stat_pp on pid 29618 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29618 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_stat_pp on pid 29618' 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29618 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.763 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-17T16:29:00.764 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.764 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.764 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.765 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.765 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-09-17T16:29:00.770 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_watch_notify 2024-09-17T16:29:00.770 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_watch_notify' 2024-09-17T16:29:00.771 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-17T16:29:00.774 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-17T16:29:00.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_watch_notify 2024-09-17T16:29:00.783 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.785 INFO:tasks.workunit.client.0.smithi160.stdout:test api_watch_notify on pid 29658 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_watch_notify 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29658 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_watch_notify on pid 29658' 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29658 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_watch_notify_pp 2024-09-17T16:29:00.786 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_watch_notify_pp' 2024-09-17T16:29:00.787 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_watch_notify_pp 2024-09-17T16:29:00.787 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.787 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.787 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.787 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.788 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.789 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-09-17T16:29:00.789 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-17T16:29:00.789 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stdout:test api_watch_notify_pp on pid 29679 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_watch_notify_pp 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29679 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_watch_notify_pp on pid 29679' 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29679 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.790 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-09-17T16:29:00.791 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-17T16:29:00.792 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_cmd 2024-09-17T16:29:00.792 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_cmd' 2024-09-17T16:29:00.793 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-17T16:29:00.794 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.796 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_cmd 2024-09-17T16:29:00.797 INFO:tasks.workunit.client.0.smithi160.stdout:test api_cmd on pid 29697 2024-09-17T16:29:00.797 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_cmd 2024-09-17T16:29:00.797 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29697 2024-09-17T16:29:00.797 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_cmd on pid 29697' 2024-09-17T16:29:00.797 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29697 2024-09-17T16:29:00.798 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.798 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.798 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-17T16:29:00.799 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.799 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.799 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.800 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.800 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_cmd: /' 2024-09-17T16:29:00.801 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-09-17T16:29:00.802 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-17T16:29:00.802 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_cmd_pp 2024-09-17T16:29:00.803 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_cmd_pp' 2024-09-17T16:29:00.807 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_cmd_pp 2024-09-17T16:29:00.808 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.810 INFO:tasks.workunit.client.0.smithi160.stdout:test api_cmd_pp on pid 29745 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_cmd_pp 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29745 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_cmd_pp on pid 29745' 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29745 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.811 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.812 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_service 2024-09-17T16:29:00.812 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-17T16:29:00.812 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_service' 2024-09-17T16:29:00.814 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.814 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.814 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.814 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.815 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-09-17T16:29:00.819 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_service 2024-09-17T16:29:00.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-17T16:29:00.819 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-17T16:29:00.819 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.824 INFO:tasks.workunit.client.0.smithi160.stdout:test api_service on pid 29778 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_service 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29778 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_service on pid 29778' 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29778 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.825 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.826 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_service_pp 2024-09-17T16:29:00.827 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_service_pp' 2024-09-17T16:29:00.827 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.829 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_service_pp 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stdout:test api_service_pp on pid 29799 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_service_pp 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29799 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_service_pp on pid 29799' 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29799 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.831 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-17T16:29:00.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.832 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.833 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.833 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-09-17T16:29:00.833 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_c_write_operations 2024-09-17T16:29:00.833 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_c_write_operations' 2024-09-17T16:29:00.838 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-17T16:29:00.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-17T16:29:00.839 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-17T16:29:00.840 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.840 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.840 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.841 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.841 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_c_write_operations 2024-09-17T16:29:00.841 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_service.log 2024-09-17T16:29:00.841 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_service: /' 2024-09-17T16:29:00.841 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-09-17T16:29:00.842 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.849 INFO:tasks.workunit.client.0.smithi160.stdout:test api_c_write_operations on pid 29849 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_c_write_operations 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29849 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_c_write_operations on pid 29849' 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29849 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.850 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s api_c_read_operations 2024-09-17T16:29:00.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' api_c_read_operations' 2024-09-17T16:29:00.851 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-17T16:29:00.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo api_c_read_operations 2024-09-17T16:29:00.851 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.853 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=api_c_read_operations 2024-09-17T16:29:00.853 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29859 2024-09-17T16:29:00.853 INFO:tasks.workunit.client.0.smithi160.stdout:test api_c_read_operations on pid 29859 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test api_c_read_operations on pid 29859' 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29859 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s list_parallel 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' list_parallel' 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo list_parallel 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.855 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-17T16:29:00.856 INFO:tasks.workunit.client.0.smithi160.stdout:test list_parallel on pid 29879 2024-09-17T16:29:00.857 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=list_parallel 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29879 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test list_parallel on pid 29879' 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29879 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s open_pools_parallel 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.858 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-09-17T16:29:00.859 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' open_pools_parallel' 2024-09-17T16:29:00.861 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.861 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.861 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.861 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.863 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-09-17T16:29:00.863 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-17T16:29:00.863 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ list_parallel: /' 2024-09-17T16:29:00.866 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-17T16:29:00.869 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-17T16:29:00.870 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.871 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo open_pools_parallel 2024-09-17T16:29:00.874 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=open_pools_parallel 2024-09-17T16:29:00.874 INFO:tasks.workunit.client.0.smithi160.stdout:test open_pools_parallel on pid 29917 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29917 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test open_pools_parallel on pid 29917' 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29917 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.875 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s delete_pools_parallel 2024-09-17T16:29:00.876 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' delete_pools_parallel' 2024-09-17T16:29:00.876 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-17T16:29:00.876 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo delete_pools_parallel 2024-09-17T16:29:00.877 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.877 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.877 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.877 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.877 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.878 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-09-17T16:29:00.879 INFO:tasks.workunit.client.0.smithi160.stdout:test delete_pools_parallel on pid 29926 2024-09-17T16:29:00.879 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=delete_pools_parallel 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29926 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test delete_pools_parallel on pid 29926' 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29926 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-17T16:29:00.880 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-17T16:29:00.882 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.882 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.882 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.882 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.882 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-17T16:29:00.888 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s cls 2024-09-17T16:29:00.889 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' cls' 2024-09-17T16:29:00.889 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-17T16:29:00.890 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo cls 2024-09-17T16:29:00.891 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.892 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-17T16:29:00.893 INFO:tasks.workunit.client.0.smithi160.stdout:test cls on pid 29941 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=cls 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29941 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test cls on pid 29941' 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29941 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s cmd 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' cmd' 2024-09-17T16:29:00.894 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-09-17T16:29:00.902 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-09-17T16:29:00.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.904 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.914 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_cls 2024-09-17T16:29:00.915 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo cmd 2024-09-17T16:29:00.916 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.916 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_cls.log 2024-09-17T16:29:00.917 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ cls: /' 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stdout:test cmd on pid 29992 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=cmd 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29992 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test cmd on pid 29992' 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=29992 2024-09-17T16:29:00.919 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.920 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.920 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-09-17T16:29:00.921 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.921 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.921 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.922 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.922 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_cmd.log 2024-09-17T16:29:00.922 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s handler_error 2024-09-17T16:29:00.922 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' handler_error' 2024-09-17T16:29:00.923 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_cmd 2024-09-17T16:29:00.923 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo handler_error 2024-09-17T16:29:00.923 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=handler_error 2024-09-17T16:29:00.926 INFO:tasks.workunit.client.0.smithi160.stdout:test handler_error on pid 30008 2024-09-17T16:29:00.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30008 2024-09-17T16:29:00.926 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test handler_error on pid 30008' 2024-09-17T16:29:00.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30008 2024-09-17T16:29:00.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.927 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-09-17T16:29:00.927 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s io 2024-09-17T16:29:00.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' io' 2024-09-17T16:29:00.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.928 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.930 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ cmd: /' 2024-09-17T16:29:00.934 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_handler_error.log 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout:5dac2640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_map(20..20 src has 1..20) 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map got epochs [20,20] > 0 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map decoding full epoch 20 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5dac2640 20 client.4450.objecter dump_active .. 0 homeless 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 librados: wait_for_osdmap done waiting 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae500a3e50 con 0x5608cba96a00 2024-09-17T16:29:00.937 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5fac6640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae4c03cb10 0x7fae4c03efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.720+0000 7fae5fac6640 10 monclient: get_auth_request con 0x7fae4c03cb10 auth_method 0 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fae500b5770 con 0x5608cba96a00 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae5dac2640 10 monclient: handle_get_version_reply finishing 1 version 20 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae5eac4640 10 client.4450.objecter _get_latest_version latest 20, have it 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae627bd880 10 client.4450.objecter create_pool name=ceph_test_rados_api_asio 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae627bd880 10 client.4450.objecter pool_op_submit 1 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae627bd880 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x5608cba4b880 con 0x5608cba96a00 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae5dac2640 10 client.4450.objecter ms_handle_connect 0x7fae4c03cb10 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.724+0000 7fae5fac6640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae4c03cb10 0x7fae4c03efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae54005410 tx=0x7fae5400c040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 7 ==== pool_op_reply(tid 1 (0) Success v21) ==== 43+0+0 (secure 0 0 0) 0x7fae500c4030 con 0x5608cba96a00 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 pool_op_reply(tid 1 (0) Success v21) 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 client.4450.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v21) 2024-09-17T16:29:00.938 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 client.4450.objecter have request 1 at 0x5608cbb5bba0 Op: create 2024-09-17T16:29:00.939 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 20 client.4450.objecter waiting for client to reach epoch 21 before calling back 2024-09-17T16:29:00.939 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:29:00.941 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_handler_error 2024-09-17T16:29:00.947 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ handler_error: /' 2024-09-17T16:29:00.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.949 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo io 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stdout: test io on pid 30055 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=io 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30055 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test io on pid 30055' 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30055 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.952 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-09-17T16:29:00.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.953 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.960 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s ec_io 2024-09-17T16:29:00.961 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' ec_io' 2024-09-17T16:29:00.971 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_io 2024-09-17T16:29:00.971 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_io.log 2024-09-17T16:29:00.972 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo ec_io 2024-09-17T16:29:00.975 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ io: /' 2024-09-17T16:29:00.979 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stdout:test ec_io on pid 30131 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=ec_io 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30131 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test ec_io on pid 30131' 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30131 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:00.982 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 monclient: _renew_subs 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=21}) -- 0x7fae4c0422e0 con 0x5608cba96a00 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.932+0000 7fae5dac2640 10 client.4450.objecter done 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== mon.4 v2:172.21.15.105:3302/0 8 ==== osd_map(21..21 src has 1..21) ==== 6540+0+0 (secure 0 0 0) 0x7fae500bea10 con 0x5608cba96a00 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae5dac2640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_map(21..21 src has 1..21) 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map got epochs [21,21] > 20 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map decoding incremental epoch 21 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae5dac2640 20 client.4450.objecter dump_active .. 0 homeless 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 10 librados: set snap read head -> 1 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 10 librados: writefull oid=exist nspace= 2024-09-17T16:29:00.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 21 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x5608cbb3d500 0x5608cbb3f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [writefull 0~5 in=5b] tid 2 osd.1 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 2 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 15 client.4450.objecter _send_op 2 to 6.1 on osd.1 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:2 6.1 6:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21) -- 0x5608cbb3fe30 con 0x5608cbb3d500 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.936+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:00.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.976+0000 7fae60ac8640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x5608cbb3d500 0x5608cbb3f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:00.985 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ ec_io: /' 2024-09-17T16:29:00.986 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_ec_io.log 2024-09-17T16:29:00.986 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_ec_io 2024-09-17T16:29:00.994 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s list 2024-09-17T16:29:00.994 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' list' 2024-09-17T16:29:01.001 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo list 2024-09-17T16:29:01.001 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.976+0000 7fae60ac8640 10 monclient: get_auth_requetest list on pid 30161 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=list 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30161 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test list on pid 30161' 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30161 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.003 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.005 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s ec_list 2024-09-17T16:29:01.005 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' ec_list' 2024-09-17T16:29:01.005 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo ec_list 2024-09-17T16:29:01.008 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-09-17T16:29:01.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.030 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.031 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_list 2024-09-17T16:29:01.032 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ list: /' 2024-09-17T16:29:01.034 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_list.log 2024-09-17T16:29:01.034 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.044 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=ec_list 2024-09-17T16:29:01.044 INFO:tasks.workunit.client.0.smithi160.stdout:test ec_list on pid 30234 2024-09-17T16:29:01.044 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30234 2024-09-17T16:29:01.044 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test ec_list on pid 30234' 2024-09-17T16:29:01.044 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30234 2024-09-17T16:29:01.045 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.045 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.045 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_ec_list.log 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_ec_list 2024-09-17T16:29:01.046 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ ec_list: /' 2024-09-17T16:29:01.047 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s misc 2024-09-17T16:29:01.049 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' misc' 2024-09-17T16:29:01.051 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo misc 2024-09-17T16:29:01.051 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.053 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=misc 2024-09-17T16:29:01.053 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30261 2024-09-17T16:29:01.053 INFO:tasks.workunit.client.0.smithi160.stdout:test misc on pid 30261 2024-09-17T16:29:01.054 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test misc on pid 30261' 2024-09-17T16:29:01.054 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30261 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s pool 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.055 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.056 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' pool' 2024-09-17T16:29:01.056 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_misc 2024-09-17T16:29:01.056 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_misc.log 2024-09-17T16:29:01.056 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo pool 2024-09-17T16:29:01.056 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.058 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=pool 2024-09-17T16:29:01.058 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30278 2024-09-17T16:29:01.058 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test pool on pid 30278' 2024-09-17T16:29:01.058 INFO:tasks.workunit.client.0.smithi160.stdout:test pool on pid 30278 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30278 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ misc: /' 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s read_operations 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' read_operations' 2024-09-17T16:29:01.059 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo read_operations 2024-09-17T16:29:01.060 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.060 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-09-17T16:29:01.061 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.061 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.062 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.062 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.062 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ pool: /' 2024-09-17T16:29:01.065 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_pool.log 2024-09-17T16:29:01.066 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_pool 2024-09-17T16:29:01.068 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=read_operations 2024-09-17T16:29:01.068 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30311 2024-09-17T16:29:01.069 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test read_operations on pid 30311' 2024-09-17T16:29:01.069 INFO:tasks.workunit.client.0.smithi160.stdout:test read_operations on pid 30311 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30311 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.070 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.071 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_read_operations 2024-09-17T16:29:01.071 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_read_operations.log 2024-09-17T16:29:01.071 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s snapshots 2024-09-17T16:29:01.071 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' snapshots' 2024-09-17T16:29:01.075 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ read_operations: /' 2024-09-17T16:29:01.075 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.078 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo snapshots 2024-09-17T16:29:01.079 INFO:tasks.workunit.client.0.smithi160.stdout:test snapshots on pid 30337 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=snapshots 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30337 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test snapshots on pid 30337' 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30337 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s watch_notify 2024-09-17T16:29:01.080 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' watch_notify' 2024-09-17T16:29:01.081 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo watch_notify 2024-09-17T16:29:01.081 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_snapshots.log 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ snapshots: /' 2024-09-17T16:29:01.082 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.083 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_snapshots 2024-09-17T16:29:01.084 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=watch_notify 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stdout:test watch_notify on pid 30354 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30354 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test watch_notify on pid 30354' 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30354 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-09-17T16:29:01.085 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:++ printf %25s write_operations 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ r=' write_operations' 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.086 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_watch_notify 2024-09-17T16:29:01.087 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ watch_notify: /' 2024-09-17T16:29:01.089 INFO:tasks.workunit.client.0.smithi160.stderr:++ echo write_operations 2024-09-17T16:29:01.089 INFO:tasks.workunit.client.0.smithi160.stderr:++ awk '{print $1}' 2024-09-17T16:29:01.091 INFO:tasks.workunit.client.0.smithi160.stderr:+ ff=write_operations 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stdout:test write_operations on pid 30370 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30370 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ echo 'test write_operations on pid 30370' 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ pids[$f]=30370 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ ret=0 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-09-17T16:29:01.092 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' 1 -eq 1 ']' 2024-09-17T16:29:01.093 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:29:01.093 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29481 2024-09-17T16:29:01.093 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29481 2024-09-17T16:29:01.093 INFO:tasks.workunit.client.0.smithi160.stderr:+ '[' -z '' ']' 2024-09-17T16:29:01.094 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.094 INFO:tasks.workunit.client.0.smithi160.stderr:+ case $- in 2024-09-17T16:29:01.094 INFO:tasks.workunit.client.0.smithi160.stderr:+ return 2024-09-17T16:29:01.094 INFO:tasks.workunit.client.0.smithi160.stderr:+ sed 's/^/ write_operations: /' 2024-09-17T16:29:01.099 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-09-17T16:29:01.105 INFO:tasks.workunit.client.0.smithi160.stderr:+ ceph_test_neorados_write_operations 2024-09-17T16:29:01.105 INFO:tasks.workunit.client.0.smithi160.stderr:+ tee ceph_test_neorados_write_operations.log 2024-09-17T16:29:02.191 INFO:tasks.workunit.client.0.smithi160.stdout:st con 0x5608cbb3d500 auth_method 0 2024-09-17T16:29:02.191 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.980+0000 7fae60ac8640 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x5608cbb3d500 0x5608cbb3f8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:00.980+0000 7fae5dac2640 10 client.4450.objecter ms_handle_connect 0x5608cbb3d500 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 1 ==== osd_map(22..22 src has 1..22) ==== 13738+0+0 (crc 0 0 0) 0x7fae5800f920 con 0x5608cbb3d500 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_map(22..22 src has 1..22) 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map got epochs [22,22] > 21 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 3 client.4450.objecter handle_osd_map decoding incremental epoch 22 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 10 client.4450.objecter checking op 2 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 20 client.4450.objecter dump_active .. 0 homeless 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.116+0000 7fae5dac2640 20 client.4450.objecter 2 6.d3a59201 osd.1 exist [writefull 0~5 in=5b] 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 2 ==== osd_op_reply(2 exist [writefull 0~5] v22'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.192 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(2 exist [writefull 0~5] v22'1 uv1 ondisk = 0) 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 2 ondisk uv 1 in 6.1 attempt 0 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 2 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 15 client.4450.objecter _finish_op 2 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 2 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 librados: Objecter returned from writefull r=0 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.193 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~256] tid 3 osd.1 2024-09-17T16:29:02.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 3 2024-09-17T16:29:02.197 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 15 client.4450.objecter _send_op 3 to 6.1 on osd.1 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:3 6.1 6:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb58720 con 0x5608cbb3d500 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb58ac0 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~256] tid 4 osd.1 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 4 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 15 client.4450.objecter _send_op 4 to 6.1 on osd.1 2024-09-17T16:29:02.198 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:4 6.1 6:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb58f70 con 0x5608cbb3d500 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.188+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 3 ondisk uv 1 in 6.1 attempt 0 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 3 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter _finish_op 3 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 3 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 4 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 4 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 4 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter _finish_op 4 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 4 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (6 ms) 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb58ac0 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~256] tid 5 osd.1 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 5 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 15 client.4450.objecter _send_op 5 to 6.1 on osd.1 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:5 6.1 6:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb59310 con 0x5608cbb3d500 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~256] tid 6 osd.1 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 6 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 15 client.4450.objecter _send_op 6 to 6.1 on osd.1 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:6 6.1 6:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb40200 con 0x5608cbb3d500 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.192+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 5 ondisk uv 1 in 6.1 attempt 0 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 5 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _finish_op 5 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 5 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 6 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 6 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 6 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _finish_op 6 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 6 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~256] tid 7 osd.1 2024-09-17T16:29:02.204 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 7 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _send_op 7 to 6.1 on osd.1 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:7 6.1 6:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb58ac0 con 0x5608cbb3d500 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb405a0 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~256] tid 8 osd.1 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 8 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _send_op 8 to 6.1 on osd.1 2024-09-17T16:29:02.205 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:8 6.1 6:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb40a50 con 0x5608cbb3d500 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 7 ondisk uv 1 in 6.1 attempt 0 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 7 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _finish_op 7 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 7 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 8 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.206 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 8 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 8 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _finish_op 8 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 8 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb52160 write_seq 1 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.207 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.212 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.212 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [write 0~5 in=5b] tid 9 osd.1 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 9 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 15 client.4450.objecter _send_op 9 to 6.1 on osd.1 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:9 6.1 6:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb405a0 con 0x5608cbb3d500 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.196+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 9 ==== osd_op_reply(9 exist [write 0~5] v22'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(9 exist [write 0~5] v22'2 uv2 ondisk = 0) 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 9 ondisk uv 2 in 6.1 attempt 0 2024-09-17T16:29:02.213 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 9 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 15 client.4450.objecter _finish_op 9 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 9 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb52160 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (12 ms) 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb40990 write_seq 2 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.214 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [write 0~5 in=5b] tid 10 osd.1 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 10 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 15 client.4450.objecter _send_op 10 to 6.1 on osd.1 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:10 6.1 6:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb405a0 con 0x5608cbb3d500 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.208+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-09-17T16:29:02.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 10 ==== osd_op_reply(10 exist [write 0~5] v22'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(10 exist [write 0~5] v22'3 uv3 ondisk = 0) 2024-09-17T16:29:02.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 10 ondisk uv 3 in 6.1 attempt 0 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 10 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter _finish_op 10 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 10 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb40990 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (8 ms) 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb40cd0 write_seq 3 2024-09-17T16:29:02.225 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.226 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [write 0~5 in=5b] tid 11 osd.1 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 11 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _send_op 11 to 6.1 on osd.1 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:11 6.1 6:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb438e0 con 0x5608cbb3d500 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.227 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 11 ==== osd_op_reply(11 exist [write 0~5] v22'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(11 exist [write 0~5] v22'4 uv4 ondisk = 0) 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 11 ondisk uv 4 in 6.1 attempt 0 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 11 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter _finish_op 11 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 11 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb40cd0 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2 ms) 2024-09-17T16:29:02.228 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~0] tid 12 osd.1 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 12 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _send_op 12 to 6.1 on osd.1 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:12 6.1 6:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb438e0 con 0x5608cbb3d500 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb40e40 2024-09-17T16:29:02.229 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~0] tid 13 osd.1 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 13 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 15 client.4450.objecter _send_op 13 to 6.1 on osd.1 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:13 6.1 6:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb412f0 con 0x5608cbb3d500 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.216+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) 2024-09-17T16:29:02.230 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 12 ondisk uv 4 in 6.1 attempt 0 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 12 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 12 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 12 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 13 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 13 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.231 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 13 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 13 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 13 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~0] tid 14 osd.1 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.232 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 14 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _send_op 14 to 6.1 on osd.1 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:14 6.1 6:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb416e0 con 0x5608cbb3d500 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb40e40 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~0] tid 15 osd.1 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 15 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _send_op 15 to 6.1 on osd.1 2024-09-17T16:29:02.233 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:15 6.1 6:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb412f0 con 0x5608cbb3d500 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 14 ondisk uv 4 in 6.1 attempt 0 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 14 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 14 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 14 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 15 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.234 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 15 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 15 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 15 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 15 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.235 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [read 0~0] tid 16 osd.1 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 16 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _send_op 16 to 6.1 on osd.1 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:16 6.1 6:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb416e0 con 0x5608cbb3d500 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb40e40 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base noexist @6 precalc_pgid 0 pgid 0.0 is_read 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target target noexist @6 -> pgid 6.a7bbaa1 2024-09-17T16:29:02.236 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.a7bbaa1 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit oid noexist '@6' '@6' [read 0~0] tid 17 osd.1 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 17 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _send_op 17 to 6.1 on osd.1 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:17 6.1 6:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb412f0 con 0x5608cbb3d500 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 5 client.4450.objecter 2 in flight 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 16 ondisk uv 4 in 6.1 attempt 0 2024-09-17T16:29:02.237 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 5 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 16 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 16 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 5 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 16 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 17 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 17 ondisk uv 0 in 6.1 attempt 0 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 17 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _finish_op 17 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.238 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 17 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb43b00 write_seq 4 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [writefull 0~5 in=5b] tid 18 osd.1 2024-09-17T16:29:02.239 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 18 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 15 client.4450.objecter _send_op 18 to 6.1 on osd.1 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:18 6.1 6:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb416e0 con 0x5608cbb3d500 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.220+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (4 ms) 2024-09-17T16:29:02.240 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 18 ==== osd_op_reply(18 exist [writefull 0~5] v22'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(18 exist [writefull 0~5] v22'5 uv5 ondisk = 0) 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 18 ondisk uv 5 in 6.1 attempt 0 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 18 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 15 client.4450.objecter _finish_op 18 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 18 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb43b00 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb44e70 write_seq 5 2024-09-17T16:29:02.241 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [writefull 0~5 in=5b] tid 19 osd.1 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 19 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 15 client.4450.objecter _send_op 19 to 6.1 on osd.1 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:19 6.1 6:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb416e0 con 0x5608cbb3d500 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.224+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 19 ==== osd_op_reply(19 exist [writefull 0~5] v22'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.242 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(19 exist [writefull 0~5] v22'6 uv6 ondisk = 0) 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 19 ondisk uv 6 in 6.1 attempt 0 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 19 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter _finish_op 19 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 19 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb44e70 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (3 ms) 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-17T16:29:02.243 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 librados: queue_aio_write 0x5608cb9dd030 completion 0x5608cbb43b00 write_seq 6 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 client.4450.objecter _op_submit op 0x5608cbb3cb40 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter _calc_target epoch 22 base exist @6 precalc_pgid 0 pgid 0.0 is_write 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter _calc_target target exist @6 -> pgid 6.d3a59201 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 client.4450.objecter _calc_target raw pgid 6.d3a59201 -> actual 6.1 acting [1,3] primary 1 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter _get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 client.4450.objecter _op_submit oid exist '@6' '@6' [writefull 0~5 in=5b] tid 20 osd.1 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter get_session s=0x5608cbb5b820 osd=1 3 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 15 client.4450.objecter _session_op_assign 1 20 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 15 client.4450.objecter _send_op 20 to 6.1 on osd.1 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- osd_op(unknown.0.0:20 6.1 6:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22) -- 0x5608cbb416e0 con 0x5608cbb3d500 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 5 client.4450.objecter 1 in flight 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-09-17T16:29:02.244 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 <== osd.1 v2:172.21.15.105:6800/3243801162 20 ==== osd_op_reply(20 exist [writefull 0~5] v22'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fae5800e070 con 0x5608cbb3d500 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter ms_dispatch 0x5608cba1af00 osd_op_reply(20 exist [writefull 0~5] v22'7 uv7 ondisk = 0) 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter in handle_osd_op_reply 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 7 client.4450.objecter handle_osd_op_reply 20 ondisk uv 7 in 6.1 attempt 0 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 10 client.4450.objecter op 0 rval 0 len 0 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter handle_osd_op_reply completed tid 20 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter _finish_op 20 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 4 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 15 client.4450.objecter _session_op_remove 1 20 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 5 client.4450.objecter 0 in flight 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 20 librados: complete_aio_write 0x5608cbb43b00 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 librados: watch_flush enter 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 librados: watch_flush exit 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 client.4450.objecter close_session for osd.1 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x5608cbb3d500 msgr2=0x5608cbb3f8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:02.245 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x5608cbb3d500 0x5608cbb3f8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter put_session s=0x5608cbb5b820 osd=1 1 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 client.4450.objecter shutdown clearing up homeless session... 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 client.4450.objecter successfully canceled tick 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae4c03cb10 msgr2=0x7fae4c03efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae4c03cb10 0x7fae4c03efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae54005410 tx=0x7fae5400c040 comp rx=0 tx=0).stop 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 10 monclient: shutdown 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 msgr2=0x5608cbbf5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 --2- 172.21.15.160:0/3113679633 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x5608cba96a00 0x5608cbbf5070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fae50000c10 tx=0x7fae500180f0 comp rx=0 tx=0).stop 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae60ac8640 1 -- 172.21.15.160:0/3113679633 reap_dead start 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 shutdown_connections 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 >> 172.21.15.160:0/3113679633 conn(0x5608cbac6af0 msgr2=0x5608cbac42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 shutdown_connections 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 -- 172.21.15.160:0/3113679633 wait complete. 2024-09-17T16:29:02.246 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.228+0000 7fae627bd880 1 librados: shutdown 2024-09-17T16:29:02.247 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] 12 tests from AsioRados (41 ms total) 2024-09-17T16:29:02.247 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: 2024-09-17T16:29:02.247 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [----------] Global test environment tear-down 2024-09-17T16:29:02.247 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (1520 ms total) 2024-09-17T16:29:02.247 INFO:tasks.workunit.client.0.smithi160.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-17T16:29:03.293 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-17T16:29:03.293 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] Global test environment set-up. 2024-09-17T16:29:03.293 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjects (456 ms) 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (31 ms) 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo2,foo3 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo2 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo3 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo4,foo5 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo4 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo5 2024-09-17T16:29:03.294 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo6,foo7 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo7 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo6 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo4 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo5 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo7 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo6 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo1 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo1 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo2 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo3 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (161 ms) 2024-09-17T16:29:03.295 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 1 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 10 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 13 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 7 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 14 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 0 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 15 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 11 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 5 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 8 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 6 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 3 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 4 0 2024-09-17T16:29:03.296 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 12 0 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 9 0 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2 0 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (174 ms) 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: x cursor=MIN 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=1 cursor=8:02547ec2:::1:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=10 cursor=8:52ea6a34:::10:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=13 cursor=8:566253c9:::13:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=7 cursor=8:5c6b0b28:::7:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=14 cursor=8:62a1935d:::14:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=0 cursor=8:6cac518f:::0:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=15 cursor=8:863748b0:::15:head 2024-09-17T16:29:03.297 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=11 cursor=8:89d3ae78:::11:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=5 cursor=8:b29083e3:::5:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=8 cursor=8:bd63b0f1:::8:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=6 cursor=8:c4fdafeb:::6:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=3 cursor=8:cfc208b3:::3:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=4 cursor=8:d83876eb:::4:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=12 cursor=8:de5d7c5f:::12:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=9 cursor=8:e960b815:::9:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > oid=2 cursor=8:f905c69b:::2:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=1 cursor=8:02547ec2:::1:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:02547ec2:::1:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:02547ec2:::1:head -> 1 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=10 cursor=8:52ea6a34:::10:head 2024-09-17T16:29:03.298 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:52ea6a34:::10:head 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=13 cursor=8:566253c9:::13:head 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:566253c9:::13:head 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:566253c9:::13:head -> 13 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=7 cursor=8:5c6b0b28:::7:head 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2024-09-17T16:29:03.299 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=14 cursor=8:62a1935d:::14:head 2024-09-17T16:29:03.366 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:62a1935d:::14:head 2024-09-17T16:29:03.366 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:62a1935d:::14:head -> 14 2024-09-17T16:29:03.366 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=0 cursor=8:6cac518f:::0:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:6cac518f:::0:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:6cac518f:::0:head -> 0 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=15 cursor=8:863748b0:::15:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:863748b0:::15:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:863748b0:::15:head -> 15 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=11 cursor=8:89d3ae78:::11:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:89d3ae78:::11:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=5 cursor=8:b29083e3:::5:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:b29083e3:::5:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:b29083e3:::5:head -> 5 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=8 cursor=8:bd63b0f1:::8:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=6 cursor=8:c4fdafeb:::6:head 2024-09-17T16:29:03.367 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=3 cursor=8:cfc208b3:::3:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:cfc208b3:::3:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=4 cursor=8:d83876eb:::4:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:d83876eb:::4:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:d83876eb:::4:head -> 4 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=12 cursor=8:de5d7c5f:::12:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=9 cursor=8:e960b815:::9:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:e960b815:::9:head 2024-09-17T16:29:03.368 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:e960b815:::9:head -> 9 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : oid=2 cursor=8:f905c69b:::2:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:f905c69b:::2:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:f905c69b:::2:head -> 2 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:02547ec2:::1:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:02547ec2:::1:head expected=8:02547ec2:::1:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:02547ec2:::1:head -> 1 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=1 expected=1 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:f905c69b:::2:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:f905c69b:::2:head expected=8:f905c69b:::2:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:f905c69b:::2:head -> 2 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=2 expected=2 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:e960b815:::9:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:e960b815:::9:head expected=8:e960b815:::9:head 2024-09-17T16:29:03.369 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:e960b815:::9:head -> 9 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=9 expected=9 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:de5d7c5f:::12:head 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:de5d7c5f:::12:head expected=8:de5d7c5f:::12:head 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:de5d7c5f:::12:head -> 12 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=12 expected=12 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:d83876eb:::4:head 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:d83876eb:::4:head expected=8:d83876eb:::4:head 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:d83876eb:::4:head -> 4 2024-09-17T16:29:03.370 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=4 expected=4 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:cfc208b3:::3:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:cfc208b3:::3:head expected=8:cfc208b3:::3:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:cfc208b3:::3:head -> 3 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=3 expected=3 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:c4fdafeb:::6:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:c4fdafeb:::6:head expected=8:c4fdafeb:::6:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:c4fdafeb:::6:head -> 6 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=6 expected=6 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:bd63b0f1:::8:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:bd63b0f1:::8:head expected=8:bd63b0f1:::8:head 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:bd63b0f1:::8:head -> 8 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=8 expected=8 2024-09-17T16:29:03.371 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:b29083e3:::5:head 2024-09-17T16:29:03.982 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : curs api_c_read_operations: Running main() from gmock_main.cc 2024-09-17T16:29:03.982 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-09-17T16:29:03.982 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (264 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (13 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (10 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (22 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-09-17T16:29:03.983 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (13 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (13 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (10 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (8 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (7 ms) 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-09-17T16:29:03.984 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (11 ms) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (26 ms) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (20 ms) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (15 ms) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (8 ms) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (454 ms total) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (3107 ms total) 2024-09-17T16:29:03.985 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: Running main() from gmock_main.cc 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [==========] Running 1 test from 1 test suite. 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] Global test environment set-up. 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] 1 test from NeoRadosCls 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ OK ] NeoRadosCls.DNE (3045 ms) 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] 1 test from NeoRadosCls (3045 ms total) 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [----------] Global test environment tear-down 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [==========] 1 test from 1 test suite ran. (3046 ms total) 2024-09-17T16:29:03.986 INFO:tasks.workunit.client.0.smithi160.stdout: cls: [ PASSED ] 1 test. 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (55 ms) 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (181 ms) 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2918 ms) 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (3154 ms total) 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: 2024-09-17T16:29:03.988 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-17T16:29:03.989 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (3154 ms total) 2024-09-17T16:29:03.989 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: Running main() from gmock_main.cc 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] Global test environment set-up. 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (3022 ms) 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] 1 test from neocls_handler_error (3022 ms total) 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [----------] Global test environment tear-down 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [==========] 1 test from 1 test suite ran. (3022 ms total) 2024-09-17T16:29:03.990 INFO:tasks.workunit.client.0.smithi160.stdout: handler_error: [ PASSED ] 1 test. 2024-09-17T16:29:04.097 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-17T16:29:04.097 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-17T16:29:04.097 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-17T16:29:04.097 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-17T16:29:04.097 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (84 ms) 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (109 ms) 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2980 ms) 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:02.999019+0000 mon.f [INF] from='client.? 172.21.15.160:0/314314766' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29504-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.017383+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29504-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:04.098 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.091936+0000 mon.f [INF] from='client.? 172.21.15.160:0/327800936' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.095743+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.096504+0000 mon.f [INF] from='client.? 172.21.15.160:0/327800936' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.113553+0000 mon.h [INF] from='client.? 172.21.15.160:0/3923822698' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.119197+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.119532+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.123413+0000 mon.h [INF] from='client.? 172.21.15.160:0/3923822698' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.123554+0000 mon.f [INF] from='client.? 172.21.15.160:0/327800936' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-29364-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.132948+0000 mon.d [INF] from='client.? 172.21.15.160:0/2217910826' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.133434+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.134167+0000 mon.h [INF] from='client.? 172.21.15.160:0/3923822698' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-29612-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.139104+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-29364-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.139984+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-29612-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.140351+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:04.099 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.140668+0000 mon.d [INF] from='client.? 172.21.15.160:0/2217910826' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.149093+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.149390+0000 mon.d [INF] from='client.? 172.21.15.160:0/2217910826' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.155980+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.957515+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi160-30138-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi160-30138-1"}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.957665+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi160-30237-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi160-30237-1"}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.957760+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29504-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.957829+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi160-29364-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.957924+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi160-29612-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.958016+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T16:29:04.100 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.963679+0000 mon.h [INF] from='client.? 172.21.15.160:0/3923822698' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-29612-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:04.382 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.964752+0000 mon.d [INF] from='client.? 172.21.15.160:0/2217910826' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-29620-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, list_parallel: process_1_[29931]: starting. 2024-09-17T16:29:04.382 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: creating pool ceph_test_rados_list_parallel.smithi160-29898 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: created object 0... 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: created object 25... 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: created object 49... 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: finishing. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_1_[29931]: shutting down. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: starting. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: listing objects. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: listed object 0... 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: listed object 25... 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: saw 50 objects 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_2_[29932]: shutting down. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: starting. 2024-09-17T16:29:04.383 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: creating pool ceph_test_rados_list_parallel.smithi160-29898 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: created object 0... 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: created object 25... 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: created object 49... 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: finishing. 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_3_[30684]: shutting down. 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: starting. 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: listing objects. 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: listed object 0... 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: listed object 25... 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: saw 45 objects 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_4_[30685]: shutting down. 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.384 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: starting. 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: removed 25 objects... 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: removed half of the objects 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: removed 50 objects... 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: removed 50 objects 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_5_[30686]: shutting down. 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: starting. 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: creating pool ceph_test_rados_list_parallel.smithi160-29898 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: created object 0... 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: created object 25... 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: created object 49... 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: finishing. 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_6_[30735]: shutting down. 2024-09-17T16:29:04.385 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: starting. 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: listing objects. 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: listed object 0... 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: listed object 25... 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: listed object 50... 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: saw 55 objects 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_7_[30736]: shutting down. 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: starting. 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: added 25 objects... 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: added half of the objects 2024-09-17T16:29:04.386 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: added 50 objects... 2024-09-17T16:29:04.387 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: added 50 objects 2024-09-17T16:29:04.387 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_8_[30737]: shutting down. 2024-09-17T16:29:04.387 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.387 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.765 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.765 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: starting. 2024-09-17T16:29:04.765 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: creating pool ceph_test_rados_list_parallel.smithi160-29898 2024-09-17T16:29:04.765 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: created object 0... 2024-09-17T16:29:04.765 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: created object 25... 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: created object 49... 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: finishing. 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_9_[31002]: shutting down. 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.766 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.767 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: starting. 2024-09-17T16:29:04.767 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: listing objects. 2024-09-17T16:29:04.767 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: listed object 0... 2024-09-17T16:29:04.767 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: listed object 25... 2024-09-17T16:29:04.767 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: listed object 50... 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: listed object 75... 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: saw 98 objects 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_10_[31003]: shutting down. 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: starting. 2024-09-17T16:29:04.768 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: removed 25 objects... 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: removed half of the objects 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: removed 50 objects... 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: removed 50 objects 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_13_[31006]: shutting down. 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: starting. 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: added 25 objects... 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: added half of the objects 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: added 50 objects... 2024-09-17T16:29:04.769 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: added 50 objects 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_12_[31005]: shutting down. 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: starting. 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: added 25 objects... 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: added half of the objects 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: added 50 objects... 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: added 50 objects 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_11_[31004]: shutting down. 2024-09-17T16:29:04.770 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: starting. 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: creating pool ceph_test_rados_list_parallel.smithi160-29898 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: created object 0... 2024-09-17T16:29:04.771 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: created object 25... 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: created object 49... 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: finishing. 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_14_[31087]: shutting down. 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.772 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.773 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:04.773 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: starting. 2024-09-17T16:29:04.773 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listing objects. 2024-09-17T16:29:04.773 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 0... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 25... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 50... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 75... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 100... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: listed object 125... 2024-09-17T16:29:05.041 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: saw 150 objects 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_15_[31088]: shutting down. 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.042 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: starting. 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: added 25 objects... 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: added half of the objects 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: added 50 objects... 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: added 50 objects 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: process_16_[31089]: shutting down. 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******************************* 2024-09-17T16:29:05.043 INFO:tasks.workunit.client.0.smithi160.stdout: list_parallel: ******* SUCCESS ********** 2024-09-17T16:29:05.090 INFO:tasks.workunit.client.0.smithi160.stdout:"erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:05.090 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.968255+0000 mon.d [INF] from='client.? 172.21.15.160:0/3891914155' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.090 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.968717+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi160-29612-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:05.090 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.969352+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi160-29620-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.971347+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.974664+0000 mon.f [INF] from='client.? 172.21.15.160:0/327800936' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-29364-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.977498+0000 mon.f [INF] from='client.? 172.21.15.160:0/3211836238' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-29348-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.977798+0000 mon.f [INF] from='client.? 172.21.15.160:0/2504746142' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-29356-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.983440+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi160-29364-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:05.091 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.983995+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-29348-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:03.991118+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-29356-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.077475+0000 mon.f [INF] from='client.? 172.21.15.160:0/2693187654' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.090820+0000 mon.h [INF] from='client.? 172.21.15.160:0/2379542388' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.114413+0000 client.admin [INF] onexx 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.128287+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.092 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.129102+0000 mon.f [INF] from='client.? 172.21.15.160:0/2693187654' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.093 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.175943+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-17T16:29:05.093 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.199100+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.093 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.201233+0000 mon.f [INF] from='client.? 172.21.15.160:0/2693187654' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-29372-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:05.093 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.202303+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-29372-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:05.093 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.968298+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.968665+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi160-29348-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.968904+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-29356-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.969041+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi160-29372-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:04.992138+0000 mon.f [INF] from='client.? 172.21.15.160:0/2693187654' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-29372-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.008341+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-29372-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-29372-24"} : dispatch 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.008715+0000 mon.e [INF] from='client.? 172.21.15.160:0/3212220088' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-30138-1"} : dispatch 2024-09-17T16:29:05.094 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.017483+0000 mon.f [INF] from='client.? 172.21.15.160:0/314314766' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29504-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.095 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.018552+0000 mon.g [INF] from='client.? 172.21.15.160:0/2173972031' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29884-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.095 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.018774+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29504-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:05.095 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.019022+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi160-30138-1"} : dispatch 2024-09-17T16:29:07.038 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.020440+0000 mon.h [INF] from='client.? 172.21.15.160:0/4294245692' entity='client.admin' open_pools_parallel: process_1_[29939]: starting. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: creating pool ceph_test_rados_open_pools_parallel.smithi160-29923 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: created object 0... 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: created object 25... 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: created object 49... 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: finishing. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_1_[29939]: shutting down. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[29940]: starting. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[29940]: rados_pool_create. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[29940]: rados_ioctx_create. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_2_[29940]: shutting down. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: starting. 2024-09-17T16:29:07.039 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: creating pool ceph_test_rados_open_pools_parallel.smithi160-29923 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: created object 0... 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: created object 25... 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: created object 49... 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: finishing. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_3_[30796]: shutting down. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[30797]: starting. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[30797]: rados_pool_create. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[30797]: rados_ioctx_create. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: process_4_[30797]: shutting down. 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-17T16:29:07.040 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******************************* 2024-09-17T16:29:07.041 INFO:tasks.workunit.client.0.smithi160.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-17T16:29:07.088 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: starting. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: creating pool ceph_test_rados_delete_pools_parallel.smithi160-29933 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: created object 0... 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: created object 25... 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: created object 49... 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: finishing. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_1_[30120]: shutting down. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[30121]: starting. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[30121]: deleting pool ceph_test_rados_delete_pools_parallel.smithi160-29933 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_2_[30121]: shutting down. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: starting. 2024-09-17T16:29:07.089 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: creating pool ceph_test_rados_delete_pools_parallel.smithi160-29933 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: created object 0... 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: created object 25... 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: created object 49... 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: finishing. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_3_[30791]: shutting down. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: starting. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: listing objects. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: listed object 0... 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: listed object 25... 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: saw 50 objects 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_5_[30793]: shutting down. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[30792]: starting. 2024-09-17T16:29:07.090 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[30792]: deleting pool ceph_test_rados_delete_pools_parallel.smithi160-29933 2024-09-17T16:29:07.091 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: process_4_[30792]: shutting down. 2024-09-17T16:29:07.091 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-17T16:29:07.091 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******************************* 2024-09-17T16:29:07.091 INFO:tasks.workunit.client.0.smithi160.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: Running main() from gmock_main.cc 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] Global test environment set-up. 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (2039 ms) 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-09-17T16:29:08.039 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2028 ms) 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3016 ms) 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] 3 tests from NeoRadosCmd (7083 ms total) 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [----------] Global test environment tear-down 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [==========] 3 tests from 1 test suite ran. (7083 ms total) 2024-09-17T16:29:08.040 INFO:tasks.workunit.client.0.smithi160.stdout: cmd: [ PASSED ] 3 tests. 2024-09-17T16:29:08.105 INFO:tasks.workunit.client.0.smithi160.stdout:cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-29682-1"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:05.021408+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi160-29682-1"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.093758+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.995613+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi160-29372-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi160-29372-24"}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.995852+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi160-29682-1"}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.996055+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi160-30138-1"}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.996204+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi160-29356-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.996330+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi160-29504-4", "tierpool": "test-rados-api-smithi160-29504-4-cache"}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:06.996493+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi160-30237-1"}]': finished 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.014523+0000 mon.d [INF] from='client.? 172.21.15.160:0/2289470623' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-30237-1"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.022329+0000 mon.f [INF] from='client.? 172.21.15.160:0/314314766' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-29504-4", "overlaypool": "test-rados-api-smithi160-29504-4-cache"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.025839+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-29504-4", "overlaypool": "test-rados-api-smithi160-29504-4-cache"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.026356+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-30237-1"} : dispatch 2024-09-17T16:29:08.106 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.027473+0000 mon.f [INF] from='client.? 172.21.15.160:0/2504746142' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-29356-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T16:29:08.107 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.029649+0000 mon.f [INF] from='client.? 172.21.15.160:0/1426024717' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-29348-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.107 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.030849+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-29356-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-17T16:29:08.107 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.031125+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-29348-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.048109+0000 mon.e [INF] from='client.? 172.21.15.160:0/3097457211' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.049649+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.127 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.129252+0000 mon.d [INF] from='client.? 172.21.15.160:0/2513411200' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-30138-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.157507+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-30138-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.335408+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.0"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.336867+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.1"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.337912+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.2"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.339556+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.3"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.341180+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.4"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.342460+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.5"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.343448+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.6"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.344487+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.7"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.345536+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.8"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.346581+0000 mon.d [INF] from='client.? 172.21.15.160:0/2933091897' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "11.9"} : dispatch 2024-09-17T16:29:08.128 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.394877+0000 mon.g [WRN] Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:07.395675+0000 mon.g [WRN] Health check failed: 2/9 mons down, quorum f,b,g,c,h,d,e (MON_DOWN) 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.010632+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi160-29393-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi160-29393-10"}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.010910+0000 mon.g [INF] from='client.5077 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi160-29425-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi160-29425-10"}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.011155+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi160-29504-4", "overlaypool": "test-rados-api-smithi160-29504-4-cache"}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.011529+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi160-30237-1"}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.011778+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi160-29356-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.012021+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi160-29348-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.012196+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29518-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.012312+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi160-30138-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.029680+0000 mon.f [INF] from='client.? 172.21.15.160:0/314314766' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-29504-4-cache", "mode": "writeback"} : dispatch 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.032148+0000 mon.f [INF] from='client.? 172.21.15.160:0/3904542203' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29884-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.033179+0000 mon.f [INF] from='client.? 172.21.15.160:0/327800936' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:08.129 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.036516+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi160-29504-4-cache", "mode": "writeback"} : dispatch 2024-09-17T16:29:08.130 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.037257+0000 mon.h [INF] from='client.? 172.21.15.160:0/3923822698' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:08.130 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.037269+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi160-29884-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:08.130 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.037816+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi160-29364-16"} : dispatch 2024-09-17T16:29:08.130 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.039299+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi160-29612-7"} : dispatch 2024-09-17T16:29:08.130 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.040720+0000 mon.d [INF] from='client.? 172.21.15.160:0/2513411200' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-30138-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-30138-2"} : dispatch 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.049878+0000 mon.c [INF] from='client.? 172.21.15.160:0/1850172674' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-29682-4","app": "rados","yes_i_rea api_io_pp: Running main() from gmock_main.cc 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: seed 29372 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-17T16:29:08.220 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (244 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (11 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (1175 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (7 ms) 2024-09-17T16:29:08.221 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (5 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (8 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (7 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (5 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (6 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (8 ms) 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-17T16:29:08.222 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (5 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (9 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (14 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (4 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (15 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (3 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (1551 ms total) 2024-09-17T16:29:08.223 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1174 ms) 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (16 ms) 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-17T16:29:08.224 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: Running main() from gmock_main.cc 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] Global test environment set-up. 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (218 ms) 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-17T16:29:10.054 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: no timeout :/ 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (44 ms) 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (8 ms) 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (4 ms) 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-17T16:29:10.055 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (4 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.RmXattr (23 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-09-17T16:29:10.056 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 14 tests from LibRadosIo (327 ms total) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1174 ms) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (32 ms) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (10 ms) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (65 ms) 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-17T16:29:10.057 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (12 ms) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (9 ms) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (10 ms) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (22 ms) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (12 ms) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1353 ms total) 2024-09-17T16:29:10.058 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: 2024-09-17T16:29:10.059 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [----------] Global test environment tear-down 2024-09-17T16:29:10.059 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [==========] 24 tests from 2 test suites ran. (9382 ms total) 2024-09-17T16:29:10.059 INFO:tasks.workunit.client.0.smithi160.stdout: api_io: [ PASSED ] 24 tests. 2024-09-17T16:29:10.064 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: Running main() from gmock_main.cc 2024-09-17T16:29:10.064 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-09-17T16:29:10.064 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] Global test environment set-up. 2024-09-17T16:29:10.064 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.Stat (213 ms) 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (39 ms) 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.StatNS (14 ms) 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-17T16:29:10.065 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (214 ms) 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 5 tests from LibRadosStat (480 ms total) 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1155 ms) 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-17T16:29:10.066 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (44 ms) 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (7 ms) 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1206 ms total) 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: 2024-09-17T16:29:10.067 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [----------] Global test environment tear-down 2024-09-17T16:29:10.068 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (9284 ms total) 2024-09-17T16:29:10.068 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat: [ PASSED ] 9 tests. 2024-09-17T16:29:10.082 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: seed 29620 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (198 ms) 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (34 ms) 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-17T16:29:10.083 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (13 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (250 ms total) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1178 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (10 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (18 ms) 2024-09-17T16:29:10.084 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1207 ms total) 2024-09-17T16:29:10.085 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: 2024-09-17T16:29:10.085 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-17T16:29:10.085 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9296 ms total) 2024-09-17T16:29:10.085 INFO:tasks.workunit.client.0.smithi160.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: Running main() from gmock_main.cc 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] Global test environment set-up. 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] 3 tests from NeoradosList 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjects (2937 ms) 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3041 ms) 2024-09-17T16:29:11.073 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4040 ms) 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] 3 tests from NeoradosList (10018 ms total) 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: [----------] Global test environment tear-down 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: [==========] 3 tests from 1 test suite ran. (10018 ms total) 2024-09-17T16:29:11.074 INFO:tasks.workunit.client.0.smithi160.stdout: list: [ PASSED ] 3 tests. 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Running main() from gmock_main.cc 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] Global test environment set-up. 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: handle_notify cookie 94735321166736 notify_id 98784247808 notifier_gid 4697 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2876 ms) 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Trying... 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: handle_notify cookie 94735333253280 notify_id 111669149698 notifier_gid 4751 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Waiting for 3.000000000s 2024-09-17T16:29:11.076 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Timed out. 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Flushing... 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: Flushed... 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7092 ms) 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9968 ms total) 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [----------] Global test environment tear-down 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9968 ms total) 2024-09-17T16:29:11.077 INFO:tasks.workunit.client.0.smithi160.stdout: watch_notify: [ PASSED ] 2 tests. 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout:lly_mean_it": true} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.050121+0000 mon.d [INF] from='client.? 172.21.15.160:0/2217910826' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.050779+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi160-30138-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi160-30138-2"} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.054440+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi160-29620-7"} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.058088+0000 mon.c [INF] from='client.4846 172.21.15.160:0/1850172674' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-29682-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.061193+0000 mon.g [INF] from='client.4846 ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi160-29682-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.104107+0000 mon.h [INF] from='client.? 172.21.15.160:0/2379542388' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.105230+0000 client.admin [INF] threexx 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: got: 2024-09-17T16:29:08.110926+0000 mon.g [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7149 ms) 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (10322 ms total) 2024-09-17T16:29:11.143 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: 2024-09-17T16:29:11.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-17T16:29:11.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (10322 ms total) 2024-09-17T16:29:11.144 INFO:tasks.workunit.client.0.smithi160.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-17T16:29:12.639 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1208 ms) 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (19 ms) 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1227 ms total) 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (153 ms) 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3376 ms) 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-17T16:29:12.640 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (7 ms) 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (8 ms) 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693644144 notify_id 141733920770 notifier_gid 4846 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693644144 notify_id 141733920771 notifier_gid 4846 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (7 ms) 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693661696 notify_id 141733920769 notifier_gid 4846 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (7 ms) 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693661696 notify_id 141733920771 notifier_gid 4846 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (7 ms) 2024-09-17T16:29:12.641 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 141733920772 notifier_gid 4846 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 141733920772 notifier_gid 4846 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (7 ms) 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 141733920773 notifier_gid 4846 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (6 ms) 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-17T16:29:12.642 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: trying... 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 141733920774 notifier_ ec_io: Running main() from gmock_main.cc 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] Global test environment set-up. 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (6012 ms) 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6415 ms) 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12427 ms total) 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [----------] Global test environment tear-down 2024-09-17T16:29:13.442 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12427 ms total) 2024-09-17T16:29:13.443 INFO:tasks.workunit.client.0.smithi160.stdout: ec_io: [ PASSED ] 2 tests. 2024-09-17T16:29:13.456 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: Running main() from gmock_main.cc 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] Global test environment set-up. 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (186 ms) 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockShared (11 ms) 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1134 ms) 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1009 ms) 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (11 ms) 2024-09-17T16:29:13.457 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.Unlock (10 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (21 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (11 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLock (2393 ms total) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1170 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (11 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1069 ms) 2024-09-17T16:29:13.458 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1031 ms) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (16 ms) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (13 ms) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (23 ms) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (10 ms) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3343 ms total) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [----------] Global test environment tear-down 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (12762 ms total) 2024-09-17T16:29:13.459 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-17T16:29:13.465 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-17T16:29:13.465 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-17T16:29:13.465 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-17T16:29:13.465 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: seed 29425 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (182 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (4 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1161 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1010 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (5 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-09-17T16:29:13.466 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (10 ms) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (4 ms) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2381 ms total) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1149 ms) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (11 ms) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1069 ms) 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-17T16:29:13.467 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1034 ms) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (11 ms) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (17 ms) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (20 ms) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (11 ms) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3322 ms total) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (12746 ms total) 2024-09-17T16:29:13.468 INFO:tasks.workunit.client.0.smithi160.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: Running main() from gmock_main.cc 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] Global test environment set-up. 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolList (1926 ms) 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (1998 ms) 2024-09-17T16:29:14.483 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2016 ms) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4046 ms) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1365 ms) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2041 ms) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] 6 tests from NeoRadosPools (13392 ms total) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [----------] Global test environment tear-down 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [==========] 6 tests from 1 test suite ran. (13392 ms total) 2024-09-17T16:29:14.484 INFO:tasks.workunit.client.0.smithi160.stdout: pool: [ PASSED ] 6 tests. 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (7 ms) 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (9 ms) 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-17T16:29:18.521 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (9 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (9 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (6 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (4 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (5 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (25 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (6177 ms) 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-17T16:29:18.522 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (1123 ms) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (9 ms) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (7 ms) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (8 ms) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (8602 ms total) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (17844 ms total) 2024-09-17T16:29:18.523 INFO:tasks.workunit.client.0.smithi160.stdout: api_io_pp: [ PASSED ] 40 tests. 2024-09-17T16:29:20.568 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify_test_cb 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (204 ms) 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94588485421440 err -107 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1068 ms) 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-17T16:29:20.569 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94588485439888 err -107 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1019 ms) 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215104 cookie 94588485439888 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (14 ms) 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215104 cookie 94588485439888 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (9 ms) 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215105 cookie 94588485439888 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (10 ms) 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215104 cookie 94588485458624 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215104 cookie 94588485481952 2024-09-17T16:29:20.570 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (19 ms) 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 103079215106 cookie 94588485481952 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 107374182403 cookie 94588485481952 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3014 ms) 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94588485481952 err -107 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_cb from 4673 notify_id 137438953474 cookie 94588485481952 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5021 ms) 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94588485481952 err -107 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1013 ms) 2024-09-17T16:29:20.571 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11391 ms total) 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: watch_notify_test_cb 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1157 ms) 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1157 ms total) 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (19757 ms total) 2024-09-17T16:29:20.572 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-17T16:29:24.457 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: Running main() from gmock_main.cc 2024-09-17T16:29:24.457 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-09-17T16:29:24.457 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] Global test environment set-up. 2024-09-17T16:29:24.457 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6983 ms) 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (6413 ms) 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9997 ms) 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] 3 tests from NeoradosECList (23393 ms total) 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [----------] Global test environment tear-down 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (23393 ms total) 2024-09-17T16:29:24.458 INFO:tasks.workunit.client.0.smithi160.stdout: ec_list: [ PASSED ] 3 tests. 2024-09-17T16:29:26.481 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: Running main() from gmock_main.cc 2024-09-17T16:29:26.481 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-09-17T16:29:26.481 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] Global test environment set-up. 2024-09-17T16:29:26.481 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-09-17T16:29:26.481 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2861 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3103 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (2975 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2386 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3048 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3058 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (7921 ms) 2024-09-17T16:29:26.482 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (25353 ms total) 2024-09-17T16:29:26.483 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: 2024-09-17T16:29:26.483 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [----------] Global test environment tear-down 2024-09-17T16:29:26.483 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (25353 ms total) 2024-09-17T16:29:26.483 INFO:tasks.workunit.client.0.smithi160.stdout: write_operations: [ PASSED ] 7 tests. 2024-09-17T16:29:26.484 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-17T16:29:26.484 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-17T16:29:26.484 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (3098 ms) 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3070 ms) 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3024 ms) 2024-09-17T16:29:26.485 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2379 ms) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3044 ms) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3059 ms) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (7935 ms) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (25609 ms total) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (25609 ms total) 2024-09-17T16:29:26.486 INFO:tasks.workunit.client.0.smithi160.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-17T16:29:28.501 INFO:tasks.workunit.client.0.smithi160.stdout:gid 4846 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: timed out 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushed 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: trying... 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 154618822661 notifier_gid 4846 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: timed out 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushed 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3009 ms) 2024-09-17T16:29:28.502 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: List watches 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 167503724546 notifier_gid 4846 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 done 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: watch_check 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: unwatch2 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: done 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3118 ms) 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: List watches 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: handle_notify cookie 93895693665248 notify_id 180388626435 notifier_gid 4846 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: notify2 done 2024-09-17T16:29:28.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: watch_check 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: unwatch2 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: flushing 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: done 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3069 ms) 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (15783 ms total) 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (27685 ms total) 2024-09-17T16:29:28.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] Global test environment set-up. 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5153 ms) 2024-09-17T16:29:28.607 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.RegisterLate (124 ms) 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: cluster: 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: id: 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: health: HEALTH_WARN 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 12 pool(s) do not have an application enabled 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: services: 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mon: 9 daemons, quorum f,b,g,c,h,d,e (age 29s), out of quorum: a, i 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mgr: x(active, since 78s) 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: osd: 6 osds: 6 up (since 73s), 6 in (since 82s) 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-17T16:29:28.608 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: data: 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pools: 24 pools, 689 pgs 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: objects: 25 objects, 578 KiB 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: usage: 166 MiB used, 540 GiB / 540 GiB avail 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pgs: 60.668% pgs unknown 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 18.433% pgs not active 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 418 unknown 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 144 active+clean 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 127 creating+peering 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: io: 2024-09-17T16:29:28.609 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: client: 0 B/s rd, 10 KiB/s wr, 0 op/s rd, 17 op/s wr 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: cluster: 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: id: 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: health: HEALTH_WARN 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2/9 mons down, quorum f,b,g,c,h,d,e 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 13 pool(s) do not have an application enabled 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: services: 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mon: 9 daemons, quorum f,b,g,c,h,d,e (age 31s), out of quorum: a, i 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: mgr: x(active, since 80s) 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: osd: 6 osds: 6 up (since 75s), 6 in (since 84s) 2024-09-17T16:29:28.610 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: data: 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pools: 25 pools, 649 pgs 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: objects: 56 objects, 49 MiB 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: usage: 203 MiB used, 540 GiB / 540 GiB avail 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: pgs: 39.753% pgs unknown 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 14.330% pgs not active 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 298 active+clean 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 258 unknown 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 93 creating+peering 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.611 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: io: 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: client: 12 MiB/s rd, 12 MiB/s wr, 255 op/s rd, 288 op/s wr 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2383 ms) 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ OK ] LibRadosService.Status (20051 ms) 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] 4 tests from LibRadosService (27711 ms total) 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [----------] Global test environment tear-down 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27711 ms total) 2024-09-17T16:29:28.612 INFO:tasks.workunit.client.0.smithi160.stdout: api_service: [ PASSED ] 4 tests. 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5123 ms) 2024-09-17T16:29:31.404 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (212 ms) 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20054 ms) 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: attempt 0 of 20 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5140 ms) 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30529 ms total) 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30529 ms total) 2024-09-17T16:29:31.405 INFO:tasks.workunit.client.0.smithi160.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-17T16:29:33.488 INFO:tasks.workunit.client.0.smithi160.stdout: misc: Running main() from gmock_main.cc 2024-09-17T16:29:33.488 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-09-17T16:29:33.488 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] Global test environment set-up. 2024-09-17T16:29:33.488 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-09-17T16:29:33.488 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Version (1928 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (2016 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongName (3026 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3039 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2383 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3042 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Exec (3040 ms) 2024-09-17T16:29:33.489 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (2885 ms) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3030 ms) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3031 ms) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2052 ms) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2942 ms) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] 12 tests from NeoRadosMisc (32414 ms total) 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [----------] Global test environment tear-down 2024-09-17T16:29:33.490 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [==========] 12 tests from 1 test suite ran. (32414 ms total) 2024-09-17T16:29:33.491 INFO:tasks.workunit.client.0.smithi160.stdout: misc: [ PASSED ] 12 tests. 2024-09-17T16:29:37.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: Running main() from gmock_main.cc 2024-09-17T16:29:37.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-17T16:29:37.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] Global test environment set-up. 2024-09-17T16:29:37.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-17T16:29:37.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2244 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3038 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3025 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3056 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (2377 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5049 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4932 ms) 2024-09-17T16:29:37.498 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5027 ms) 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7976 ms) 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] 9 tests from LibRadosPools (36724 ms total) 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [----------] Global test environment tear-down 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (36724 ms total) 2024-09-17T16:29:37.499 INFO:tasks.workunit.client.0.smithi160.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-17T16:29:42.465 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: Running main() from gmock_main.cc 2024-09-17T16:29:42.465 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-09-17T16:29:42.465 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] Global test environment set-up. 2024-09-17T16:29:42.465 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-09-17T16:29:42.465 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2895 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3056 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3035 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (2369 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3053 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3037 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3035 ms) 2024-09-17T16:29:42.466 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2901 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2971 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3073 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2969 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3025 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2983 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2957 ms) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41360 ms total) 2024-09-17T16:29:42.467 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: 2024-09-17T16:29:42.468 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [----------] Global test environment tear-down 2024-09-17T16:29:42.468 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41360 ms total) 2024-09-17T16:29:42.468 INFO:tasks.workunit.client.0.smithi160.stdout: read_operations: [ PASSED ] 14 tests. 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: Running main() from gmock_main.cc 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] Global test environment set-up. 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] 14 tests from NeoRadosIo 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Limits (2963 ms) 2024-09-17T16:29:42.490 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3058 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.ReadOp (3036 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.SparseRead (2374 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3042 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3038 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3032 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (2915 ms) 2024-09-17T16:29:42.491 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2973 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Trunc (3084 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.Remove (2959 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3040 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.RmXattr (2958 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2984 ms) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] 14 tests from NeoRadosIo (41456 ms total) 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: 2024-09-17T16:29:42.492 INFO:tasks.workunit.client.0.smithi160.stdout: io: [----------] Global test environment tear-down 2024-09-17T16:29:42.493 INFO:tasks.workunit.client.0.smithi160.stdout: io: [==========] 14 tests from 1 test suite ran. (41456 ms total) 2024-09-17T16:29:42.493 INFO:tasks.workunit.client.0.smithi160.stdout: io: [ PASSED ] 14 tests. 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] Global test environment set-up. 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-17T16:29:43.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:00.820+0000 7fc54a736880 -1 failed for service _ceph-mon._tcp 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:00.824+0000 7fc54a736880 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (83 ms) 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5012 ms) 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5095 ms total) 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 0x555fa43bb050 2024-09-17T16:29:43.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 0x555fa4e78180 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: started 2 aios 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: waiting 0x555fa43bb050 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: waiting 0x555fa4e78180 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: done. 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (5245 ms) 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (5245 ms total) 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.Exec (110 ms) 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-17T16:29:43.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (30 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4924 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (25668 ms) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] 8 tests from LibRadosMisc (30734 ms total) 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [----------] Global test environment tear-down 2024-09-17T16:29:43.507 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (42743 ms total) 2024-09-17T16:29:43.508 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (1989 ms) 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2071 ms) 2024-09-17T16:29:55.503 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (1999 ms) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2014 ms) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8073 ms total) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4159 ms) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4059 ms) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4895 ms) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13113 ms total) 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-17T16:29:55.504 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2973 ms) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2078 ms) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2004 ms) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1961 ms) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9016 ms total) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4031 ms) 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-17T16:29:55.505 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3967 ms) 2024-09-17T16:29:55.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7998 ms total) 2024-09-17T16:29:55.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: 2024-09-17T16:29:55.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-17T16:29:55.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (54737 ms total) 2024-09-17T16:29:55.506 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-09-17T16:30:09.109 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-17T16:30:09.110 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T16:30:09.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T16:30:09.113 INFO:tasks.ceph.mon.a:Started 2024-09-17T16:30:09.113 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-17T16:30:09.113 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-17T16:30:09.113 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-17T16:30:09.116 INFO:tasks.ceph.mon.i:Started 2024-09-17T16:30:09.117 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-17T16:30:09.117 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-17T16:30:09.117 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-17T16:30:09.117 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-17T16:30:09.117 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:30:09.204 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-09-17T16:30:09.196+0000 7f12ddd0be00 -1 mon.a@-1(???).log v0 log_external_backlog rewinding external_log_to from 28 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:30:15.598 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-09-17T16:30:15.598 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: seed 29598 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (1952 ms) 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2062 ms) 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2003 ms) 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2016 ms) 2024-09-17T16:30:15.599 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3405 ms) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11438 ms total) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4165 ms) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3792 ms) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6078 ms) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3979 ms) 2024-09-17T16:30:15.600 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2161 ms) 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: ./src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi160-29598-7 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17822 ms) 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37997 ms total) 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-17T16:30:15.601 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (36 ms) 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5054 ms) 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5091 ms total) 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3016 ms) 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (1986 ms) 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-17T16:30:15.602 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (7131 ms) 2024-09-17T16:30:19.476 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_ api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-17T16:30:19.476 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-17T16:30:19.476 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-17T16:30:19.476 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-17T16:30:19.476 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: seed 29470 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (27 ms) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (167 ms) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (28 ms) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (19 ms) 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-17T16:30:19.477 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (1 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (10 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-17T16:30:19.478 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (6 ms) 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (4056 ms) 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (459 ms) 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: done waiting, doing copies 2024-09-17T16:30:19.479 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: done waiting 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (61198 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (13 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (6 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (10472 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-17T16:30:19.480 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-17T16:30:24.118 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-17T16:30:24.119 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-17T16:30:24.119 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-17T16:30:24.119 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-17T16:30:24.119 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:30:24.120 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:30:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: get_monmap_and_config 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: build_initial_monmap 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: monmap: 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:24.248852+0000 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:24.248852+0000 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:24.251 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:24.252 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 Processor -- start 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 -- start start 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: init 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient: _add_conns ranks=[9,4,8,1,0,3,6,7,2,5,10] 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-a con 0x7fb5840a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-c con 0x7fb5840a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-e con 0x7fb5841396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb5840a3fc0 con 0x7fb5840a68e0 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb5840a63c0 con 0x7fb5840a4910 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb5840a7740 con 0x7fb5841396a0 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb58b6eb640 10 monclient(hunting): authenticate will time out at 1107.514621s 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:59276/0 (socket says 172.21.15.105:59276) 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 -- 172.21.15.105:0/2985058116 learned_addr learned my addr 172.21.15.105:0/2985058116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36918/0 (socket says 172.21.15.105:36918) 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 10 monclient(hunting): get_auth_request con 0x7fb5840a4910 auth_method 0 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 10 monclient(hunting): get_auth_request con 0x7fb5841396a0 auth_method 0 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 10 monclient(hunting): get_auth_request con 0x7fb5840a68e0 auth_method 0 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 msgr2=0x7fb584139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 msgr2=0x7fb584139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 msgr2=0x7fb5840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 msgr2=0x7fb5840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.244+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 10 monclient(hunting): get_auth_request con 0x7fb5840a4910 auth_method 0 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 10 monclient(hunting): get_auth_request con 0x7fb5841396a0 auth_method 0 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 10 monclient(hunting): get_auth_request con 0x7fb5840a68e0 auth_method 0 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 msgr2=0x7fb5840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 msgr2=0x7fb5840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 msgr2=0x7fb584139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 msgr2=0x7fb584139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.448+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): get_auth_request con 0x7fb5840a4910 auth_method 0 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> v2:172.21.15.105:3300/0 conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): get_auth_request con 0x7fb5841396a0 auth_method 0 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): get_auth_request con 0x7fb5840a68e0 auth_method 0 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient(hunting): handle_auth_done global_id 34110 payload 274 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 msgr2=0x7fb584139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient: _finish_hunting 0 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 monclient: found mon.noname-a 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589487640 1 -- 172.21.15.105:0/2985058116 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58406ebc0 con 0x7fb5840a68e0 2024-09-17T16:30:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb588c86640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.852+0000 7fb589c88640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:24.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb589487640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5840a68e0 0x7fb5840a6cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb578000ee0 tx=0x7fb57801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66546410c39b0f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:24.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 1 -- 172.21.15.105:0/2985058116 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb5780ab070 con 0x7fb5840a68e0 2024-09-17T16:30:24.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:24.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:24.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: dump: 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:24.865 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 1 -- 172.21.15.105:0/2985058116 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5780a5ad0 con 0x7fb5840a68e0 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 5 monclient: authenticate success, global_id 34110 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 4 set_mon_vals no callback set 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 1 -- 172.21.15.105:0/2985058116 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb57800f040 con 0x7fb5840a68e0 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:24.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb577fff640 10 monclient: dump: 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:24.867 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:24.868 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:24.868 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 10 monclient: shutdown 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 1 -- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5840a68e0 msgr2=0x7fb5840a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.856+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5840a68e0 0x7fb5840a6cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb578000ee0 tx=0x7fb57801c900 comp rx=0 tx=0).stop 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- 172.21.15.105:0/2985058116 shutdown_connections 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb5841396a0 0x7fb584139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a4910 0x7fb5840a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/2985058116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5840a68e0 0x7fb5840a6cc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- 172.21.15.105:0/2985058116 >> 172.21.15.105:0/2985058116 conn(0x7fb584068ba0 msgr2=0x7fb584068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- 172.21.15.105:0/2985058116 shutdown_connections 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- 172.21.15.105:0/2985058116 wait complete. 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient: build_initial_monmap 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient: monmap: 2024-09-17T16:30:24.874 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:24.875 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 librados: starting msgr at 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 librados: starting objecter 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 Processor -- start 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- start start 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 librados: setting wanted keys 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 librados: calling monclient init 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient: init 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient: _add_conns ranks=[5,4,7,6,8,0,1,2,3] 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb5840a4910 0x7fb5841d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-h con 0x7fb5840a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-c con 0x7fb5840a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 0x7fb58413d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): picked mon.noname-i con 0x7fb5841396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb58406e630 con 0x7fb5840a68e0 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb5840a7af0 con 0x7fb5840a4910 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb584143320 con 0x7fb5841396a0 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb58b6eb640 10 monclient(hunting): authenticate will time out at 1108.130675s 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:59308/0 (socket says 172.21.15.105:59308) 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 -- 172.21.15.105:0/1001234827 learned_addr learned my addr 172.21.15.105:0/1001234827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): get_auth_request con 0x7fb5840a68e0 auth_method 0 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589487640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb5840a4910 0x7fb5841d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 0x7fb58413d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589487640 10 monclient(hunting): get_auth_request con 0x7fb5840a4910 auth_method 0 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589487640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589487640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589487640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 10 monclient(hunting): get_auth_request con 0x7fb5841396a0 auth_method 0 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient(hunting): handle_auth_done global_id 22468 payload 824 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 -- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 msgr2=0x7fb58413d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:24.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 0x7fb58413d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 -- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb5840a4910 msgr2=0x7fb5841d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb5840a4910 0x7fb5841d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient: _finish_hunting 0 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 monclient: found mon.noname-c 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 -- 172.21.15.105:0/1001234827 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5840a9e70 con 0x7fb5840a68e0 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb589c88640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 0x7fb58413d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb588c86640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007dd0 tx=0x7fb57c0115a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 10 client.?.objecter ms_handle_connect 0x7fb5840a68e0 2024-09-17T16:30:24.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb57c01a020 con 0x7fb5840a68e0 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.860+0000 7fb575ffb640 10 monclient: dump: 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:24.881 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:24.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 5 monclient: authenticate success, global_id 22468 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c011e60 con 0x7fb5840a68e0 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _renew_subs 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5841234b0 con 0x7fb5840a68e0 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb57c016d30 con 0x7fb5840a68e0 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb5777fe640 4 set_mon_vals no callback set 2024-09-17T16:30:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 monclient: dump: 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 client.22468.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _renew_subs 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58413fb20 con 0x7fb5840a68e0 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 1 librados: init done 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:24.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb584139b50 con 0x7fb5840a68e0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb57c01d020 con 0x7fb5840a68e0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 0x7fb568040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(96..96 src has 1..96) ==== 7569+0+0 (secure 0 0 0) 0x7fb57c012040 con 0x7fb5840a68e0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 client.22468.objecter ms_dispatch 0x7fb584000ba0 osd_map(96..96 src has 1..96) 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 3 client.22468.objecter handle_osd_map got epochs [96,96] > 0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 3 client.22468.objecter handle_osd_map decoding full epoch 96 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 20 client.22468.objecter dump_active .. 0 homeless 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb589487640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 0x7fb568040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb589487640 10 monclient: get_auth_request con 0x7fb56803e580 auth_method 0 2024-09-17T16:30:24.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb575ffb640 10 client.22468.objecter ms_handle_connect 0x7fb56803e580 2024-09-17T16:30:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.864+0000 7fb589487640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 0x7fb568040a40 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7fb5780b0000 tx=0x7fb5780a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.868+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb57c024030 con 0x7fb5840a68e0 2024-09-17T16:30:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.868+0000 7fb575ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:24.868+0000 7fb575ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb5840746d0 con 0x7fb5840a68e0 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb575ffb640 1 -- 172.21.15.105:0/1001234827 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3200 (secure 0 0 0) 0x7fb5840a9e70 con 0x7fb5840a68e0 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb575ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb575ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:25.119 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":26,"quorum":[0,2,3,4,6,7,8],"quorum_names":["a","b","g","c","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 librados: watch_flush enter 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 librados: watch_flush exit 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 20 client.22468.objecter shutdown clearing up homeless session... 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 client.22468.objecter successfully canceled tick 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 msgr2=0x7fb568040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 0x7fb568040a40 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7fb5780b0000 tx=0x7fb5780a7070 comp rx=0 tx=0).stop 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 10 monclient: shutdown 2024-09-17T16:30:25.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 msgr2=0x7fb5841d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007dd0 tx=0x7fb57c0115a0 comp rx=0 tx=0).stop 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 shutdown_connections 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb56803e580 0x7fb568040a40 unknown :-1 s=CLOSED pgs=3337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5841396a0 0x7fb58413d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5840a68e0 0x7fb5841d1a80 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 --2- 172.21.15.105:0/1001234827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb5840a4910 0x7fb5841d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 >> 172.21.15.105:0/1001234827 conn(0x7fb584068ba0 msgr2=0x7fb5840a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 shutdown_connections 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 -- 172.21.15.105:0/1001234827 wait complete. 2024-09-17T16:30:25.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:25.112+0000 7fb58b6eb640 1 librados: shutdown 2024-09-17T16:30:28.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:30:28.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: get_monmap_and_config 2024-09-17T16:30:28.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: build_initial_monmap 2024-09-17T16:30:28.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: monmap: 2024-09-17T16:30:28.267 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:28.265635+0000 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:28.265635+0000 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:28.268 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:28.269 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 Processor -- start 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 -- start start 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: init 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient: _add_conns ranks=[3,5,9,4,1,0,2,6,7,10,8] 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff044138870 0x7ff044138c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-g con 0x7ff044138870 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff04409ea50 0x7ff0440a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-h con 0x7ff04409ea50 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0440a4c90 0x7ff0440a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-a con 0x7ff0440a4c90 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff04409dd90 con 0x7ff0440a4c90 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff04409e070 con 0x7ff044138870 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff04409e350 con 0x7ff04409ea50 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04c0ca640 10 monclient(hunting): authenticate will time out at 1111.531528s 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0440a4c90 0x7ff0440a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0440a4c90 0x7ff0440a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51904/0 (socket says 172.21.15.105:51904) 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 1 -- 172.21.15.105:0/3933736354 learned_addr learned my addr 172.21.15.105:0/3933736354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): get_auth_request con 0x7ff0440a4c90 auth_method 0 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.260+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049665640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff04409ea50 0x7ff0440a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049e66640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff044138870 0x7ff044138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049665640 10 monclient(hunting): get_auth_request con 0x7ff04409ea50 auth_method 0 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049665640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049665640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049665640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049e66640 10 monclient(hunting): get_auth_request con 0x7ff044138870 auth_method 0 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049e66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049e66640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff049e66640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 10 monclient(hunting): handle_auth_done global_id 34119 payload 274 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 -- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff04409ea50 msgr2=0x7ff0440a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff04409ea50 0x7ff0440a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 -- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff044138870 msgr2=0x7ff044138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff044138870 0x7ff044138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 10 monclient: _finish_hunting 0 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 monclient: found mon.noname-a 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 -- 172.21.15.105:0/3933736354 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04406cd40 con 0x7ff0440a4c90 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04a667640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440a4c90 0x7ff0440a5140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009e50 tx=0x7ff03c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a661460fdf13c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 1 -- 172.21.15.105:0/3933736354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c0090d0 con 0x7ff0440a4c90 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: dump: 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:28.281 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: _finish_auth 0 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: _check_auth_tickets 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:28.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 1 -- 172.21.15.105:0/3933736354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c0a59b0 con 0x7ff0440a4c90 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 5 monclient: authenticate success, global_id 34119 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 1 -- 172.21.15.105:0/3933736354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c0a5ba0 con 0x7ff0440a4c90 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 4 set_mon_vals no callback set 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff048e64640 10 monclient: dump: 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.283 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 10 monclient: shutdown 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:28.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 -- 172.21.15.105:0/3933736354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440a4c90 msgr2=0x7ff0440a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440a4c90 0x7ff0440a5140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009e50 tx=0x7ff03c01c8f0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 -- 172.21.15.105:0/3933736354 shutdown_connections 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0440a4c90 0x7ff0440a5140 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff04409ea50 0x7ff0440a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/3933736354 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff044138870 0x7ff044138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 -- 172.21.15.105:0/3933736354 >> 172.21.15.105:0/3933736354 conn(0x7ff0440a5840 msgr2=0x7ff0440a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 -- 172.21.15.105:0/3933736354 shutdown_connections 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 1 -- 172.21.15.105:0/3933736354 wait complete. 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 10 monclient: build_initial_monmap 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.264+0000 7ff04c0ca640 10 monclient: monmap: 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:28.285 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 librados: starting msgr at 2024-09-17T16:30:28.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 librados: starting objecter 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 Processor -- start 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- start start 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 librados: setting wanted keys 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 librados: calling monclient init 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: init 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _add_conns ranks=[7,4,0,6,1,5,3,2,8] 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff04409ea50 0x7ff04413c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-i con 0x7ff04409ea50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff0440a4c90 0x7ff04413c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-c con 0x7ff0440a4c90 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): picked mon.noname-a con 0x7ff044138870 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff044143500 con 0x7ff044138870 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff044143680 con 0x7ff0440a4c90 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff044143800 con 0x7ff04409ea50 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient(hunting): authenticate will time out at 1111.536728s 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51914/0 (socket says 172.21.15.105:51914) 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 -- 172.21.15.105:0/1944005348 learned_addr learned my addr 172.21.15.105:0/1944005348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): get_auth_request con 0x7ff044138870 auth_method 0 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff0440a4c90 0x7ff04413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): get_auth_request con 0x7ff0440a4c90 auth_method 0 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049665640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:28.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff04409ea50 0x7ff04413c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 10 monclient(hunting): get_auth_request con 0x7ff04409ea50 auth_method 0 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient(hunting): handle_auth_done global_id 34128 payload 824 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 -- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff04409ea50 msgr2=0x7ff04413c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff04409ea50 0x7ff04413c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 -- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff0440a4c90 msgr2=0x7ff04413c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff0440a4c90 0x7ff04413c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient: _finish_hunting 0 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 monclient: found mon.noname-a 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 -- 172.21.15.105:0/1944005348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04413fb40 con 0x7ff044138870 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04a667640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff03c01c720 tx=0x7ff03c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 client.?.objecter ms_handle_connect 0x7ff044138870 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c003e10 con 0x7ff044138870 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: dump: 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:28.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 5 monclient: authenticate success, global_id 34128 2024-09-17T16:30:28.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c0bc070 con 0x7ff044138870 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _renew_subs 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04413fe20 con 0x7ff044138870 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c0a5d70 con 0x7ff044138870 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 client.34128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff037fff640 4 set_mon_vals no callback set 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 monclient: dump: 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:28.293 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:28.294 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _renew_subs 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0440a7be0 con 0x7ff044138870 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 librados: init done 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff03c0039a0 con 0x7ff044138870 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0440a4750 con 0x7ff044138870 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 0x7ff028040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:28.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9774+0+0 (secure 0 0 0) 0x7ff03c0eb7c0 con 0x7ff044138870 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 10 client.34128.objecter ms_dispatch 0x7ff044000ba0 osd_map(99..99 src has 1..99) 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 3 client.34128.objecter handle_osd_map got epochs [99,99] > 0 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 3 client.34128.objecter handle_osd_map decoding full epoch 99 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff0367fc640 20 client.34128.objecter dump_active .. 0 homeless 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 0x7ff028040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.268+0000 7ff049e66640 10 monclient: get_auth_request con 0x7ff02803e550 auth_method 0 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.272+0000 7ff0367fc640 10 client.34128.objecter ms_handle_connect 0x7ff02803e550 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.272+0000 7ff049e66640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 0x7ff028040a10 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b9d0 tx=0x7ff038008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.272+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff03c016600 con 0x7ff044138870 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.272+0000 7ff0367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:28.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.272+0000 7ff0367fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:28.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.532+0000 7ff04c0ca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:30:28.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.532+0000 7ff04c0ca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:28.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.532+0000 7ff04c0ca640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:28.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.532+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff04406e230 con 0x7ff044138870 2024-09-17T16:30:28.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff0367fc640 1 -- 172.21.15.105:0/1944005348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3200 (secure 0 0 0) 0x7ff03c0e8e50 con 0x7ff044138870 2024-09-17T16:30:28.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff0367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:28.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff0367fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:28.542 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":26,"quorum":[0,2,3,4,6,7,8],"quorum_names":["a","b","g","c","d","i","e"],"quorum_leader_name":"g","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 10 librados: watch_flush enter 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 10 librados: watch_flush exit 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 20 client.34128.objecter shutdown clearing up homeless session... 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 10 client.34128.objecter successfully canceled tick 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 msgr2=0x7ff028040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 0x7ff028040a10 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b9d0 tx=0x7ff038008040 comp rx=0 tx=0).stop 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 10 monclient: shutdown 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 msgr2=0x7ff04413ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:28.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff03c01c720 tx=0x7ff03c019690 comp rx=0 tx=0).stop 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 shutdown_connections 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff02803e550 0x7ff028040a10 unknown :-1 s=CLOSED pgs=3338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff044138870 0x7ff04413ce50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff0440a4c90 0x7ff04413c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 --2- 172.21.15.105:0/1944005348 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff04409ea50 0x7ff04413c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 >> 172.21.15.105:0/1944005348 conn(0x7ff0440a5840 msgr2=0x7ff0440a7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 shutdown_connections 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 -- 172.21.15.105:0/1944005348 wait complete. 2024-09-17T16:30:28.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:28.536+0000 7ff04c0ca640 1 librados: shutdown 2024-09-17T16:30:29.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 22 tests from LibRadosMisc snapshots: Running main() from gmock_main.cc 2024-09-17T16:30:29.561 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-09-17T16:30:29.561 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] Global test environment set-up. 2024-09-17T16:30:29.561 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-09-17T16:30:29.561 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4935 ms) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5034 ms) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3407 ms) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5058 ms) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6951 ms) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25385 ms total) 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-17T16:30:29.562 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5020 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5969 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7995 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5996 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4014 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi160-30349-11. 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: Waiting for snaps to purge. 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (34072 ms) 2024-09-17T16:30:29.563 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (63066 ms total) 2024-09-17T16:30:29.564 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: 2024-09-17T16:30:29.564 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [----------] Global test environment tear-down 2024-09-17T16:30:29.564 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (88451 ms total) 2024-09-17T16:30:29.564 INFO:tasks.workunit.client.0.smithi160.stdout: snapshots: [ PASSED ] 11 tests. 2024-09-17T16:30:31.562 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:30:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: get_monmap_and_config 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: build_initial_monmap 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: monmap: 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:31.685655+0000 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:31.685655+0000 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:31.689 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 Processor -- start 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 -- start start 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: init 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient: _add_conns ranks=[9,5,1,4,0,2,3,6,10,8,7] 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ea80a2620 0x7f3ea80a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-a con 0x7f3ea80a2620 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ea80a2f40 0x7f3ea8137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-h con 0x7f3ea80a2f40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3ea8137940 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:31.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea806f2e0 con 0x7f3ea8137940 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea806e580 con 0x7f3ea80a2620 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f3ea80a15e0 con 0x7f3ea80a2f40 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3eacf35640 10 monclient(hunting): authenticate will time out at 1114.951522s 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ea80a2620 0x7f3ea80a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3ea80a2620 0x7f3ea80a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51918/0 (socket says 172.21.15.105:51918) 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 1 -- 172.21.15.105:0/3949154113 learned_addr learned my addr 172.21.15.105:0/3949154113 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): get_auth_request con 0x7f3ea80a2620 auth_method 0 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea5d9b640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ea80a2f40 0x7f3ea8137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ea80a2f40 auth_method 0 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.680+0000 7f3ea5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 10 monclient(hunting): handle_auth_done global_id 34155 payload 274 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 -- 172.21.15.105:0/3949154113 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ea80a2f40 msgr2=0x7f3ea8137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ea80a2f40 0x7f3ea8137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 -- 172.21.15.105:0/3949154113 >> v1:172.21.15.160:6789/0 conn(0x7f3ea8137940 legacy=0x7f3ea813e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 10 monclient: _finish_hunting 0 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 monclient: found mon.noname-a 2024-09-17T16:30:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 -- 172.21.15.105:0/3949154113 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea80a3c30 con 0x7f3ea80a2620 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea659c640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ea80a2620 0x7f3ea80a2a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009870 tx=0x7f3e9c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b6c6a1a27d1a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 1 -- 172.21.15.105:0/3949154113 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0b1070 con 0x7f3ea80a2620 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: dump: 2024-09-17T16:30:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: _finish_auth 0 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: _check_auth_tickets 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 1 -- 172.21.15.105:0/3949154113 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c0ab830 con 0x7f3ea80a2620 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 5 monclient: authenticate success, global_id 34155 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 4 set_mon_vals no callback set 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 1 -- 172.21.15.105:0/3949154113 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0aa320 con 0x7f3ea80a2620 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3ea559a640 10 monclient: dump: 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 10 monclient: shutdown 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 -- 172.21.15.105:0/3949154113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ea80a2620 msgr2=0x7f3ea80a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ea80a2620 0x7f3ea80a2a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009870 tx=0x7f3e9c01c420 comp rx=0 tx=0).stop 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 -- 172.21.15.105:0/3949154113 shutdown_connections 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ea80a2f40 0x7f3ea8137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3949154113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ea80a2620 0x7f3ea80a2a00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 -- 172.21.15.105:0/3949154113 >> 172.21.15.105:0/3949154113 conn(0x7f3ea8068ba0 msgr2=0x7f3ea8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 -- 172.21.15.105:0/3949154113 shutdown_connections 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 -- 172.21.15.105:0/3949154113 wait complete. 2024-09-17T16:30:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 10 monclient: build_initial_monmap 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 10 monclient: monmap: 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 librados: starting msgr at 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.684+0000 7f3eacf35640 1 librados: starting objecter 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 Processor -- start 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- start start 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 librados: setting wanted keys 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 librados: calling monclient init 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: init 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _add_conns ranks=[4,6,2,0,8,5,3,7,1] 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-c con 0x7f3ea80a2620 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:31.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3ea80a2f40 0x7f3ea8135e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-d con 0x7f3ea80a2f40 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ea8137940 0x7f3ea8136370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): picked mon.noname-b con 0x7f3ea8137940 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea8069ad0 con 0x7f3ea8137940 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3ea8069950 con 0x7f3ea80a2620 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3ea80a1dd0 con 0x7f3ea80a2f40 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient(hunting): authenticate will time out at 1114.956750s 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3ea80a2f40 0x7f3ea8135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ea8137940 0x7f3ea8136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46348/0 (socket says 172.21.15.105:46348) 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 -- 172.21.15.105:0/3617199320 learned_addr learned my addr 172.21.15.105:0/3617199320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): get_auth_request con 0x7f3ea80a2620 auth_method 0 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): get_auth_request con 0x7f3ea80a2f40 auth_method 0 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): get_auth_request con 0x7f3ea8137940 auth_method 0 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient(hunting): handle_auth_done global_id 22495 payload 824 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 -- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3ea80a2f40 msgr2=0x7f3ea8135e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3ea80a2f40 0x7f3ea8135e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 -- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ea8137940 msgr2=0x7f3ea8136370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ea8137940 0x7f3ea8136370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient: _finish_hunting 0 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 monclient: found mon.noname-c 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 -- 172.21.15.105:0/3617199320 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea809e170 con 0x7f3ea80a2620 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea659c640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0af040 tx=0x7f3e9c0191c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 client.?.objecter ms_handle_connect 0x7f3ea80a2620 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0ba070 con 0x7f3ea80a2620 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: dump: 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:31.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c0044b0 con 0x7f3ea80a2620 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e9c0b1040 con 0x7f3ea80a2620 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 5 monclient: authenticate success, global_id 22495 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _renew_subs 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea8072840 con 0x7f3ea80a2620 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea4d99640 4 set_mon_vals no callback set 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 monclient: dump: 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:31.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:31.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 client.22495.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _renew_subs 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea813f160 con 0x7f3ea80a2620 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 librados: init done 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea80a2a00 con 0x7f3ea80a2620 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3e9c004040 con 0x7f3ea80a2620 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 0x7f3e88040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(103..103 src has 1..103) ==== 8518+0+0 (secure 0 0 0) 0x7f3e9c0e9c00 con 0x7f3ea80a2620 2024-09-17T16:30:31.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 10 client.22495.objecter ms_dispatch 0x7f3ea8000ba0 osd_map(103..103 src has 1..103) 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 3 client.22495.objecter handle_osd_map got epochs [103,103] > 0 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 3 client.22495.objecter handle_osd_map decoding full epoch 103 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3e96ffd640 20 client.22495.objecter dump_active .. 0 homeless 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 0x7f3e88040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.688+0000 7f3ea5d9b640 10 monclient: get_auth_request con 0x7f3e8803e550 auth_method 0 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.692+0000 7f3e96ffd640 10 client.22495.objecter ms_handle_connect 0x7f3e8803e550 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.692+0000 7f3ea5d9b640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 0x7f3e88040a10 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007a60 tx=0x7f3e98008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.692+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e9c016130 con 0x7f3ea80a2620 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.692+0000 7f3e96ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.692+0000 7f3e96ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:31.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.928+0000 7f3eacf35640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:30:31.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.928+0000 7f3eacf35640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.928+0000 7f3eacf35640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.928+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3ea8072620 con 0x7f3ea80a2620 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3e96ffd640 1 -- 172.21.15.105:0/3617199320 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f3e9c0e7770 con 0x7f3ea80a2620 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3e96ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3e96ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:31.937 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 10 librados: watch_flush enter 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 10 librados: watch_flush exit 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 20 client.22495.objecter shutdown clearing up homeless session... 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 10 client.22495.objecter successfully canceled tick 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 msgr2=0x7f3e88040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 0x7f3e88040a10 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007a60 tx=0x7f3e98008040 comp rx=0 tx=0).stop 2024-09-17T16:30:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 10 monclient: shutdown 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 msgr2=0x7f3ea813e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0af040 tx=0x7f3e9c0191c0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 shutdown_connections 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3e8803e550 0x7f3e88040a10 unknown :-1 s=CLOSED pgs=3341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3ea8137940 0x7f3ea8136370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3ea80a2f40 0x7f3ea8135e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 --2- 172.21.15.105:0/3617199320 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ea80a2620 0x7f3ea813e770 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 >> 172.21.15.105:0/3617199320 conn(0x7f3ea8068ba0 msgr2=0x7f3ea80a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 shutdown_connections 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 -- 172.21.15.105:0/3617199320 wait complete. 2024-09-17T16:30:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:31.932+0000 7f3eacf35640 1 librados: shutdown 2024-09-17T16:30:31.956 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:30:31.956 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:30:31.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: monmap: 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.072363+0000 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.072363+0000 2024-09-17T16:30:32.074 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.075 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 Processor -- start 2024-09-17T16:30:32.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 -- start start 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: init 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient: _add_conns ranks=[8,1,3,5,0,2,4,7,10,9,6] 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-e con 0x7fb8d40a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb8d40a4910 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8d40a7200 0x7fb8d4139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-g con 0x7fb8d40a7200 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d406ebc0 con 0x7fb8d40a4910 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d40a3fc0 con 0x7fb8d40a7200 2024-09-17T16:30:32.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb8d40a63c0 con 0x7fb8d40a68e0 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8da619640 10 monclient(hunting): authenticate will time out at 1115.338535s 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45160/0 (socket says 172.21.15.105:45160) 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 -- 172.21.15.105:0/3473391950 learned_addr learned my addr 172.21.15.105:0/3473391950 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request con 0x7fb8d40a68e0 auth_method 0 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_done global_id 23327 payload 274 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 -- 172.21.15.105:0/3473391950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8d40a7200 msgr2=0x7fb8d4139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d8bb6640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8d40a7200 0x7fb8d4139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8d40a7200 0x7fb8d4139db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 -- 172.21.15.105:0/3473391950 >> v1:172.21.15.160:6789/0 conn(0x7fb8d40a4910 legacy=0x7fb8d41396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 monclient: found mon.noname-e 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:32.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.068+0000 7fb8d3fff640 1 -- 172.21.15.105:0/3473391950 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40a3690 con 0x7fb8d40a68e0 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40098a0 tx=0x7fb8c401c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c93a22c8ea9e4800 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 1 -- 172.21.15.105:0/3473391950 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c40b0070 con 0x7fb8d40a68e0 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: dump: 2024-09-17T16:30:32.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.088 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 1 -- 172.21.15.105:0/3473391950 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c40aa810 con 0x7fb8d40a68e0 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 1 -- 172.21.15.105:0/3473391950 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c40a9300 con 0x7fb8d40a68e0 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 5 monclient: authenticate success, global_id 23327 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 4 set_mon_vals no callback set 2024-09-17T16:30:32.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d37fe640 10 monclient: dump: 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: shutdown 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- 172.21.15.105:0/3473391950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 msgr2=0x7fb8d40a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40098a0 tx=0x7fb8c401c410 comp rx=0 tx=0).stop 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- 172.21.15.105:0/3473391950 shutdown_connections 2024-09-17T16:30:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8d40a7200 0x7fb8d4139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- 172.21.15.105:0/3473391950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb8d40a68e0 0x7fb8d40a6cc0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- 172.21.15.105:0/3473391950 >> 172.21.15.105:0/3473391950 conn(0x7fb8d4068ba0 msgr2=0x7fb8d4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- 172.21.15.105:0/3473391950 shutdown_connections 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- 172.21.15.105:0/3473391950 wait complete. 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: monmap: 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 librados: starting msgr at 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 librados: starting objecter 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 Processor -- start 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- start start 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 librados: setting wanted keys 2024-09-17T16:30:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 librados: calling monclient init 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: init 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient: _add_conns ranks=[7,5,4,6,0,3,8,1,2] 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb8d40a4910 0x7fb8d41d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-i con 0x7fb8d40a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb8d40a68e0 0x7fb8d41d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-h con 0x7fb8d40a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): picked mon.noname-c con 0x7fb8d40a7200 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb8d413a2f0 con 0x7fb8d40a7200 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb8d40a7610 con 0x7fb8d40a68e0 2024-09-17T16:30:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb8d413a470 con 0x7fb8d40a4910 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8da619640 10 monclient(hunting): authenticate will time out at 1115.343593s 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46358/0 (socket says 172.21.15.105:46358) 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 1 -- 172.21.15.105:0/858348331 learned_addr learned my addr 172.21.15.105:0/858348331 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): get_auth_request con 0x7fb8d40a7200 auth_method 0 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d8bb6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb8d40a4910 0x7fb8d41d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.072+0000 7fb8cb7fe640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb8d40a68e0 0x7fb8d41d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request con 0x7fb8d40a4910 auth_method 0 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 10 monclient(hunting): get_auth_request con 0x7fb8d40a68e0 auth_method 0 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 10 monclient(hunting): handle_auth_done global_id 22504 payload 824 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb8d40a4910 msgr2=0x7fb8d41d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb8d40a4910 0x7fb8d41d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb8d40a68e0 msgr2=0x7fb8d41d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb8d40a68e0 0x7fb8d41d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 monclient: found mon.noname-c 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d4123000 con 0x7fb8d40a7200 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d8bb6640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00cc30 tx=0x7fb8cc00a5c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.?.objecter ms_handle_connect 0x7fb8d40a7200 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8cc00ce40 con 0x7fb8d40a7200 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: dump: 2024-09-17T16:30:32.097 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.098 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc00f860 con 0x7fb8d40a7200 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8cc0173b0 con 0x7fb8d40a7200 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: dump: 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.099 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.100 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d2ffd640 4 set_mon_vals no callback set 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 5 monclient: authenticate success, global_id 22504 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 monclient: _renew_subs 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d41232e0 con 0x7fb8d40a7200 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 client.22504.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 monclient: _renew_subs 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d4141880 con 0x7fb8d40a7200 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 1 librados: init done 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 0x7fb8d413f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4051e80 con 0x7fb8d4141200 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 0x7fb8d413f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient: get_auth_request con 0x7fb8d4141200 auth_method 0 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb8cc00f3f0 con 0x7fb8d40a7200 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_done global_id 34164 payload 824 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8d40a6cc0 con 0x7fb8d4141200 2024-09-17T16:30:32.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8b403e550 0x7fb8b4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(103..103 src has 1..103) ==== 8518+0+0 (secure 0 0 0) 0x7fb8cc009030 con 0x7fb8d40a7200 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.22504.objecter ms_dispatch 0x7fb8d4000ba0 osd_map(103..103 src has 1..103) 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 3 client.22504.objecter handle_osd_map got epochs [103,103] > 0 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 0x7fb8d413f700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8d406e5a0 tx=0x7fb8c40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 3 client.22504.objecter handle_osd_map decoding full epoch 103 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 20 client.22504.objecter dump_active .. 0 homeless 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.22504.objecter ms_handle_connect 0x7fb8d4141200 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.22504.objecter resend_mon_ops 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c40b9030 con 0x7fb8d4141200 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: dump: 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.103 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.104 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8b403e550 0x7fb8b4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 10 monclient: get_auth_request con 0x7fb8b403e550 auth_method 0 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb8c40b1040 con 0x7fb8d4141200 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 msgr2=0x7fb8d413f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 0x7fb8d413f700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8d406e5a0 tx=0x7fb8c40191b0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 msgr2=0x7fb8d413f700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d4141200 0x7fb8d413f700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8d17fa640 10 client.22504.objecter ms_handle_connect 0x7fb8b403e550 2024-09-17T16:30:32.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.076+0000 7fb8cb7fe640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8b403e550 0x7fb8b4040a10 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4074500 tx=0x7fb8bc009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 0x7fb8d4140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4140590 con 0x7fb8d413fc40 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 0x7fb8d4140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient: get_auth_request con 0x7fb8d413fc40 auth_method 0 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 10 monclient(hunting): handle_auth_done global_id 34173 payload 824 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 1 -- 172.21.15.105:0/858348331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb8d4123520 con 0x7fb8d413fc40 2024-09-17T16:30:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d3fff640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 0x7fb8d4140050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c10 tx=0x7fb8c40a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 client.22504.objecter ms_handle_connect 0x7fb8d413fc40 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 client.22504.objecter resend_mon_ops 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8c40b9040 con 0x7fb8d413fc40 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 monclient: dump: 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7fb8c40be460 con 0x7fb8d413fc40 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 msgr2=0x7fb8d4140050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 0x7fb8d4140050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c10 tx=0x7fb8c40a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:32.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 msgr2=0x7fb8d4140050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d17fa640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8d413fc40 0x7fb8d4140050 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4000c10 tx=0x7fb8c40a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 22887, 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:32.111 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.112 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:32.114 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:32.115 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.116 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:32.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.122 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.124 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.129 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.131 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:32.132 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 librados: watch_flush enter 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 librados: watch_flush exit 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 20 client.22504.objecter shutdown clearing up homeless session... 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 client.22504.objecter successfully canceled tick 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8b403e550 msgr2=0x7fb8b4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8b403e550 0x7fb8b4040a10 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4074500 tx=0x7fb8bc009040 comp rx=0 tx=0).stop 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 10 monclient: shutdown 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 msgr2=0x7fb8d413f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8d8bb6640 1 -- 172.21.15.105:0/858348331 reap_dead start 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00cc30 tx=0x7fb8cc00a5c0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 shutdown_connections 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 --2- 172.21.15.105:0/858348331 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb8d40a7200 0x7fb8d413f1c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 >> 172.21.15.105:0/858348331 conn(0x7fb8d4068ba0 msgr2=0x7fb8d409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 shutdown_connections 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 -- 172.21.15.105:0/858348331 wait complete. 2024-09-17T16:30:32.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.084+0000 7fb8da619640 1 librados: shutdown 2024-09-17T16:30:32.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:32.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:30:32.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:30:32.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: monmap: 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.255987+0000 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.255987+0000 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.258 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.259 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 Processor -- start 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 -- start start 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: init 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient: _add_conns ranks=[6,10,7,3,2,9,4,0,5,1,8] 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): picked mon.noname-d con 0x7f10900a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f10900a4910 0x7f10900a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): picked mon.noname-f con 0x7f10900a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 0x7f1090139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): picked mon.noname-i con 0x7f10901396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f10900a3fc0 con 0x7f10900a4910 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f10900a63c0 con 0x7f10900a68e0 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f10900a7740 con 0x7f10901396a0 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1097205640 10 monclient(hunting): authenticate will time out at 1115.521685s 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37448/0 (socket says 172.21.15.105:37448) 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 -- 172.21.15.105:0/1004839778 learned_addr learned my addr 172.21.15.105:0/1004839778 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): get_auth_request con 0x7f10900a68e0 auth_method 0 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f10957a2640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 0x7f1090139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient(hunting): handle_auth_done global_id 23352 payload 274 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f108ffff640 1 --2- 172.21.15.105:0/1004839778 >> v2:172.21.15.160:3300/0 conn(0x7f10900a4910 0x7f10900a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 -- 172.21.15.105:0/1004839778 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 msgr2=0x7f1090139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 0x7f1090139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 -- 172.21.15.105:0/1004839778 >> v2:172.21.15.160:3300/0 conn(0x7f10900a4910 msgr2=0x7f10900a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 --2- 172.21.15.105:0/1004839778 >> v2:172.21.15.160:3300/0 conn(0x7f10900a4910 0x7f10900a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 monclient: found mon.noname-d 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f1094fa1640 1 -- 172.21.15.105:0/1004839778 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109006ebc0 con 0x7f10900a68e0 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f10957a2640 10 monclient: get_auth_request con 0x7f10901396a0 auth_method 0 2024-09-17T16:30:32.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.252+0000 7f10957a2640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 0x7f1090139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1080009870 tx=0x7f108001c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9bcdcd6a599d4ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 1 -- 172.21.15.105:0/1004839778 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10800b2070 con 0x7f10900a68e0 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: dump: 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.271 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.272 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 1 -- 172.21.15.105:0/1004839778 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10800ab7b0 con 0x7f10900a68e0 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 1 -- 172.21.15.105:0/1004839778 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10800abe50 con 0x7f10900a68e0 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 5 monclient: authenticate success, global_id 23352 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 4 set_mon_vals no callback set 2024-09-17T16:30:32.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108f7fe640 10 monclient: dump: 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: shutdown 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/1004839778 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 msgr2=0x7f10900a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f1080009870 tx=0x7f108001c3f0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/1004839778 shutdown_connections 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10901396a0 0x7f1090139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- 172.21.15.105:0/1004839778 >> v2:172.21.15.160:3300/0 conn(0x7f10900a4910 0x7f10900a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- 172.21.15.105:0/1004839778 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f10900a68e0 0x7f10900a6cc0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/1004839778 >> 172.21.15.105:0/1004839778 conn(0x7f1090068ba0 msgr2=0x7f1090068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/1004839778 shutdown_connections 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/1004839778 wait complete. 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: monmap: 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 librados: starting msgr at 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 librados: starting objecter 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 Processor -- start 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- start start 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 librados: setting wanted keys 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 librados: calling monclient init 2024-09-17T16:30:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: init 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _add_conns ranks=[5,4,3,0,2,8,7,6,1] 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f10900a4910 0x7f10901d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): picked mon.noname-h con 0x7f10900a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 0x7f10901d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): picked mon.noname-c con 0x7f10900a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10901396a0 0x7f109013d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): picked mon.noname-g con 0x7f10901396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f109006e630 con 0x7f10901396a0 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f10900a7af0 con 0x7f10900a68e0 2024-09-17T16:30:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1090143320 con 0x7f10900a4910 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient(hunting): authenticate will time out at 1115.526603s 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 0x7f10901d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 0x7f10901d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46370/0 (socket says 172.21.15.105:46370) 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 -- 172.21.15.105:0/2942726886 learned_addr learned my addr 172.21.15.105:0/2942726886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): get_auth_request con 0x7f10900a68e0 auth_method 0 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f10900a4910 0x7f10901d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f10957a2640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10901396a0 0x7f109013d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 10 monclient(hunting): get_auth_request con 0x7f10900a4910 auth_method 0 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1094fa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient(hunting): handle_auth_done global_id 22513 payload 824 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f10900a4910 msgr2=0x7f10901d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f10900a4910 0x7f10901d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10901396a0 msgr2=0x7f109013d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10901396a0 0x7f109013d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 monclient: found mon.noname-c 2024-09-17T16:30:32.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1090123680 con 0x7f10900a68e0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f10957a2640 10 monclient: get_auth_request con 0x7f10901396a0 auth_method 0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f10957a2640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10901396a0 0x7f109013d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108ffff640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 0x7f10901d1b20 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f108400c820 tx=0x7f108400cce0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 client.?.objecter ms_handle_connect 0x7f10900a68e0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f108400ce10 con 0x7f10900a68e0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: dump: 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.281 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.282 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10840044a0 con 0x7f10900a68e0 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1084007ca0 con 0x7f10900a68e0 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 5 monclient: authenticate success, global_id 22513 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108e7fc640 4 set_mon_vals no callback set 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f108d7fa640 10 monclient: dump: 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.283 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.284 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _renew_subs 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1090122df0 con 0x7f10900a68e0 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 client.22513.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _renew_subs 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1090077b60 con 0x7f10900a68e0 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.256+0000 7f1097205640 1 librados: init done 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1097205640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1097205640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 0x7f109013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1090051e80 con 0x7f109013dab0 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 0x7f109013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient: get_auth_request con 0x7f109013dab0 auth_method 0 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1084007830 con 0x7f10900a68e0 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 10 monclient(hunting): handle_auth_done global_id 23645 payload 824 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10900a9fc0 con 0x7f109013dab0 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107c03e580 0x7f107c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(103..103 src has 1..103) ==== 8518+0+0 (secure 0 0 0) 0x7f1084050870 con 0x7f10900a68e0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 client.22513.objecter ms_dispatch 0x7f1090000ba0 osd_map(103..103 src has 1..103) 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 3 client.22513.objecter handle_osd_map got epochs [103,103] > 0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 3 client.22513.objecter handle_osd_map decoding full epoch 103 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f1094fa1640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 0x7f109013de90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f109006e5a0 tx=0x7f1080019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 20 client.22513.objecter dump_active .. 0 homeless 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 client.22513.objecter ms_handle_connect 0x7f109013dab0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 client.22513.objecter resend_mon_ops 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10800b2040 con 0x7f109013dab0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 monclient: dump: 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.287 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f10957a2640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107c03e580 0x7f107c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f10957a2640 10 monclient: get_auth_request con 0x7f107c03e580 auth_method 0 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f10800036e0 con 0x7f109013dab0 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 msgr2=0x7f109013de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 0x7f109013de90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f109006e5a0 tx=0x7f1080019190 comp rx=0 tx=0).stop 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 msgr2=0x7f109013de90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013dab0 0x7f109013de90 secure :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f109006e5a0 tx=0x7f1080019190 comp rx=0 tx=0).stop 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f108d7fa640 10 client.22513.objecter ms_handle_connect 0x7f107c03e580 2024-09-17T16:30:32.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.260+0000 7f10957a2640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107c03e580 0x7f107c040a40 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7f1088007ea0 tx=0x7f10880073f0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1097205640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1097205640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 0x7f10900a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1097205640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1090139b50 con 0x7f109013e3d0 2024-09-17T16:30:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 0x7f10900a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient: get_auth_request con 0x7f109013e3d0 auth_method 0 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.264+0000 7f1094fa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1094fa1640 10 monclient(hunting): handle_auth_done global_id 23654 payload 824 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1094fa1640 1 -- 172.21.15.105:0/2942726886 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f109013f790 con 0x7f109013e3d0 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 client.22513.objecter ms_handle_connect 0x7f109013e3d0 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 client.22513.objecter resend_mon_ops 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1094fa1640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 0x7f10900a0a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f108001c220 tx=0x7f10800a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10800b2040 con 0x7f109013e3d0 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 monclient: dump: 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f10800bf2b0 con 0x7f109013e3d0 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 msgr2=0x7f10900a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 0x7f10900a0a20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f108001c220 tx=0x7f10800a7000 comp rx=0 tx=0).stop 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 msgr2=0x7f10900a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f108d7fa640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f109013e3d0 0x7f10900a0a20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 179911, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:32.295 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.296 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.297 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:32.298 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:32.299 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.300 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.301 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.303 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.305 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.310 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.313 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.314 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.315 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.316 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:32.317 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 10 librados: watch_flush enter 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 10 librados: watch_flush exit 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 20 client.22513.objecter shutdown clearing up homeless session... 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 10 client.22513.objecter successfully canceled tick 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107c03e580 msgr2=0x7f107c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107c03e580 0x7f107c040a40 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7f1088007ea0 tx=0x7f10880073f0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 10 monclient: shutdown 2024-09-17T16:30:32.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 msgr2=0x7f10901d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 --2- 172.21.15.105:0/2942726886 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f10900a68e0 0x7f10901d1b20 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f108400c820 tx=0x7f108400cce0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f10957a2640 1 -- 172.21.15.105:0/2942726886 reap_dead start 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f10957a2640 1 -- 172.21.15.105:0/2942726886 reap_dead start 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 shutdown_connections 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 >> 172.21.15.105:0/2942726886 conn(0x7f1090068ba0 msgr2=0x7f10900a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 shutdown_connections 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 -- 172.21.15.105:0/2942726886 wait complete. 2024-09-17T16:30:32.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.268+0000 7f1097205640 1 librados: shutdown 2024-09-17T16:30:32.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:32.320 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:30:32.321 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:30:32.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: monmap: 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.432691+0000 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.432691+0000 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.435 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.436 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 Processor -- start 2024-09-17T16:30:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 -- start start 2024-09-17T16:30:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: init 2024-09-17T16:30:32.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient: _add_conns ranks=[10,6,0,1,4,7,2,3,5,9,8] 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13c00a4750 0x7f13c00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-f con 0x7f13c00a4750 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-d con 0x7f13c009eec0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f13c0139530 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c009df90 con 0x7f13c0139530 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f13c00a55b0 con 0x7f13c00a4750 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f13c009dcb0 con 0x7f13c009eec0 2024-09-17T16:30:32.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c859a640 10 monclient(hunting): authenticate will time out at 1115.698560s 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37464/0 (socket says 172.21.15.105:37464) 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 -- 172.21.15.105:0/1336834555 learned_addr learned my addr 172.21.15.105:0/1336834555 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): get_auth_request con 0x7f13c009eec0 auth_method 0 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c6336640 1 --2- 172.21.15.105:0/1336834555 >> v2:172.21.15.160:3300/0 conn(0x7f13c00a4750 0x7f13c00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c6336640 10 monclient(hunting): get_auth_request con 0x7f13c00a4750 auth_method 0 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c6336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c6336640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c6336640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient(hunting): handle_auth_done global_id 23361 payload 274 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 -- 172.21.15.105:0/1336834555 >> v2:172.21.15.160:3300/0 conn(0x7f13c00a4750 msgr2=0x7f13c00a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 --2- 172.21.15.105:0/1336834555 >> v2:172.21.15.160:3300/0 conn(0x7f13c00a4750 0x7f13c00a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 -- 172.21.15.105:0/1336834555 >> v1:172.21.15.105:6789/0 conn(0x7f13c0139530 legacy=0x7f13c01399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 monclient: found mon.noname-d 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5334640 1 -- 172.21.15.105:0/1336834555 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 598051691 0 0) 0x7f13b00014a0 con 0x7f13c0139530 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5b35640 1 -- 172.21.15.105:0/1336834555 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c009e270 con 0x7f13c009eec0 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.428+0000 7f13c5334640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5b35640 1 --2- 172.21.15.105:0/1336834555 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009870 tx=0x7f13b401c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=82b155800605889a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 1 -- 172.21.15.105:0/1336834555 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b40b0070 con 0x7f13c009eec0 2024-09-17T16:30:32.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: dump: 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 1 -- 172.21.15.105:0/1336834555 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b40aa810 con 0x7f13c009eec0 2024-09-17T16:30:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 1 -- 172.21.15.105:0/1336834555 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b40aae30 con 0x7f13c009eec0 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 5 monclient: authenticate success, global_id 23361 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 4 set_mon_vals no callback set 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c5334640 10 monclient: dump: 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.447 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.448 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: shutdown 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- 172.21.15.105:0/1336834555 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 msgr2=0x7f13c00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- 172.21.15.105:0/1336834555 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009870 tx=0x7f13b401c420 comp rx=0 tx=0).stop 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- 172.21.15.105:0/1336834555 shutdown_connections 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- 172.21.15.105:0/1336834555 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c009eec0 0x7f13c00a5070 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- 172.21.15.105:0/1336834555 >> v2:172.21.15.160:3300/0 conn(0x7f13c00a4750 0x7f13c00a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- 172.21.15.105:0/1336834555 >> 172.21.15.105:0/1336834555 conn(0x7f13c00a5840 msgr2=0x7f13c00a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- 172.21.15.105:0/1336834555 shutdown_connections 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- 172.21.15.105:0/1336834555 wait complete. 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: monmap: 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.450 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 librados: starting msgr at 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 librados: starting objecter 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 Processor -- start 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- start start 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 librados: setting wanted keys 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 librados: calling monclient init 2024-09-17T16:30:32.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: init 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient: _add_conns ranks=[1,5,2,0,3,7,4,6,8] 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c009eec0 0x7f13c01d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-f con 0x7f13c009eec0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c00a4750 0x7f13c01d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-h con 0x7f13c00a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): picked mon.noname-b con 0x7f13c0139530 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01443d0 con 0x7f13c009eec0 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f13c0144550 con 0x7f13c0139530 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f13c01446d0 con 0x7f13c00a4750 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c859a640 10 monclient(hunting): authenticate will time out at 1115.703699s 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56524/0 (socket says 172.21.15.105:56524) 2024-09-17T16:30:32.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 1 -- 172.21.15.105:0/254684982 learned_addr learned my addr 172.21.15.105:0/254684982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): get_auth_request con 0x7f13c0139530 auth_method 0 2024-09-17T16:30:32.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.432+0000 7f13c6b37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c009eec0 0x7f13c01d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c00a4750 0x7f13c01d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): get_auth_request con 0x7f13c009eec0 auth_method 0 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 10 monclient(hunting): handle_auth_done global_id 23663 payload 824 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c00a4750 msgr2=0x7f13c01d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c00a4750 0x7f13c01d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c009eec0 msgr2=0x7f13c01d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c009eec0 0x7f13c01d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 monclient: found mon.noname-b 2024-09-17T16:30:32.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0121630 con 0x7f13c0139530 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 10 monclient: get_auth_request con 0x7f13c00a4750 auth_method 0 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c00a4750 0x7f13c01d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6b37640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c730 tx=0x7f13b000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.?.objecter ms_handle_connect 0x7f13c0139530 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b0003df0 con 0x7f13c0139530 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:30:32.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: dump: 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.467 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 5 monclient: authenticate success, global_id 23663 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0005070 con 0x7f13c0139530 2024-09-17T16:30:32.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 monclient: _renew_subs 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c0122b60 con 0x7f13c0139530 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b0005690 con 0x7f13c0139530 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c4b33640 4 set_mon_vals no callback set 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: dump: 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.469 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.470 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 client.23663.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 monclient: _renew_subs 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c0121030 con 0x7f13c0139530 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 1 librados: init done 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 0x7f13c013cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f13c006a210 con 0x7f13c013c760 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 0x7f13c013cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient: get_auth_request con 0x7f13c013c760 auth_method 0 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f13b0003980 con 0x7f13c0139530 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 10 monclient(hunting): handle_auth_done global_id 22522 payload 824 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13c00a7d50 con 0x7f13c013c760 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13ac03e550 0x7f13ac040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(103..103 src has 1..103) ==== 8518+0+0 (secure 0 0 0) 0x7f13b0050940 con 0x7f13c0139530 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.23663.objecter ms_dispatch 0x7f13c0000ba0 osd_map(103..103 src has 1..103) 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 3 client.23663.objecter handle_osd_map got epochs [103,103] > 0 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 3 client.23663.objecter handle_osd_map decoding full epoch 103 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c6336640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 0x7f13c013cb40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f13b800f4b0 tx=0x7f13b800f970 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 20 client.23663.objecter dump_active .. 0 homeless 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.23663.objecter ms_handle_connect 0x7f13c013c760 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.23663.objecter resend_mon_ops 2024-09-17T16:30:32.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b8019020 con 0x7f13c013c760 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: dump: 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.474 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13ac03e550 0x7f13ac040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 10 monclient: get_auth_request con 0x7f13ac03e550 auth_method 0 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f13b8014670 con 0x7f13c013c760 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 msgr2=0x7f13c013cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 0x7f13c013cb40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f13b800f4b0 tx=0x7f13b800f970 comp rx=0 tx=0).stop 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 msgr2=0x7f13c013cb40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013c760 0x7f13c013cb40 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13beffd640 10 client.23663.objecter ms_handle_connect 0x7f13ac03e550 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.436+0000 7f13c5b35640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13ac03e550 0x7f13ac040a10 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ad000 tx=0x7f13b40a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 0x7f13c011f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f13c006cd40 con 0x7f13c013d080 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 0x7f13c011f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient: get_auth_request con 0x7f13c013d080 auth_method 0 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 10 monclient(hunting): handle_auth_done global_id 22531 payload 824 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 1 -- 172.21.15.105:0/254684982 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13c00a3030 con 0x7f13c013d080 2024-09-17T16:30:32.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6336640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 0x7f13c011f390 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f13b801a000 tx=0x7f13b80046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 client.23663.objecter ms_handle_connect 0x7f13c013d080 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 client.23663.objecter resend_mon_ops 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b8019040 con 0x7f13c013d080 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 monclient: dump: 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.478 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.479 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f13b8014e40 con 0x7f13c013d080 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 msgr2=0x7f13c011f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 0x7f13c011f390 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f13b801a000 tx=0x7f13b80046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 msgr2=0x7f13c011f390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13beffd640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c013d080 0x7f13c011f390 secure :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f13b801a000 tx=0x7f13b80046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180083, 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:32.481 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:32.482 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:32.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.484 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:32.485 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.486 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.490 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:32.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.494 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.496 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.502 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.505 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.506 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:32.507 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:32.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 librados: watch_flush enter 2024-09-17T16:30:32.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 librados: watch_flush exit 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 20 client.23663.objecter shutdown clearing up homeless session... 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 client.23663.objecter successfully canceled tick 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13ac03e550 msgr2=0x7f13ac040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13ac03e550 0x7f13ac040a10 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7f13b40ad000 tx=0x7f13b40a6000 comp rx=0 tx=0).stop 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 10 monclient: shutdown 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 msgr2=0x7f13c01d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c730 tx=0x7f13b000cbf0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c6b37640 1 -- 172.21.15.105:0/254684982 reap_dead start 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 shutdown_connections 2024-09-17T16:30:32.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 --2- 172.21.15.105:0/254684982 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f13c0139530 0x7f13c01d2060 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 >> 172.21.15.105:0/254684982 conn(0x7f13c00a5840 msgr2=0x7f13c00a6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 shutdown_connections 2024-09-17T16:30:32.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 -- 172.21.15.105:0/254684982 wait complete. 2024-09-17T16:30:32.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.444+0000 7f13c859a640 1 librados: shutdown 2024-09-17T16:30:32.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:32.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:30:32.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:30:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.620+0000 7f80e856f640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.620+0000 7f80e856f640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.620+0000 7f80e856f640 10 monclient: monmap: 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.626770+0000 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.626770+0000 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.630 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 Processor -- start 2024-09-17T16:30:32.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 -- start start 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient: init 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient: _add_conns ranks=[6,1,7,9,8,10,4,5,3,0,2] 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-d con 0x7f80e00a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f80e009e740 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80e009e0b0 0x7f80e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-i con 0x7f80e009e0b0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e00a0a80 con 0x7f80e009e740 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f80e009ef60 con 0x7f80e00a4750 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f80e00a31c0 con 0x7f80e009e0b0 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient(hunting): authenticate will time out at 1115.892870s 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37472/0 (socket says 172.21.15.105:37472) 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 -- 172.21.15.105:0/226865346 learned_addr learned my addr 172.21.15.105:0/226865346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): get_auth_request con 0x7f80e00a4750 auth_method 0 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e6b0c640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80e009e0b0 0x7f80e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e6b0c640 10 monclient(hunting): get_auth_request con 0x7f80e009e0b0 auth_method 0 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e6b0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e6b0c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e6b0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient(hunting): handle_auth_done global_id 23370 payload 274 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 -- 172.21.15.105:0/226865346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80e009e0b0 msgr2=0x7f80e00a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80e009e0b0 0x7f80e00a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 -- 172.21.15.105:0/226865346 >> v1:172.21.15.160:6789/0 conn(0x7f80e009e740 legacy=0x7f80e0136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 monclient: found mon.noname-d 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 -- 172.21.15.105:0/226865346 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e00a6710 con 0x7f80e00a4750 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e630b640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc01c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=adb4a6542757cfa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 1 -- 172.21.15.105:0/226865346 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80dc0b0070 con 0x7f80e00a4750 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: dump: 2024-09-17T16:30:32.642 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.643 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 1 -- 172.21.15.105:0/226865346 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc0aa810 con 0x7f80e00a4750 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 5 monclient: authenticate success, global_id 23370 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 1 -- 172.21.15.105:0/226865346 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80dc0a9300 con 0x7f80e00a4750 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e856f640 4 set_mon_vals no callback set 2024-09-17T16:30:32.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.624+0000 7f80e5309640 10 monclient: dump: 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.645 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: shutdown 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/226865346 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 msgr2=0x7f80e00a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc01c440 comp rx=0 tx=0).stop 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/226865346 shutdown_connections 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80e009e0b0 0x7f80e00a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- 172.21.15.105:0/226865346 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e00a4750 0x7f80e00a4b30 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/226865346 >> 172.21.15.105:0/226865346 conn(0x7f80e00a5840 msgr2=0x7f80e00a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/226865346 shutdown_connections 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/226865346 wait complete. 2024-09-17T16:30:32.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: monmap: 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 librados: starting msgr at 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 librados: starting objecter 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 Processor -- start 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- start start 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 librados: setting wanted keys 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 librados: calling monclient init 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: init 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _add_conns ranks=[4,0,1,3,7,5,8,6,2] 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 0x7f80e01390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-c con 0x7f80e009e0b0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e009e740 0x7f80e0139600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-a con 0x7f80e009e740 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80e00a4750 0x7f80e0139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): picked mon.noname-f con 0x7f80e00a4750 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80e013f300 con 0x7f80e009e740 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f80e013f180 con 0x7f80e00a4750 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f80e013f480 con 0x7f80e009e0b0 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): authenticate will time out at 1115.897968s 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 0x7f80e01390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 0x7f80e01390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46384/0 (socket says 172.21.15.105:46384) 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 -- 172.21.15.105:0/3409004545 learned_addr learned my addr 172.21.15.105:0/3409004545 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): get_auth_request con 0x7f80e009e0b0 auth_method 0 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e009e740 0x7f80e0139600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): get_auth_request con 0x7f80e009e740 auth_method 0 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e6b0c640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80e00a4750 0x7f80e0139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e6b0c640 10 monclient(hunting): get_auth_request con 0x7f80e00a4750 auth_method 0 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e6b0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e6b0c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e6b0c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient(hunting): handle_auth_done global_id 22540 payload 824 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80e00a4750 msgr2=0x7f80e0139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80e00a4750 0x7f80e0139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e009e740 msgr2=0x7f80e0139600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e009e740 0x7f80e0139600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 monclient: found mon.noname-c 2024-09-17T16:30:32.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e013cee0 con 0x7f80e009e0b0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80e009e740 0x7f80e0139600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e630b640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 0x7f80e01390c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f80dc01c270 tx=0x7f80dc0191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 client.?.objecter ms_handle_connect 0x7f80e009e0b0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80dc0ba020 con 0x7f80e009e0b0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: dump: 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.652 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc0044b0 con 0x7f80e009e0b0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 5 monclient: authenticate success, global_id 22540 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _renew_subs 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e00a5c20 con 0x7f80e009e0b0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80dc0b0040 con 0x7f80e009e0b0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 10 monclient: dump: 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.655 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 client.22540.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _renew_subs 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e4b08640 4 set_mon_vals no callback set 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e013c070 con 0x7f80e009e0b0 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 librados: init done 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 0x7f80e002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f80e0051e80 con 0x7f80e01cfed0 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 0x7f80e002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient: get_auth_request con 0x7f80e01cfed0 auth_method 0 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.628+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f80dc0a4d10 con 0x7f80e009e0b0 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_done global_id 23379 payload 824 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e5b0a640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80e0136500 con 0x7f80e01cfed0 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f80c403e540 0x7f80c4040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 0x7f80e002d450 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0004510 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 client.22540.objecter ms_handle_connect 0x7f80e01cfed0 2024-09-17T16:30:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 client.22540.objecter resend_mon_ops 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(104..104 src has 1..104) ==== 8321+0+0 (secure 0 0 0) 0x7f80dc0ea680 con 0x7f80e009e0b0 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 client.22540.objecter ms_dispatch 0x7f80e0000ba0 osd_map(104..104 src has 1..104) 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 3 client.22540.objecter handle_osd_map got epochs [104,104] > 0 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 3 client.22540.objecter handle_osd_map decoding full epoch 104 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 20 client.22540.objecter dump_active .. 0 homeless 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80d0005030 con 0x7f80e01cfed0 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 monclient: dump: 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e6b0c640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f80c403e540 0x7f80c4040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e6b0c640 10 monclient: get_auth_request con 0x7f80c403e540 auth_method 0 2024-09-17T16:30:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f80d000dad0 con 0x7f80e01cfed0 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 msgr2=0x7f80e002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 0x7f80e002d450 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0004510 comp rx=0 tx=0).stop 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 msgr2=0x7f80e002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01cfed0 0x7f80e002d450 secure :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0004510 comp rx=0 tx=0).stop 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80d6ffd640 10 client.22540.objecter ms_handle_connect 0x7f80c403e540 2024-09-17T16:30:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.632+0000 7f80e6b0c640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f80c403e540 0x7f80c4040a00 secure :-1 s=READY pgs=3345 cs=0 l=1 rev1=1 crypto rx=0x7f80e011f3d0 tx=0x7f80d800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 0x7f80e002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f80e013c7d0 con 0x7f80e01d02b0 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 0x7f80e002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient: get_auth_request con 0x7f80e01d02b0 auth_method 0 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 10 monclient(hunting): handle_auth_done global_id 23388 payload 824 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 1 -- 172.21.15.105:0/3409004545 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f80e013ab80 con 0x7f80e01d02b0 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 client.22540.objecter ms_handle_connect 0x7f80e01d02b0 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 client.22540.objecter resend_mon_ops 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e5b0a640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 0x7f80e002d990 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0003770 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80d000de70 con 0x7f80e01d02b0 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 monclient: dump: 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f80d000de70 con 0x7f80e01d02b0 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 msgr2=0x7f80e002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 0x7f80e002d990 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0003770 comp rx=0 tx=0).stop 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 msgr2=0x7f80e002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80d6ffd640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f80e01d02b0 0x7f80e002d990 secure :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f80d000fae0 tx=0x7f80d0003770 comp rx=0 tx=0).stop 2024-09-17T16:30:32.668 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:32.668 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.668 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180275, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:32.669 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.670 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.671 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:32.672 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.673 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:32.674 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:32.675 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.677 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.679 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.682 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.684 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:32.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.689 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.690 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:32.691 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 10 librados: watch_flush enter 2024-09-17T16:30:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 10 librados: watch_flush exit 2024-09-17T16:30:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 20 client.22540.objecter shutdown clearing up homeless session... 2024-09-17T16:30:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.636+0000 7f80e856f640 10 client.22540.objecter successfully canceled tick 2024-09-17T16:30:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f80c403e540 msgr2=0x7f80c4040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f80c403e540 0x7f80c4040a00 secure :-1 s=READY pgs=3345 cs=0 l=1 rev1=1 crypto rx=0x7f80e011f3d0 tx=0x7f80d800b040 comp rx=0 tx=0).stop 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 10 monclient: shutdown 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 msgr2=0x7f80e01390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 --2- 172.21.15.105:0/3409004545 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f80e009e0b0 0x7f80e01390c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f80dc01c270 tx=0x7f80dc0191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e6b0c640 1 -- 172.21.15.105:0/3409004545 reap_dead start 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e6b0c640 1 -- 172.21.15.105:0/3409004545 reap_dead start 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 shutdown_connections 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 >> 172.21.15.105:0/3409004545 conn(0x7f80e00a5840 msgr2=0x7f80e00a0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 shutdown_connections 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 -- 172.21.15.105:0/3409004545 wait complete. 2024-09-17T16:30:32.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.640+0000 7f80e856f640 1 librados: shutdown 2024-09-17T16:30:32.695 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:32.695 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:30:32.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: monmap: 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.808229+0000 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.808229+0000 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.810 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.811 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 Processor -- start 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 -- start start 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: init 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient: _add_conns ranks=[7,3,6,9,10,2,4,8,5,0,1] 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fae840a68e0 0x7fae840a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-i con 0x7fae840a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae840a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-g con 0x7fae840a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-d con 0x7fae841396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fae840a3fc0 con 0x7fae840a4910 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fae840a63c0 con 0x7fae841396a0 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fae840a7740 con 0x7fae840a68e0 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8bf65640 10 monclient(hunting): authenticate will time out at 1116.074029s 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37496/0 (socket says 172.21.15.105:37496) 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 -- 172.21.15.105:0/2195341851 learned_addr learned my addr 172.21.15.105:0/2195341851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): get_auth_request con 0x7fae841396a0 auth_method 0 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89d01640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fae840a68e0 0x7fae840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89d01640 10 monclient(hunting): get_auth_request con 0x7fae840a68e0 auth_method 0 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89d01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89d01640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89d01640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient(hunting): handle_auth_done global_id 23397 payload 274 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae89500640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae840a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 -- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fae840a68e0 msgr2=0x7fae840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fae840a68e0 0x7fae840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 -- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 msgr2=0x7fae840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae840a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 monclient: found mon.noname-d 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.804+0000 7fae8a502640 1 -- 172.21.15.105:0/2195341851 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8406ec10 con 0x7fae841396a0 2024-09-17T16:30:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098a0 tx=0x7fae7c01cea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8379a34abfbaa570 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 1 -- 172.21.15.105:0/2195341851 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c0b0070 con 0x7fae841396a0 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: dump: 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 1 -- 172.21.15.105:0/2195341851 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c0aa810 con 0x7fae841396a0 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 5 monclient: authenticate success, global_id 23397 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 1 -- 172.21.15.105:0/2195341851 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c0a9300 con 0x7fae841396a0 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 4 set_mon_vals no callback set 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae88cff640 10 monclient: dump: 2024-09-17T16:30:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: shutdown 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- 172.21.15.105:0/2195341851 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 msgr2=0x7fae84139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098a0 tx=0x7fae7c01cea0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- 172.21.15.105:0/2195341851 shutdown_connections 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fae841396a0 0x7fae84139ae0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae840a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2195341851 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fae840a68e0 0x7fae840a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- 172.21.15.105:0/2195341851 >> 172.21.15.105:0/2195341851 conn(0x7fae84068ba0 msgr2=0x7fae84068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- 172.21.15.105:0/2195341851 shutdown_connections 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- 172.21.15.105:0/2195341851 wait complete. 2024-09-17T16:30:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: monmap: 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 librados: starting msgr at 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 librados: starting objecter 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 Processor -- start 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- start start 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 librados: setting wanted keys 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 librados: calling monclient init 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: init 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient: _add_conns ranks=[3,8,4,6,2,5,1,7,0] 2024-09-17T16:30:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae841d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-g con 0x7fae840a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae840a68e0 0x7fae841d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-e con 0x7fae840a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 0x7fae8413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): picked mon.noname-c con 0x7fae841396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fae84143320 con 0x7fae840a4910 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fae8406e630 con 0x7fae841396a0 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fae841434a0 con 0x7fae840a68e0 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8bf65640 10 monclient(hunting): authenticate will time out at 1116.079127s 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 0x7fae8413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae840a68e0 0x7fae841d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 0x7fae8413d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46400/0 (socket says 172.21.15.105:46400) 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 learned_addr learned my addr 172.21.15.105:0/2404029981 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): get_auth_request con 0x7fae841396a0 auth_method 0 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): get_auth_request con 0x7fae840a68e0 auth_method 0 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89500640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89d01640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae841d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89d01640 10 monclient(hunting): get_auth_request con 0x7fae840a4910 auth_method 0 2024-09-17T16:30:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89d01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89d01640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae89d01640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient(hunting): handle_auth_done global_id 22549 payload 824 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae840a68e0 msgr2=0x7fae841d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae840a68e0 0x7fae841d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 msgr2=0x7fae841d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae840a4910 0x7fae841d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient: _finish_hunting 0 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 monclient: found mon.noname-c 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841236c0 con 0x7fae841396a0 2024-09-17T16:30:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae8a502640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 0x7fae8413d640 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fae7c01c270 tx=0x7fae7c0191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae767fc640 10 client.?.objecter ms_handle_connect 0x7fae841396a0 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.808+0000 7fae767fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c0b8020 con 0x7fae841396a0 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: dump: 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c0044b0 con 0x7fae841396a0 2024-09-17T16:30:32.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c0b0040 con 0x7fae841396a0 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 5 monclient: authenticate success, global_id 22549 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 monclient: _renew_subs 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8413f460 con 0x7fae841396a0 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae77fff640 4 set_mon_vals no callback set 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: dump: 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.838 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 client.22549.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 monclient: _renew_subs 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841233e0 con 0x7fae841396a0 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 1 librados: init done 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 0x7fae8413df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fae84051e80 con 0x7fae8413db80 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 0x7fae8413df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient: get_auth_request con 0x7fae8413db80 auth_method 0 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fae7c0bb020 con 0x7fae841396a0 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 10 monclient(hunting): handle_auth_done global_id 23345 payload 824 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae8413fc40 con 0x7fae8413db80 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae6803e540 0x7fae68040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(104..104 src has 1..104) ==== 8321+0+0 (secure 0 0 0) 0x7fae7c0ea240 con 0x7fae841396a0 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 client.22549.objecter ms_dispatch 0x7fae84000ba0 osd_map(104..104 src has 1..104) 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 3 client.22549.objecter handle_osd_map got epochs [104,104] > 0 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 3 client.22549.objecter handle_osd_map decoding full epoch 104 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89d01640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 0x7fae8413df60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fae7800f4b0 tx=0x7fae7800f970 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 20 client.22549.objecter dump_active .. 0 homeless 2024-09-17T16:30:32.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 client.22549.objecter ms_handle_connect 0x7fae8413db80 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 client.22549.objecter resend_mon_ops 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae78019020 con 0x7fae8413db80 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: dump: 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89500640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae6803e540 0x7fae68040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89500640 10 monclient: get_auth_request con 0x7fae6803e540 auth_method 0 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fae78014670 con 0x7fae8413db80 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 msgr2=0x7fae8413df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 0x7fae8413df60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fae7800f4b0 tx=0x7fae7800f970 comp rx=0 tx=0).stop 2024-09-17T16:30:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 msgr2=0x7fae8413df60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413db80 0x7fae8413df60 secure :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fae7800f4b0 tx=0x7fae7800f970 comp rx=0 tx=0).stop 2024-09-17T16:30:32.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae767fc640 10 client.22549.objecter ms_handle_connect 0x7fae6803e540 2024-09-17T16:30:32.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.812+0000 7fae89500640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae6803e540 0x7fae68040a00 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7fae8413ec80 tx=0x7fae80007750 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 0x7fae840a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fae84122850 con 0x7fae8413e4a0 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 0x7fae840a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient: get_auth_request con 0x7fae8413e4a0 auth_method 0 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:32.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 10 monclient(hunting): handle_auth_done global_id 23354 payload 824 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 1 -- 172.21.15.105:0/2404029981 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fae84139ae0 con 0x7fae8413e4a0 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 client.22549.objecter ms_handle_connect 0x7fae8413e4a0 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 client.22549.objecter resend_mon_ops 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae89d01640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 0x7fae840a0a20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fae7801a000 tx=0x7fae780046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fae78019040 con 0x7fae8413e4a0 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 monclient: dump: 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:32.847 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:32.848 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fae7801e420 con 0x7fae8413e4a0 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 msgr2=0x7fae840a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 0x7fae840a0a20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fae7801a000 tx=0x7fae780046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 msgr2=0x7fae840a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae767fc640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fae8413e4a0 0x7fae840a0a20 secure :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fae7801a000 tx=0x7fae780046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:32.849 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180463, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.850 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.851 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:32.853 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:32.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:32.855 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.856 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.861 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.864 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.865 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.866 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.867 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:32.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:32.875 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:32.875 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:32.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 librados: watch_flush enter 2024-09-17T16:30:32.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 librados: watch_flush exit 2024-09-17T16:30:32.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 20 client.22549.objecter shutdown clearing up homeless session... 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 client.22549.objecter successfully canceled tick 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae6803e540 msgr2=0x7fae68040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fae6803e540 0x7fae68040a00 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7fae8413ec80 tx=0x7fae80007750 comp rx=0 tx=0).stop 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 10 monclient: shutdown 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 msgr2=0x7fae8413d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 --2- 172.21.15.105:0/2404029981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fae841396a0 0x7fae8413d640 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fae7c01c270 tx=0x7fae7c0191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 reap_dead start 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8a502640 1 -- 172.21.15.105:0/2404029981 reap_dead start 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 shutdown_connections 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 >> 172.21.15.105:0/2404029981 conn(0x7fae84068ba0 msgr2=0x7fae840a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 shutdown_connections 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 -- 172.21.15.105:0/2404029981 wait complete. 2024-09-17T16:30:32.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.820+0000 7fae8bf65640 1 librados: shutdown 2024-09-17T16:30:32.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:32.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:30:32.878 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:30:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: get_monmap_and_config 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: build_initial_monmap 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: monmap: 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:32.992036+0000 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:32.992036+0000 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:32.995 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 Processor -- start 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 -- start start 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: init 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient: _add_conns ranks=[0,3,10,6,8,9,2,7,1,4,5] 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fece40a6a30 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fece40a4ab0 0x7fece413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-g con 0x7fece40a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fece40a7520 0x7fece4140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-f con 0x7fece40a7520 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece40a4080 con 0x7fece40a6a30 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fece40a7c40 con 0x7fece40a7520 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fece40a37e0 con 0x7fece40a4ab0 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 10 monclient(hunting): authenticate will time out at 1116.257911s 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fece40a6a30 0x7fece40a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43122/0 (socket says 172.21.15.105:43122) 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 1 -- 172.21.15.105:0/514791457 learned_addr learned my addr 172.21.15.105:0/514791457 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 --2- 172.21.15.105:0/514791457 >> v2:172.21.15.160:3300/0 conn(0x7fece40a7520 0x7fece4140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fece40a4ab0 0x7fece413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): get_auth_request con 0x7fece40a4ab0 auth_method 0 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256791457 0 0) 0x7fece40a4080 con 0x7fece40a6a30 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): get_auth_request con 0x7fece40a7520 auth_method 0 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): _negotiate 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): my global_id is 34200 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecd0003530 con 0x7fece40a6a30 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823685829 0 0) 0x7fecd0003530 con 0x7fece40a6a30 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fece40a4ab0 msgr2=0x7fece413ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fece40a4ab0 0x7fece413ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 >> v2:172.21.15.160:3300/0 conn(0x7fece40a7520 msgr2=0x7fece4140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 --2- 172.21.15.105:0/514791457 >> v2:172.21.15.160:3300/0 conn(0x7fece40a7520 0x7fece4140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece40a3ac0 con 0x7fece40a6a30 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fece0001970 con 0x7fece40a6a30 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: dump: 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.011 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 >> v1:172.21.15.105:6789/0 conn(0x7fece40a6a30 legacy=0x7fece40a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient: _add_conns ranks=[2,6,0,3,8,7,5,4,1] 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0004210 0x7fecd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): picked mon.b con 0x7fecd0004210 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:33.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fecd0004da0 0x7fecd0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): picked mon.d con 0x7fecd0004da0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 0x7fecd000bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): picked mon.a con 0x7fecd0009ab0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fece40a7c40 con 0x7fecd0009ab0 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fece40a37e0 con 0x7fecd0004210 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fecd000c850 con 0x7fecd0004da0 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fecdbfff640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:30:33.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0004210 0x7fecd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fecd0004da0 0x7fecd0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 0x7fecd000bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): get_auth_request con 0x7fecd0004da0 auth_method 0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): get_auth_request con 0x7fecd0004210 auth_method 0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): get_auth_request con 0x7fecd0009ab0 auth_method 0 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient(hunting): handle_auth_done global_id 34200 payload 293 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 -- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fecd0004da0 msgr2=0x7fecd0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fecd0004da0 0x7fecd0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 -- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0004210 msgr2=0x7fecd0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0004210 0x7fecd0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 monclient: found mon.a 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 1 -- 172.21.15.105:0/514791457 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd0003530 con 0x7fecd0009ab0 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9a17640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7fece9216640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0004210 0x7fecd0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:33.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 5 monclient: authenticate success, global_id 34200 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.988+0000 7feceb47a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fece9a17640 1 -- 172.21.15.105:0/514791457 reap_dead start 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fece9a17640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 0x7fecd000bf40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009f90 tx=0x7fecdc01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133924c3cc554207 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fece0007b40 con 0x7fecd0009ab0 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: dump: 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.017 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc0ac3a0 con 0x7fecd0009ab0 2024-09-17T16:30:33.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 1 -- 172.21.15.105:0/514791457 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecdc009c10 con 0x7fecd0009ab0 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 4 set_mon_vals no callback set 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7fecdbfff640 10 monclient: dump: 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.019 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: shutdown 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 msgr2=0x7fecd000bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 0x7fecd000bf40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009f90 tx=0x7fecdc01cc00 comp rx=0 tx=0).stop 2024-09-17T16:30:33.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- 172.21.15.105:0/514791457 shutdown_connections 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 --2- 172.21.15.105:0/514791457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecd0009ab0 0x7fecd000bf40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- 172.21.15.105:0/514791457 >> 172.21.15.105:0/514791457 conn(0x7fece40a8d00 msgr2=0x7fece40a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- 172.21.15.105:0/514791457 shutdown_connections 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- 172.21.15.105:0/514791457 wait complete. 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: monmap: 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:33.021 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 librados: starting msgr at 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 librados: starting objecter 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 Processor -- start 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- start start 2024-09-17T16:30:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 librados: setting wanted keys 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 librados: calling monclient init 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: init 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient: _add_conns ranks=[2,6,8,3,5,7,0,1,4] 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-b con 0x7fecd0009ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fece41d10e0 0x7fece41d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-d con 0x7fece41d10e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fece41263f0 0x7fece4126850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): picked mon.noname-e con 0x7fece41263f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fece40a6e10 con 0x7fecd0009ab0 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fece406b5b0 con 0x7fece41d10e0 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fece40a6f90 con 0x7fece41263f0 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.992+0000 7feceb47a640 10 monclient(hunting): authenticate will time out at 1116.263971s 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56542/0 (socket says 172.21.15.105:56542) 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 -- 172.21.15.105:0/4143113779 learned_addr learned my addr 172.21.15.105:0/4143113779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fece41d10e0 0x7fece41d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): get_auth_request con 0x7fecd0009ab0 auth_method 0 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fece41263f0 0x7fece4126850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): get_auth_request con 0x7fece41d10e0 auth_method 0 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): get_auth_request con 0x7fece41263f0 auth_method 0 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient(hunting): handle_auth_done global_id 23672 payload 824 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fece41263f0 msgr2=0x7fece4126850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fece41263f0 0x7fece4126850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fece41d10e0 msgr2=0x7fece41d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fece41d10e0 0x7fece41d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 monclient: found mon.noname-b 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece4129ee0 con 0x7fecd0009ab0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fece41d10e0 0x7fece41d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9216640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fece0007290 tx=0x7fece0007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.?.objecter ms_handle_connect 0x7fecd0009ab0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fece001a070 con 0x7fecd0009ab0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: dump: 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.027 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece00077c0 con 0x7fecd0009ab0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 5 monclient: authenticate success, global_id 23672 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 monclient: _renew_subs 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece412a1c0 con 0x7fecd0009ab0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fece000fe30 con 0x7fecd0009ab0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: dump: 2024-09-17T16:30:33.029 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.030 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecdb7fe640 4 set_mon_vals no callback set 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 client.23672.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 monclient: _renew_subs 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fece4140b90 con 0x7fecd0009ab0 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 1 librados: init done 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 0x7fece4122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fece4051e80 con 0x7fece4126d90 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fece000f9c0 con 0x7fecd0009ab0 2024-09-17T16:30:33.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 0x7fece4122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fecc803e550 0x7fecc8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient: get_auth_request con 0x7fece4126d90 auth_method 0 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 8321+0+0 (secure 0 0 0) 0x7fece0010040 con 0x7fecd0009ab0 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.23672.objecter ms_dispatch 0x7fece4000ba0 osd_map(104..104 src has 1..104) 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 3 client.23672.objecter handle_osd_map got epochs [104,104] > 0 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 3 client.23672.objecter handle_osd_map decoding full epoch 104 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 20 client.23672.objecter dump_active .. 0 homeless 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fecc803e550 0x7fecc8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 10 monclient: get_auth_request con 0x7fecc803e550 auth_method 0 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 10 monclient(hunting): handle_auth_done global_id 24454 payload 824 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fece406ded0 con 0x7fece4126d90 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.23672.objecter ms_handle_connect 0x7fece4126d90 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.23672.objecter resend_mon_ops 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece8a15640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 0x7fece4122680 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fece40a7e50 tx=0x7fecd4012700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 client.23672.objecter ms_handle_connect 0x7fecc803e550 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fece9a17640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fecc803e550 0x7fecc8040a10 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c7f0 tx=0x7fecdc004ac0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecd40044a0 con 0x7fece4126d90 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:32.996+0000 7fecd9ffb640 10 monclient: dump: 2024-09-17T16:30:33.033 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.034 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fecd4004030 con 0x7fece4126d90 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 msgr2=0x7fece4122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 0x7fece4122680 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fece40a7e50 tx=0x7fecd4012700 comp rx=0 tx=0).stop 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 msgr2=0x7fece4122680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.000+0000 7fecd9ffb640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4126d90 0x7fece4122680 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7feceb47a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7feceb47a640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 0x7fece4122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7feceb47a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fece40a72c0 con 0x7fece4122bc0 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 0x7fece4122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient: get_auth_request con 0x7fece4122bc0 auth_method 0 2024-09-17T16:30:33.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 10 monclient(hunting): handle_auth_done global_id 24463 payload 824 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 1 -- 172.21.15.105:0/4143113779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fece4129350 con 0x7fece4122bc0 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fece8a15640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 0x7fece4122fd0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fece40a7e50 tx=0x7fecd4004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fecd9ffb640 10 client.23672.objecter ms_handle_connect 0x7fece4122bc0 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.004+0000 7fecd9ffb640 10 client.23672.objecter resend_mon_ops 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecd4003d60 con 0x7fece4122bc0 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:33.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 10 monclient: dump: 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.039 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fecd40038f0 con 0x7fece4122bc0 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 msgr2=0x7fece4122fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 0x7fece4122fd0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fece40a7e50 tx=0x7fecd4004740 comp rx=0 tx=0).stop 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 msgr2=0x7fece4122fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fecd9ffb640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fece4122bc0 0x7fece4122fd0 secure :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fece40a7e50 tx=0x7fecd4004740 comp rx=0 tx=0).stop 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:33.040 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180608, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:33.041 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.042 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.043 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.044 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.045 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.047 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.050 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.052 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:33.054 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.057 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:33.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.060 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.061 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:33.062 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 10 librados: watch_flush enter 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 10 librados: watch_flush exit 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 20 client.23672.objecter shutdown clearing up homeless session... 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 10 client.23672.objecter successfully canceled tick 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fecc803e550 msgr2=0x7fecc8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fecc803e550 0x7fecc8040a10 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c7f0 tx=0x7fecdc004ac0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 10 monclient: shutdown 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 msgr2=0x7fece41289c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fece0007290 tx=0x7fece0007690 comp rx=0 tx=0).stop 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7fece9a17640 1 -- 172.21.15.105:0/4143113779 reap_dead start 2024-09-17T16:30:33.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 shutdown_connections 2024-09-17T16:30:33.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 --2- 172.21.15.105:0/4143113779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fecd0009ab0 0x7fece41289c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 >> 172.21.15.105:0/4143113779 conn(0x7fece40a8d00 msgr2=0x7fece409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 shutdown_connections 2024-09-17T16:30:33.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 -- 172.21.15.105:0/4143113779 wait complete. 2024-09-17T16:30:33.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.008+0000 7feceb47a640 1 librados: shutdown 2024-09-17T16:30:33.065 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:33.065 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:30:33.065 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:30:33.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: get_monmap_and_config 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: monmap: 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:33.176943+0000 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:33.176943+0000 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.179 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:33.180 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 Processor -- start 2024-09-17T16:30:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 -- start start 2024-09-17T16:30:33.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: init 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient: _add_conns ranks=[10,6,3,4,9,1,8,0,5,7,2] 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a5860 0x7fd1e00a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-f con 0x7fd1e00a5860 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-d con 0x7fd1e00a18f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 0x7fd1e0137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-g con 0x7fd1e00a4070 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00a66c0 con 0x7fd1e00a5860 2024-09-17T16:30:33.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e00a1d70 con 0x7fd1e00a4070 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd1e00a33d0 con 0x7fd1e00a18f0 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1e5ede640 10 monclient(hunting): authenticate will time out at 1116.443372s 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37510/0 (socket says 172.21.15.105:37510) 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 -- 172.21.15.105:0/3495911124 learned_addr learned my addr 172.21.15.105:0/3495911124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): get_auth_request con 0x7fd1e00a18f0 auth_method 0 2024-09-17T16:30:33.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/3495911124 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a5860 0x7fd1e00a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1dffff640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 0x7fd1e0137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request con 0x7fd1e00a5860 auth_method 0 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient(hunting): handle_auth_done global_id 23415 payload 274 2024-09-17T16:30:33.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 -- 172.21.15.105:0/3495911124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 msgr2=0x7fd1e0137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 0x7fd1e0137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 -- 172.21.15.105:0/3495911124 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a5860 msgr2=0x7fd1e00a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 --2- 172.21.15.105:0/3495911124 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a5860 0x7fd1e00a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 monclient: found mon.noname-d 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1deffd640 1 -- 172.21.15.105:0/3495911124 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e00a36b0 con 0x7fd1e00a18f0 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1dffff640 10 monclient: get_auth_request con 0x7fd1e00a4070 auth_method 0 2024-09-17T16:30:33.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.172+0000 7fd1dffff640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 0x7fd1e0137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:33.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1deffd640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009870 tx=0x7fd1d401c160 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1d0947ee247e35e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 1 -- 172.21.15.105:0/3495911124 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d40b0070 con 0x7fd1e00a18f0 2024-09-17T16:30:33.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:33.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: dump: 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.194 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 1 -- 172.21.15.105:0/3495911124 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d40aa810 con 0x7fd1e00a18f0 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 1 -- 172.21.15.105:0/3495911124 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d40a9300 con 0x7fd1e00a18f0 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 5 monclient: authenticate success, global_id 23415 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 4 set_mon_vals no callback set 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1de7fc640 10 monclient: dump: 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.196 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 10 monclient: shutdown 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- 172.21.15.105:0/3495911124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 msgr2=0x7fd1e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009870 tx=0x7fd1d401c160 comp rx=0 tx=0).stop 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- 172.21.15.105:0/3495911124 shutdown_connections 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4070 0x7fd1e0137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/3495911124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd1e00a18f0 0x7fd1e00a6180 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/3495911124 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a5860 0x7fd1e00a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- 172.21.15.105:0/3495911124 >> 172.21.15.105:0/3495911124 conn(0x7fd1e00aa450 msgr2=0x7fd1e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- 172.21.15.105:0/3495911124 shutdown_connections 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- 172.21.15.105:0/3495911124 wait complete. 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 10 monclient: monmap: 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.198 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 librados: starting msgr at 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 librados: starting objecter 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 Processor -- start 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 -- start start 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 librados: setting wanted keys 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 1 librados: calling monclient init 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.176+0000 7fd1e5ede640 10 monclient: init 2024-09-17T16:30:33.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _add_conns ranks=[3,0,8,1,5,4,6,7,2] 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a18f0 0x7fd1e0136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-g con 0x7fd1e00a18f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 0x7fd1e0136a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-a con 0x7fd1e00a4070 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a5860 0x7fd1e01204a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): picked mon.noname-e con 0x7fd1e00a5860 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0141480 con 0x7fd1e00a4070 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e0141300 con 0x7fd1e00a18f0 2024-09-17T16:30:33.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fd1e0141600 con 0x7fd1e00a5860 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): authenticate will time out at 1116.448265s 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 0x7fd1e0136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a5860 0x7fd1e01204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 0x7fd1e0136a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51944/0 (socket says 172.21.15.105:51944) 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 -- 172.21.15.105:0/4087868730 learned_addr learned my addr 172.21.15.105:0/4087868730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): get_auth_request con 0x7fd1e00a5860 auth_method 0 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): get_auth_request con 0x7fd1e00a4070 auth_method 0 2024-09-17T16:30:33.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a18f0 0x7fd1e0136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request con 0x7fd1e00a18f0 auth_method 0 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient(hunting): handle_auth_done global_id 34209 payload 824 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a5860 msgr2=0x7fd1e01204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a5860 0x7fd1e01204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a18f0 msgr2=0x7fd1e0136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a18f0 0x7fd1e0136500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 monclient: found mon.noname-a 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e013d8f0 con 0x7fd1e00a4070 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a5860 0x7fd1e01204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1deffd640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 0x7fd1e0136a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40ae040 tx=0x7fd1d4018f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.?.objecter ms_handle_connect 0x7fd1e00a4070 2024-09-17T16:30:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d40b9070 con 0x7fd1e00a4070 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: dump: 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.204 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d4003620 con 0x7fd1e00a4070 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d40b0040 con 0x7fd1e00a4070 2024-09-17T16:30:33.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 5 monclient: authenticate success, global_id 34209 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _renew_subs 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e00a8e40 con 0x7fd1e00a4070 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1ddffb640 4 set_mon_vals no callback set 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 monclient: dump: 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.206 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 client.34209.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _renew_subs 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e0123900 con 0x7fd1e00a4070 2024-09-17T16:30:33.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 librados: init done 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 0x7fd1e0120dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e01c69a0 con 0x7fd1e01209e0 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd1d4004100 con 0x7fd1e00a4070 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 0x7fd1e0120dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1cc03e580 0x7fd1cc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient: get_auth_request con 0x7fd1e01209e0 auth_method 0 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8321+0+0 (secure 0 0 0) 0x7fd1d40e9c60 con 0x7fd1e00a4070 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.34209.objecter ms_dispatch 0x7fd1e0000ba0 osd_map(104..104 src has 1..104) 2024-09-17T16:30:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 3 client.34209.objecter handle_osd_map got epochs [104,104] > 0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 3 client.34209.objecter handle_osd_map decoding full epoch 104 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 20 client.34209.objecter dump_active .. 0 homeless 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1cc03e580 0x7fd1cc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 10 monclient: get_auth_request con 0x7fd1cc03e580 auth_method 0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_done global_id 19020 payload 824 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1e00a5df0 con 0x7fd1e01209e0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.34209.objecter ms_handle_connect 0x7fd1cc03e580 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1dffff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1cc03e580 0x7fd1cc040a40 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800d8a0 tx=0x7fd1d8007780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.34209.objecter ms_handle_connect 0x7fd1e01209e0 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1c7fff640 10 client.34209.objecter resend_mon_ops 2024-09-17T16:30:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.180+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 0x7fd1e0120dc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d000f900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d0019030 con 0x7fd1e01209e0 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 10 monclient: dump: 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1d0014670 con 0x7fd1e01209e0 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 msgr2=0x7fd1e0120dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 0x7fd1e0120dc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d000f900 comp rx=0 tx=0).stop 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 msgr2=0x7fd1e0120dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.184+0000 7fd1c7fff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e01209e0 0x7fd1e0120dc0 secure :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d000f900 comp rx=0 tx=0).stop 2024-09-17T16:30:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1e5ede640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 0x7fd1e006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1e5ede640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e0051e80 con 0x7fd1e0121300 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 0x7fd1e006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient: get_auth_request con 0x7fd1e0121300 auth_method 0 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 10 monclient(hunting): handle_auth_done global_id 19029 payload 824 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.188+0000 7fd1df7fe640 1 -- 172.21.15.105:0/4087868730 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd1e0069fa0 con 0x7fd1e0121300 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1df7fe640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 0x7fd1e006e230 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d00046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 client.34209.objecter ms_handle_connect 0x7fd1e0121300 2024-09-17T16:30:33.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 client.34209.objecter resend_mon_ops 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d001f240 con 0x7fd1e0121300 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 monclient: dump: 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.215 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7fd1d001f240 con 0x7fd1e0121300 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 msgr2=0x7fd1e006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 0x7fd1e006e230 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d00046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 msgr2=0x7fd1e006e230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1c7fff640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e0121300 0x7fd1e006e230 secure :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000f440 tx=0x7fd1d00046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180816, 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:33.217 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.218 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.219 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:33.220 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:33.221 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.222 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:33.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.225 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.230 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:33.231 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.232 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.234 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.235 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.236 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.237 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.238 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.239 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.240 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.240 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.240 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:33.240 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 10 librados: watch_flush enter 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 10 librados: watch_flush exit 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 20 client.34209.objecter shutdown clearing up homeless session... 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 10 client.34209.objecter successfully canceled tick 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1cc03e580 msgr2=0x7fd1cc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1cc03e580 0x7fd1cc040a40 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800d8a0 tx=0x7fd1d8007780 comp rx=0 tx=0).stop 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 10 monclient: shutdown 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 msgr2=0x7fd1e0136a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 --2- 172.21.15.105:0/4087868730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4070 0x7fd1e0136a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40ae040 tx=0x7fd1d4018f00 comp rx=0 tx=0).stop 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1dffff640 1 -- 172.21.15.105:0/4087868730 reap_dead start 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1dffff640 1 -- 172.21.15.105:0/4087868730 reap_dead start 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 shutdown_connections 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 >> 172.21.15.105:0/4087868730 conn(0x7fd1e00aa450 msgr2=0x7fd1e00a7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 shutdown_connections 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 -- 172.21.15.105:0/4087868730 wait complete. 2024-09-17T16:30:33.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.192+0000 7fd1e5ede640 1 librados: shutdown 2024-09-17T16:30:33.243 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:33.243 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:30:33.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:30:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: get_monmap_and_config 2024-09-17T16:30:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: monmap: 2024-09-17T16:30:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:33.356507+0000 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:33.356507+0000 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:33.360 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 Processor -- start 2024-09-17T16:30:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 -- start start 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: init 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient: _add_conns ranks=[1,4,7,6,0,3,10,9,8,2,5] 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc66c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-c con 0x7fc66c0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc66c0a73d0 0x7fc66c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-i con 0x7fc66c0a73d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc66c06ebc0 con 0x7fc66c0a68e0 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc66c0a3fc0 con 0x7fc66c0a4910 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc66c0a63c0 con 0x7fc66c0a73d0 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc67476a640 10 monclient(hunting): authenticate will time out at 1116.622474s 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46408/0 (socket says 172.21.15.105:46408) 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 -- 172.21.15.105:0/4137462557 learned_addr learned my addr 172.21.15.105:0/4137462557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): get_auth_request con 0x7fc66c0a4910 auth_method 0 2024-09-17T16:30:33.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc672d07640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc66c0a73d0 0x7fc66c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc672d07640 10 monclient(hunting): get_auth_request con 0x7fc66c0a73d0 auth_method 0 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc672d07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc672d07640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc672d07640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient(hunting): handle_auth_done global_id 22558 payload 274 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 -- 172.21.15.105:0/4137462557 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc66c0a73d0 msgr2=0x7fc66c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc66c0a73d0 0x7fc66c141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 -- 172.21.15.105:0/4137462557 >> v1:172.21.15.160:6789/0 conn(0x7fc66c0a68e0 legacy=0x7fc66c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 monclient: found mon.noname-c 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.352+0000 7fc671d05640 1 -- 172.21.15.105:0/4137462557 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c0a3690 con 0x7fc66c0a4910 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc6600098a0 tx=0x7fc66001c410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=199439e13e777b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 1 -- 172.21.15.105:0/4137462557 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6600b2070 con 0x7fc66c0a4910 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: dump: 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 1 -- 172.21.15.105:0/4137462557 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6600a9070 con 0x7fc66c0a4910 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 1 -- 172.21.15.105:0/4137462557 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6600abc40 con 0x7fc66c0a4910 2024-09-17T16:30:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 5 monclient: authenticate success, global_id 22558 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 4 set_mon_vals no callback set 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671504640 10 monclient: dump: 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: shutdown 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- 172.21.15.105:0/4137462557 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 msgr2=0x7fc66c13eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fc6600098a0 tx=0x7fc66001c410 comp rx=0 tx=0).stop 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- 172.21.15.105:0/4137462557 shutdown_connections 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc66c0a73d0 0x7fc66c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- 172.21.15.105:0/4137462557 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc66c0a4910 0x7fc66c13eaf0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- 172.21.15.105:0/4137462557 >> 172.21.15.105:0/4137462557 conn(0x7fc66c068ba0 msgr2=0x7fc66c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- 172.21.15.105:0/4137462557 shutdown_connections 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- 172.21.15.105:0/4137462557 wait complete. 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: monmap: 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 librados: starting msgr at 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 librados: starting objecter 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 Processor -- start 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- start start 2024-09-17T16:30:33.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 librados: setting wanted keys 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 librados: calling monclient init 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: init 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient: _add_conns ranks=[3,2,0,8,1,7,6,4,5] 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66c0a4910 0x7fc66c12a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-g con 0x7fc66c0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc66c0a68e0 0x7fc66c1264e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-b con 0x7fc66c0a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): picked mon.noname-a con 0x7fc66c0a73d0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c143f20 con 0x7fc66c0a73d0 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc66c143da0 con 0x7fc66c0a68e0 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc66c1440a0 con 0x7fc66c0a4910 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc67476a640 10 monclient(hunting): authenticate will time out at 1116.627718s 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc66c0a68e0 0x7fc66c1264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51952/0 (socket says 172.21.15.105:51952) 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 1 -- 172.21.15.105:0/2544886371 learned_addr learned my addr 172.21.15.105:0/2544886371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): get_auth_request con 0x7fc66c0a73d0 auth_method 0 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 10 monclient(hunting): get_auth_request con 0x7fc66c0a68e0 auth_method 0 2024-09-17T16:30:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc671d05640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.356+0000 7fc672d07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66c0a4910 0x7fc66c12a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): get_auth_request con 0x7fc66c0a4910 auth_method 0 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 10 monclient(hunting): handle_auth_done global_id 34218 payload 824 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66c0a4910 msgr2=0x7fc66c12a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66c0a4910 0x7fc66c12a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc66c0a68e0 msgr2=0x7fc66c1264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc66c0a68e0 0x7fc66c1264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 monclient: found mon.noname-a 2024-09-17T16:30:33.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c129300 con 0x7fc66c0a73d0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672d07640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc664007e60 tx=0x7fc66400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.?.objecter ms_handle_connect 0x7fc66c0a73d0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc66400edd0 con 0x7fc66c0a73d0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: dump: 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.388 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.389 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 5 monclient: authenticate success, global_id 34218 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 monclient: _renew_subs 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc664019070 con 0x7fc66c0a73d0 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c127e10 con 0x7fc66c0a73d0 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc664014d00 con 0x7fc66c0a73d0 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 client.34218.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: dump: 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc670d03640 4 set_mon_vals no callback set 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 monclient: _renew_subs 2024-09-17T16:30:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c128c00 con 0x7fc66c0a73d0 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 1 librados: init done 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc664014420 con 0x7fc66c0a73d0 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 0x7fc66c1d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc66c129df0 con 0x7fc66c126f60 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc65803e550 0x7fc658040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8321+0+0 (secure 0 0 0) 0x7fc664050f00 con 0x7fc66c0a73d0 2024-09-17T16:30:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.34218.objecter ms_dispatch 0x7fc66c000ba0 osd_map(104..104 src has 1..104) 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 3 client.34218.objecter handle_osd_map got epochs [104,104] > 0 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 3 client.34218.objecter handle_osd_map decoding full epoch 104 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 0x7fc66c1d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient: get_auth_request con 0x7fc66c126f60 auth_method 0 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 20 client.34218.objecter dump_active .. 0 homeless 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc65803e550 0x7fc658040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 10 monclient: get_auth_request con 0x7fc65803e550 auth_method 0 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 10 monclient(hunting): handle_auth_done global_id 19121 payload 824 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc66c0a2530 con 0x7fc66c126f60 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc672506640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 0x7fc66c1d1100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00e4b0 tx=0x7fc65c00e970 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.34218.objecter ms_handle_connect 0x7fc66c126f60 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.34218.objecter resend_mon_ops 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 client.34218.objecter ms_handle_connect 0x7fc65803e550 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc671d05640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc65803e550 0x7fc658040a10 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7fc6600af000 tx=0x7fc6600a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc65c0052e0 con 0x7fc66c126f60 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.360+0000 7fc66affd640 10 monclient: dump: 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.396 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc65c018070 con 0x7fc66c126f60 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:33.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 msgr2=0x7fc66c1d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 0x7fc66c1d1100 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00e4b0 tx=0x7fc65c00e970 comp rx=0 tx=0).stop 2024-09-17T16:30:33.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 msgr2=0x7fc66c1d1100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.364+0000 7fc66affd640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c126f60 0x7fc66c1d1100 secure :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00e4b0 tx=0x7fc65c00e970 comp rx=0 tx=0).stop 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc67476a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc67476a640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 0x7fc66c1d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc67476a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc66c1d1f90 con 0x7fc66c1d1640 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 0x7fc66c1d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient: get_auth_request con 0x7fc66c1d1640 auth_method 0 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 10 monclient(hunting): handle_auth_done global_id 19130 payload 824 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 1 -- 172.21.15.105:0/2544886371 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc66c127340 con 0x7fc66c1d1640 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc672506640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 0x7fc66c1d1a50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc65c017000 tx=0x7fc65c001130 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 10 client.34218.objecter ms_handle_connect 0x7fc66c1d1640 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 10 client.34218.objecter resend_mon_ops 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc65c01de30 con 0x7fc66c1d1640 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.368+0000 7fc66affd640 10 monclient: dump: 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.402 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fc65c003810 con 0x7fc66c1d1640 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 msgr2=0x7fc66c1d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 0x7fc66c1d1a50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc65c017000 tx=0x7fc65c001130 comp rx=0 tx=0).stop 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 msgr2=0x7fc66c1d1a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc66affd640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc66c1d1640 0x7fc66c1d1a50 secure :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc65c017000 tx=0x7fc65c001130 comp rx=0 tx=0).stop 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 180972, 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:33.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.405 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:33.406 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.407 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:33.408 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.409 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:33.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:33.412 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.413 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:33.414 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.415 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:33.416 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:33.417 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.418 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.419 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.422 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.424 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.426 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:33.427 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.428 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.429 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.430 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:33.431 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.431 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.431 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:33.431 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 10 librados: watch_flush enter 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 10 librados: watch_flush exit 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 20 client.34218.objecter shutdown clearing up homeless session... 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 10 client.34218.objecter successfully canceled tick 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc65803e550 msgr2=0x7fc658040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc65803e550 0x7fc658040a10 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7fc6600af000 tx=0x7fc6600a7000 comp rx=0 tx=0).stop 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 10 monclient: shutdown 2024-09-17T16:30:33.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 msgr2=0x7fc66c126a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc664007e60 tx=0x7fc66400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc672d07640 1 -- 172.21.15.105:0/2544886371 reap_dead start 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 shutdown_connections 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 --2- 172.21.15.105:0/2544886371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc66c0a73d0 0x7fc66c126a20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 >> 172.21.15.105:0/2544886371 conn(0x7fc66c068ba0 msgr2=0x7fc66c09de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 shutdown_connections 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 -- 172.21.15.105:0/2544886371 wait complete. 2024-09-17T16:30:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.372+0000 7fc67476a640 1 librados: shutdown 2024-09-17T16:30:33.434 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:33.435 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:30:33.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: get_monmap_and_config 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: monmap: 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:33.555964+0000 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:33.555964+0000 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 Processor -- start 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 -- start start 2024-09-17T16:30:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: init 2024-09-17T16:30:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient: _add_conns ranks=[5,1,8,9,6,10,4,2,7,0,3] 2024-09-17T16:30:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6c00a5860 0x7fb6c00a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-h con 0x7fb6c00a5860 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:33.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb6c00a18f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-e con 0x7fb6c00a4070 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c00a1d70 con 0x7fb6c00a18f0 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb6c00a66c0 con 0x7fb6c00a5860 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb6c00a33d0 con 0x7fb6c00a4070 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6c600a640 10 monclient(hunting): authenticate will time out at 1116.821636s 2024-09-17T16:30:33.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45218/0 (socket says 172.21.15.105:45218) 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 -- 172.21.15.105:0/2088089835 learned_addr learned my addr 172.21.15.105:0/2088089835 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): get_auth_request con 0x7fb6c00a4070 auth_method 0 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6c00a5860 0x7fb6c00a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c00a5860 auth_method 0 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient(hunting): handle_auth_done global_id 23390 payload 274 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 -- 172.21.15.105:0/2088089835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6c00a5860 msgr2=0x7fb6c00a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6c00a5860 0x7fb6c00a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 -- 172.21.15.105:0/2088089835 >> v1:172.21.15.160:6789/0 conn(0x7fb6c00a18f0 legacy=0x7fb6c0137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 monclient: found mon.noname-e 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6be7fc640 1 -- 172.21.15.105:0/2088089835 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510603129 0 0) 0x7fb6b4001120 con 0x7fb6c00a18f0 2024-09-17T16:30:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6bffff640 1 -- 172.21.15.105:0/2088089835 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c00a36b0 con 0x7fb6c00a4070 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.552+0000 7fb6be7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009e80 tx=0x7fb6b801c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5421dff42765f56d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 1 -- 172.21.15.105:0/2088089835 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b80090d0 con 0x7fb6c00a4070 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: dump: 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 1 -- 172.21.15.105:0/2088089835 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b80a5b40 con 0x7fb6c00a4070 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 1 -- 172.21.15.105:0/2088089835 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b80a5cf0 con 0x7fb6c00a4070 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6be7fc640 10 monclient: dump: 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 5 monclient: authenticate success, global_id 23390 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 4 set_mon_vals no callback set 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: shutdown 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/2088089835 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 msgr2=0x7fb6c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009e80 tx=0x7fb6b801c910 comp rx=0 tx=0).stop 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/2088089835 shutdown_connections 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6c00a4070 0x7fb6c00a6180 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- 172.21.15.105:0/2088089835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6c00a5860 0x7fb6c00a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/2088089835 >> 172.21.15.105:0/2088089835 conn(0x7fb6c00aa450 msgr2=0x7fb6c00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/2088089835 shutdown_connections 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/2088089835 wait complete. 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: monmap: 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 librados: starting msgr at 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 librados: starting objecter 2024-09-17T16:30:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 Processor -- start 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- start start 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 librados: setting wanted keys 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 librados: calling monclient init 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: init 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _add_conns ranks=[3,2,1,0,8,6,4,5,7] 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6c00a18f0 0x7fb6c01d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-g con 0x7fb6c00a18f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 0x7fb6c01d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-b con 0x7fb6c00a4070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6c00a5860 0x7fb6c006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): picked mon.noname-f con 0x7fb6c00a5860 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0142490 con 0x7fb6c00a5860 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c0142310 con 0x7fb6c00a4070 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c0142610 con 0x7fb6c00a18f0 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient(hunting): authenticate will time out at 1116.826669s 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 0x7fb6c01d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 0x7fb6c01d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56552/0 (socket says 172.21.15.105:56552) 2024-09-17T16:30:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 -- 172.21.15.105:0/554290262 learned_addr learned my addr 172.21.15.105:0/554290262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): get_auth_request con 0x7fb6c00a4070 auth_method 0 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6c00a18f0 0x7fb6c01d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6c00a5860 0x7fb6c006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c00a18f0 auth_method 0 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): get_auth_request con 0x7fb6c00a5860 auth_method 0 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient(hunting): handle_auth_done global_id 23699 payload 824 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6c00a18f0 msgr2=0x7fb6c01d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6c00a18f0 0x7fb6c01d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6c00a5860 msgr2=0x7fb6c006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6c00a5860 0x7fb6c006d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 monclient: found mon.noname-b 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c013d9c0 con 0x7fb6c00a4070 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6beffd640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 0x7fb6c01d1d40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c700 tx=0x7fb6b400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 client.?.objecter ms_handle_connect 0x7fb6c00a4070 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b400ccf0 con 0x7fb6c00a4070 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.588 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.589 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b400a640 con 0x7fb6c00a4070 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b400ac60 con 0x7fb6c00a4070 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 5 monclient: authenticate success, global_id 23699 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _renew_subs 2024-09-17T16:30:33.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c013dca0 con 0x7fb6c00a4070 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6bdffb640 4 set_mon_vals no callback set 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.591 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 client.23699.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:33.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _renew_subs 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c00a07b0 con 0x7fb6c00a4070 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.556+0000 7fb6c600a640 1 librados: init done 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6c600a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6c600a640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 0x7fb6c006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb6c0051e80 con 0x7fb6c006d760 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb6b400a7f0 con 0x7fb6c00a4070 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 0x7fb6c006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient: get_auth_request con 0x7fb6c006d760 auth_method 0 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb6ac03e550 0x7fb6ac040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(105..105 src has 1..105) ==== 8210+0+0 (secure 0 0 0) 0x7fb6b4050300 con 0x7fb6c00a4070 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 client.23699.objecter ms_dispatch 0x7fb6c0000ba0 osd_map(105..105 src has 1..105) 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 3 client.23699.objecter handle_osd_map got epochs [105,105] > 0 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 3 client.23699.objecter handle_osd_map decoding full epoch 105 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_done global_id 34180 payload 824 2024-09-17T16:30:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 20 client.23699.objecter dump_active .. 0 homeless 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6c00a6140 con 0x7fb6c006d760 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bffff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb6ac03e550 0x7fb6ac040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bffff640 10 monclient: get_auth_request con 0x7fb6ac03e550 auth_method 0 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 0x7fb6c006db40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0011900 tx=0x7fb6b0011dc0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 client.23699.objecter ms_handle_connect 0x7fb6c006d760 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 client.23699.objecter resend_mon_ops 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b0019070 con 0x7fb6c006d760 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:33.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 client.23699.objecter ms_handle_connect 0x7fb6ac03e550 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6bffff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb6ac03e550 0x7fb6ac040a10 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009e80 tx=0x7fb6b80a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb6b0004300 con 0x7fb6c006d760 2024-09-17T16:30:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 msgr2=0x7fb6c006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 0x7fb6c006db40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0011900 tx=0x7fb6b0011dc0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 msgr2=0x7fb6c006db40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.560+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006d760 0x7fb6c006db40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.564+0000 7fb6c600a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.564+0000 7fb6c600a640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 0x7fb6c0138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.564+0000 7fb6c600a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.564+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb6c0074a30 con 0x7fb6c006e080 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 0x7fb6c0138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient: get_auth_request con 0x7fb6c006e080 auth_method 0 2024-09-17T16:30:33.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 10 monclient(hunting): handle_auth_done global_id 34189 payload 824 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 1 -- 172.21.15.105:0/554290262 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6c01375e0 con 0x7fb6c006e080 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bf7fe640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 0x7fb6c0138690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0011900 tx=0x7fb6b00046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 client.23699.objecter ms_handle_connect 0x7fb6c006e080 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 client.23699.objecter resend_mon_ops 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6b0019040 con 0x7fb6c006e080 2024-09-17T16:30:33.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.602 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7fb6b001e450 con 0x7fb6c006e080 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 msgr2=0x7fb6c0138690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 0x7fb6c0138690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0011900 tx=0x7fb6b00046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 msgr2=0x7fb6c0138690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:33.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb6c006e080 0x7fb6c0138690 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 24367, 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:33.604 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:30:33.605 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.606 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.607 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:33.608 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:33.609 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.610 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.612 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.614 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.615 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.616 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.618 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.620 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.625 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.628 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:33.629 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:33.630 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:33.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 10 librados: watch_flush enter 2024-09-17T16:30:33.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 10 librados: watch_flush exit 2024-09-17T16:30:33.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 20 client.23699.objecter shutdown clearing up homeless session... 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 10 client.23699.objecter successfully canceled tick 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb6ac03e550 msgr2=0x7fb6ac040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb6ac03e550 0x7fb6ac040a10 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009e80 tx=0x7fb6b80a7070 comp rx=0 tx=0).stop 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 10 monclient: shutdown 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 msgr2=0x7fb6c01d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 --2- 172.21.15.105:0/554290262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6c00a4070 0x7fb6c01d1d40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c700 tx=0x7fb6b400cbc0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bffff640 1 -- 172.21.15.105:0/554290262 reap_dead start 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6bffff640 1 -- 172.21.15.105:0/554290262 reap_dead start 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 shutdown_connections 2024-09-17T16:30:33.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 >> 172.21.15.105:0/554290262 conn(0x7fb6c00aa450 msgr2=0x7fb6c00a9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 shutdown_connections 2024-09-17T16:30:33.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 -- 172.21.15.105:0/554290262 wait complete. 2024-09-17T16:30:33.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.568+0000 7fb6c600a640 1 librados: shutdown 2024-09-17T16:30:33.634 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:33.634 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:30:33.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.736+0000 7fe70ab46640 10 monclient: get_monmap_and_config 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.736+0000 7fe70ab46640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.736+0000 7fe70ab46640 10 monclient: monmap: 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:33.743002+0000 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:33.743002+0000 2024-09-17T16:30:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:33.747 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 Processor -- start 2024-09-17T16:30:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 -- start start 2024-09-17T16:30:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient: init 2024-09-17T16:30:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient: _add_conns ranks=[5,10,6,7,3,2,0,4,8,9,1] 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7040a68e0 0x7fe7040a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-h con 0x7fe7040a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 0x7fe7040a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-f con 0x7fe7040a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-d con 0x7fe7041396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe7040a3fc0 con 0x7fe7040a4910 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe7040a63c0 con 0x7fe7040a68e0 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe7040a7740 con 0x7fe7041396a0 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient(hunting): authenticate will time out at 1117.008913s 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37524/0 (socket says 172.21.15.105:37524) 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 -- 172.21.15.105:0/602800444 learned_addr learned my addr 172.21.15.105:0/602800444 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): get_auth_request con 0x7fe7041396a0 auth_method 0 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7088e2640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7040a68e0 0x7fe7040a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fbfff640 1 --2- 172.21.15.105:0/602800444 >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 0x7fe7040a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7088e2640 10 monclient(hunting): get_auth_request con 0x7fe7040a68e0 auth_method 0 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7088e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7088e2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7088e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient(hunting): handle_auth_done global_id 23442 payload 274 2024-09-17T16:30:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 -- 172.21.15.105:0/602800444 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7040a68e0 msgr2=0x7fe7040a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7040a68e0 0x7fe7040a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 -- 172.21.15.105:0/602800444 >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 msgr2=0x7fe7040a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 --2- 172.21.15.105:0/602800444 >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 0x7fe7040a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 monclient: found mon.noname-d 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 -- 172.21.15.105:0/602800444 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70406ebc0 con 0x7fe7041396a0 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fbfff640 10 monclient: get_auth_request con 0x7fe7040a4910 auth_method 0 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fbfff640 1 --2- 172.21.15.105:0/602800444 >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 0x7fe7040a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe7090e3640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e50 tx=0x7fe6fc01c8c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f9558634d669bc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 1 -- 172.21.15.105:0/602800444 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6fc0090d0 con 0x7fe7041396a0 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: dump: 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 1 -- 172.21.15.105:0/602800444 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0a5b90 con 0x7fe7041396a0 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 5 monclient: authenticate success, global_id 23442 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 1 -- 172.21.15.105:0/602800444 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6fc0a5d80 con 0x7fe7041396a0 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe70ab46640 4 set_mon_vals no callback set 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.740+0000 7fe6fb7fe640 10 monclient: dump: 2024-09-17T16:30:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: shutdown 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/602800444 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 msgr2=0x7fe704139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e50 tx=0x7fe6fc01c8c0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/602800444 shutdown_connections 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7041396a0 0x7fe704139b50 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- 172.21.15.105:0/602800444 >> v2:172.21.15.160:3300/0 conn(0x7fe7040a4910 0x7fe7040a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- 172.21.15.105:0/602800444 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7040a68e0 0x7fe7040a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/602800444 >> 172.21.15.105:0/602800444 conn(0x7fe704068ba0 msgr2=0x7fe704068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/602800444 shutdown_connections 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/602800444 wait complete. 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: build_initial_monmap 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: monmap: 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 librados: starting msgr at 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 librados: starting objecter 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 Processor -- start 2024-09-17T16:30:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- start start 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 librados: setting wanted keys 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 librados: calling monclient init 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: init 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _add_conns ranks=[8,7,3,5,0,6,4,1,2] 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-e con 0x7fe7040a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7040a68e0 0x7fe7041d1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-i con 0x7fe7040a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7041396a0 0x7fe7041d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): picked mon.noname-g con 0x7fe7041396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe70406e630 con 0x7fe7041396a0 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe7040a7af0 con 0x7fe7040a68e0 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe704143320 con 0x7fe7040a4910 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient(hunting): authenticate will time out at 1117.014048s 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45234/0 (socket says 172.21.15.105:45234) 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 -- 172.21.15.105:0/947283219 learned_addr learned my addr 172.21.15.105:0/947283219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): get_auth_request con 0x7fe7040a4910 auth_method 0 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6fbfff640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7040a68e0 0x7fe7041d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6fbfff640 10 monclient(hunting): get_auth_request con 0x7fe7040a68e0 auth_method 0 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7090e3640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7041396a0 0x7fe7041d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7090e3640 10 monclient(hunting): get_auth_request con 0x7fe7041396a0 auth_method 0 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7090e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7090e3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7090e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient(hunting): handle_auth_done global_id 23399 payload 824 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 -- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7040a68e0 msgr2=0x7fe7041d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7040a68e0 0x7fe7041d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 -- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7041396a0 msgr2=0x7fe7041d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7041396a0 0x7fe7041d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient: _finish_hunting 0 2024-09-17T16:30:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 monclient: found mon.noname-e 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 -- 172.21.15.105:0/947283219 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7040a9e70 con 0x7fe7040a4910 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe7088e2640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d800 tx=0x7fe70000dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 client.?.objecter ms_handle_connect 0x7fe7040a4910 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe70000bb30 con 0x7fe7040a4910 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: dump: 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: _finish_auth 0 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: _check_auth_tickets 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700019070 con 0x7fe7040a4910 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 5 monclient: authenticate success, global_id 23399 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _renew_subs 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70413f690 con 0x7fe7040a4910 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe70000bcf0 con 0x7fe7040a4910 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 client.23399.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6faffd640 4 set_mon_vals no callback set 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 10 monclient: dump: 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.775 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:33.776 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _renew_subs 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7040ab9a0 con 0x7fe7040a4910 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 librados: init done 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe70000b6c0 con 0x7fe7040a4910 2024-09-17T16:30:33.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.744+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704139b50 con 0x7fe7040a4910 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 0x7fe6f0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(105..105 src has 1..105) ==== 8210+0+0 (secure 0 0 0) 0x7fe70000f040 con 0x7fe7040a4910 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 10 client.23399.objecter ms_dispatch 0x7fe704000ba0 osd_map(105..105 src has 1..105) 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 3 client.23399.objecter handle_osd_map got epochs [105,105] > 0 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 3 client.23399.objecter handle_osd_map decoding full epoch 105 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 20 client.23399.objecter dump_active .. 0 homeless 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6fbfff640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 0x7fe6f0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6fbfff640 10 monclient: get_auth_request con 0x7fe6f003e580 auth_method 0 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6f97fa640 10 client.23399.objecter ms_handle_connect 0x7fe6f003e580 2024-09-17T16:30:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.748+0000 7fe6fbfff640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 0x7fe6f0040a40 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7fe70413d2e0 tx=0x7fe6f4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.752+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe700022020 con 0x7fe7040a4910 2024-09-17T16:30:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.752+0000 7fe6f97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.752+0000 7fe6f97fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:33.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.984+0000 7fe70ab46640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:30:33.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.984+0000 7fe70ab46640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:30:33.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.984+0000 7fe70ab46640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:33.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.984+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fe7040746d0 con 0x7fe7040a4910 2024-09-17T16:30:33.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe6f97fa640 1 -- 172.21.15.105:0/947283219 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fe7040a9e70 con 0x7fe7040a4910 2024-09-17T16:30:33.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe6f97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:30:33.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe6f97fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=8,config=1,health=12,kv=10,logm=69} crc {auth=1413233592,config=1228700967,health=3458545639,kv=2863082231,logm=1363057153}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=3448240269,config=692619927,health=1886805112,kv=2863082231,logm=3671217215}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=12,kv=10,logm=66} crc {auth=3448240269,config=692619927,health=852488049,kv=2863082231,logm=333468260}) 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.000 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.991+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2330237948,config=692619927,health=2719800742,kv=2863082231,logm=373795762}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=9,kv=10,logm=70} crc {auth=1505571646,config=692619927,health=3346653861,kv=2863082231,logm=3643065237}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=10,config=2,health=12,kv=9,logm=67} crc {auth=1005096183,config=692619927,health=3751686554,kv=2502644681,logm=138179183}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=8,config=2,health=12,kv=9,logm=69} crc {auth=1416329185,config=692619927,health=3458545639,kv=3449169938,logm=87021951}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=10,logm=68} crc {auth=2234971087,config=692619927,health=3655986371,kv=2863082231,logm=377142181}) 2024-09-17T16:30:34.001 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.995+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=9,config=2,health=10,kv=10,logm=69} crc {auth=1505571646,config=692619927,health=2051106853,kv=2863082231,logm=395010509}) 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 10 librados: watch_flush enter 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 10 librados: watch_flush exit 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 20 client.23399.objecter shutdown clearing up homeless session... 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 10 client.23399.objecter successfully canceled tick 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 msgr2=0x7fe6f0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 0x7fe6f0040a40 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7fe70413d2e0 tx=0x7fe6f4008040 comp rx=0 tx=0).stop 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 10 monclient: shutdown 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 msgr2=0x7fe7041d1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d800 tx=0x7fe70000dcc0 comp rx=0 tx=0).stop 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 shutdown_connections 2024-09-17T16:30:34.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe6f003e580 0x7fe6f0040a40 unknown :-1 s=CLOSED pgs=3353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7041396a0 0x7fe7041d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7040a68e0 0x7fe7041d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 --2- 172.21.15.105:0/947283219 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7040a4910 0x7fe7041d1240 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 >> 172.21.15.105:0/947283219 conn(0x7fe704068ba0 msgr2=0x7fe7040a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 shutdown_connections 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 -- 172.21.15.105:0/947283219 wait complete. 2024-09-17T16:30:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:33.992+0000 7fe70ab46640 1 librados: shutdown 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=659676587}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1245335885}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3473959547}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3914415841}) 2024-09-17T16:30:34.006 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2734968579}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3152454580}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1374080232}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1982857424}) 2024-09-17T16:30:34.007 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:33.999+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=225496129}) 2024-09-17T16:30:34.008 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2173358634}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=669914252}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1935709379}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=671396497}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.009 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1528977818}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3033044558}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=857587035}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836995927}) 2024-09-17T16:30:34.010 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.003+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3167630101}) 2024-09-17T16:30:34.013 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.013 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=502809401}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3581758257}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1464125190}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1786877399}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2269630647}) 2024-09-17T16:30:34.014 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402425507}) 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=863152840}) 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1739243558}) 2024-09-17T16:30:34.015 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.007+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=150333943}) 2024-09-17T16:30:34.017 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.017 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.017 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3244235914}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523865543}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3610874498}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2170027023}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3562476497}) 2024-09-17T16:30:34.018 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2952974657}) 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1072695222}) 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1582556467}) 2024-09-17T16:30:34.019 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.011+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4275971907}) 2024-09-17T16:30:34.020 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:30:34.021 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1719907312}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=459339588}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3563481649}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1761406684}) 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.022 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730937930}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3890259192}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2614150495}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808454047}) 2024-09-17T16:30:34.023 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.015+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1659265729}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3248420841}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3875424871}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3101560808}) 2024-09-17T16:30:34.025 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2126243098}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139905438}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=758292181}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2368613103}) 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.026 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488099624}) 2024-09-17T16:30:34.027 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.019+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1060611509}) 2024-09-17T16:30:34.028 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.028 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.028 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292235422}) 2024-09-17T16:30:34.028 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.028 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1981400720}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600835981}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2844416263}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3577036370}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.029 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1208866507}) 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3845895206}) 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3538487677}) 2024-09-17T16:30:34.030 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.023+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1034535652}) 2024-09-17T16:30:34.032 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.032 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.032 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2891083803}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4015158351}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=89978955}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1147161335}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=929762066}) 2024-09-17T16:30:34.033 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3216767276}) 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=42391948}) 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562131240}) 2024-09-17T16:30:34.034 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.027+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067790823}) 2024-09-17T16:30:34.036 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.036 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.036 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1876087940}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1378006077}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=197630954}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1559171391}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.037 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1589075062}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3409278973}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2180205970}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056592372}) 2024-09-17T16:30:34.038 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.031+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3299861513}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3151880918}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2668803310}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=50403558}) 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.041 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1274973704}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=668535227}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=309474179}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=650865973}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462698100}) 2024-09-17T16:30:34.042 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.035+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=740638310}) 2024-09-17T16:30:34.044 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.044 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.044 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649537661}) 2024-09-17T16:30:34.044 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3067734599}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3548532176}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3545266850}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=397839559}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.045 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1359029406}) 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=38215521}) 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519803110}) 2024-09-17T16:30:34.046 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=668434799}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1039268497}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4291979212}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=709864265}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.048 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2098431059}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619653694}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.039+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4150962452}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=269672677}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409148317}) 2024-09-17T16:30:34.049 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=267301657}) 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=689404247}) 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3367414124}) 2024-09-17T16:30:34.051 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078351667}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1290953182}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486611314}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=802202987}) 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.052 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.053 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2557469498}) 2024-09-17T16:30:34.053 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.053 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=349287498}) 2024-09-17T16:30:34.053 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.043+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2577131297}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156924820}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2411899758}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=367539836}) 2024-09-17T16:30:34.055 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1170459378}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3424396117}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225080704}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2411826467}) 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.056 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597009285}) 2024-09-17T16:30:34.057 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.047+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1638034676}) 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3425033060}) 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1044122823}) 2024-09-17T16:30:34.058 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1726256064}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1567843742}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2873957102}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=197231721}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=66757832}) 2024-09-17T16:30:34.059 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.060 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820357861}) 2024-09-17T16:30:34.060 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.051+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2581774594}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=45,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=8} crc {logm=340516629,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=3263000992}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=46,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=7} crc {logm=3220967357,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=1038210086}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.062 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=43,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=8,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=596728264,mdsmap=1248375647,mgr=702605724,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=3256049655,monmap=1360384041,nvmeofgw=3140514009,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=473443481}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=46,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=9} crc {logm=2308185185,mdsmap=1248375647,mgr=3370157582,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=3140514009,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=652149423,osd_snap=755866839}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=3638164364,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=635754567,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2169398879}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=3460011344,mdsmap=1248375647,mgr=3567252286,mgr_metadata=3792311847,mgrstat=775597981,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=10799308}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=42,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=11} crc {logm=1940448958,mdsmap=1248375647,mgr=702605724,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233959,osd_pg_creating=652149423,osd_snap=1614297645}) 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.063 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=926416034,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1395118888,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=2626835052}) 2024-09-17T16:30:34.064 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.055+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=41,mdsmap=2,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1037661082,mdsmap=3140514009,mgr=3567252286,mgr_command_descs=1770323759,mgrstat=1395118888,mon_config_key=346208105,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=652149423,osd_snap=473443481}) 2024-09-17T16:30:34.066 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.066 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.066 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=1269350848}) 2024-09-17T16:30:34.066 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=1168754689}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=3683814650}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=2656115473}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=8,osdmap=92} crc {osd_snap=4059779232,osdmap=1809589260}) 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.067 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=2606575713}) 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=8,osdmap=92} crc {osd_snap=1600964085,osdmap=4264236261}) 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=995777473}) 2024-09-17T16:30:34.068 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.059+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=9,osdmap=91} crc {osd_snap=945729246,osdmap=3267107296}) 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=1073180694}) 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=179202552}) 2024-09-17T16:30:34.084 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=73} crc {osdmap=3677723929}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=76} crc {osdmap=17364044}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=166069317}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=77} crc {osdmap=862121161}) 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.085 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.075+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.086 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.079+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=77} crc {osdmap=2657220537}) 2024-09-17T16:30:34.086 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.079+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:30:34.086 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.079+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=77} crc {osdmap=1019981977}) 2024-09-17T16:30:34.086 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:30:34.079+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=76} crc {osdmap=2430229952}) 2024-09-17T16:30:35.021 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:30:35.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:30:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: get_monmap_and_config 2024-09-17T16:30:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: monmap: 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:35.145777+0000 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:35.145777+0000 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:35.149 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 Processor -- start 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 -- start start 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: init 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient: _add_conns ranks=[1,0,6,2,9,4,8,7,5,10,3] 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f28f40a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f28f40a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 0x7f28f4142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-d con 0x7f28f40a73d0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f406ebc0 con 0x7f28f40a4910 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40a3690 con 0x7f28f40a68e0 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f28f40a3fc0 con 0x7f28f40a73d0 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fc144640 10 monclient(hunting): authenticate will time out at 1118.411658s 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 0x7f28f4142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28f96df640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28f40a4910 0x7f28f413eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43124/0 (socket says 172.21.15.105:43124) 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28f96df640 1 -- 172.21.15.105:0/2576420941 learned_addr learned my addr 172.21.15.105:0/2576420941 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): get_auth_request con 0x7f28f40a73d0 auth_method 0 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.140+0000 7f28fa6e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 10 monclient(hunting): handle_auth_done global_id 23451 payload 274 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 1 -- 172.21.15.105:0/2576420941 >> v1:172.21.15.160:6789/0 conn(0x7f28f40a68e0 legacy=0x7f28f40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 1 -- 172.21.15.105:0/2576420941 >> v1:172.21.15.105:6789/0 conn(0x7f28f40a4910 legacy=0x7f28f413eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 1 monclient: found mon.noname-d 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 1 -- 172.21.15.105:0/2576420941 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233266209 0 0) 0x7f28f406ebc0 con 0x7f28f40a4910 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 1 -- 172.21.15.105:0/2576420941 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40a3970 con 0x7f28f40a73d0 2024-09-17T16:30:35.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fa6e1640 1 --2- 172.21.15.105:0/2576420941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 0x7f28f4142220 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009dc0 tx=0x7f28ec01c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=875828303c0e843f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 1 -- 172.21.15.105:0/2576420941 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28ec0b2070 con 0x7f28f40a73d0 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: dump: 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 1 -- 172.21.15.105:0/2576420941 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec0a5a50 con 0x7f28f40a73d0 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 5 monclient: authenticate success, global_id 23451 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 1 -- 172.21.15.105:0/2576420941 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28ec0a5c40 con 0x7f28f40a73d0 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 4 set_mon_vals no callback set 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28f8ede640 10 monclient: dump: 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.162 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 10 monclient: shutdown 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 -- 172.21.15.105:0/2576420941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 msgr2=0x7f28f4142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 --2- 172.21.15.105:0/2576420941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 0x7f28f4142220 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009dc0 tx=0x7f28ec01c850 comp rx=0 tx=0).stop 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 -- 172.21.15.105:0/2576420941 shutdown_connections 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 --2- 172.21.15.105:0/2576420941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f28f40a73d0 0x7f28f4142220 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 -- 172.21.15.105:0/2576420941 >> 172.21.15.105:0/2576420941 conn(0x7f28f4068ba0 msgr2=0x7f28f4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 -- 172.21.15.105:0/2576420941 shutdown_connections 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 1 -- 172.21.15.105:0/2576420941 wait complete. 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.144+0000 7f28fc144640 10 monclient: monmap: 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 librados: starting msgr at 2024-09-17T16:30:35.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 librados: starting objecter 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 Processor -- start 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- start start 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 librados: setting wanted keys 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 librados: calling monclient init 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: init 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _add_conns ranks=[8,0,1,2,3,4,6,7,5] 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-e con 0x7f28f40a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f40a68e0 0x7f28f4126570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-a con 0x7f28f40a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 0x7f28f4126ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): picked mon.noname-f con 0x7f28f40a73d0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4145310 con 0x7f28f40a68e0 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4145190 con 0x7f28f40a73d0 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f28f4145490 con 0x7f28f40a4910 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient(hunting): authenticate will time out at 1118.416780s 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45242/0 (socket says 172.21.15.105:45242) 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 -- 172.21.15.105:0/2029583450 learned_addr learned my addr 172.21.15.105:0/2029583450 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): get_auth_request con 0x7f28f40a4910 auth_method 0 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f40a68e0 0x7f28f4126570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): get_auth_request con 0x7f28f40a68e0 auth_method 0 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fa6e1640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 0x7f28f4126ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient(hunting): handle_auth_done global_id 23408 payload 824 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 -- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 msgr2=0x7f28f4126ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 0x7f28f4126ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 -- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f40a68e0 msgr2=0x7f28f4126570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f40a68e0 0x7f28f4126570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 -- 172.21.15.105:0/2029583450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f4129e80 con 0x7f28f40a4910 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fa6e1640 10 monclient: get_auth_request con 0x7f28f40a73d0 auth_method 0 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fa6e1640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 0x7f28f4126ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f9ee0640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c450 tx=0x7f28e400c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 client.?.objecter ms_handle_connect 0x7f28f40a4910 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28e4012aa0 con 0x7f28f40a4910 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: dump: 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.170 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4017070 con 0x7f28f40a4910 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 5 monclient: authenticate success, global_id 23408 2024-09-17T16:30:35.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28e4012c80 con 0x7f28f40a4910 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ebfff640 4 set_mon_vals no callback set 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 monclient: dump: 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.172 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _renew_subs 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f406de60 con 0x7f28f40a4910 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 client.23408.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _renew_subs 2024-09-17T16:30:35.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f4129ba0 con 0x7f28f40a4910 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 librados: init done 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40a6cc0 con 0x7f28f40a4910 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f28e4012630 con 0x7f28f40a4910 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 0x7f28d8040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(106..106 src has 1..106) ==== 9008+0+0 (secure 0 0 0) 0x7f28e40506e0 con 0x7f28f40a4910 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 10 client.23408.objecter ms_dispatch 0x7f28f4000ba0 osd_map(106..106 src has 1..106) 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 3 client.23408.objecter handle_osd_map got epochs [106,106] > 0 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 3 client.23408.objecter handle_osd_map decoding full epoch 106 2024-09-17T16:30:35.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28ea7fc640 20 client.23408.objecter dump_active .. 0 homeless 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 0x7f28d8040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.148+0000 7f28f96df640 10 monclient: get_auth_request con 0x7f28d803e480 auth_method 0 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.152+0000 7f28ea7fc640 10 client.23408.objecter ms_handle_connect 0x7f28d803e480 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.152+0000 7f28f96df640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 0x7f28d8040940 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007fd0 tx=0x7f28f0007ed0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.152+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f28e404e070 con 0x7f28f40a4910 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.152+0000 7f28ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.152+0000 7f28ea7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28f4129770 con 0x7f28f40a4910 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28ea7fc640 1 -- 172.21.15.105:0/2029583450 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f28e401cc10 con 0x7f28f40a4910 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28ea7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:35.395 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:30:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 librados: watch_flush enter 2024-09-17T16:30:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 librados: watch_flush exit 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 20 client.23408.objecter shutdown clearing up homeless session... 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 client.23408.objecter successfully canceled tick 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 msgr2=0x7f28d8040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 0x7f28d8040940 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7f28f0007fd0 tx=0x7f28f0007ed0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 10 monclient: shutdown 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 msgr2=0x7f28f4126030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c450 tx=0x7f28e400c910 comp rx=0 tx=0).stop 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 shutdown_connections 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f28d803e480 0x7f28d8040940 unknown :-1 s=CLOSED pgs=3354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f40a73d0 0x7f28f4126ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28f40a68e0 0x7f28f4126570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 --2- 172.21.15.105:0/2029583450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f28f40a4910 0x7f28f4126030 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 >> 172.21.15.105:0/2029583450 conn(0x7f28f4068ba0 msgr2=0x7f28f409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 shutdown_connections 2024-09-17T16:30:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 -- 172.21.15.105:0/2029583450 wait complete. 2024-09-17T16:30:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.388+0000 7f28fc144640 1 librados: shutdown 2024-09-17T16:30:35.413 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:30:35.413 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:30:35.413 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:30:35.413 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 10 monclient: get_monmap_and_config 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 10 monclient: monmap: 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:35.550421+0000 2024-09-17T16:30:35.552 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:35.550421+0000 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:35.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 1 Processor -- start 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 1 -- start start 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.544+0000 7f17a3f6e640 10 monclient: init 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient: _add_conns ranks=[8,3,5,9,10,7,0,2,1,6,4] 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-e con 0x7f179c09b410 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179c09b7f0 0x7f179c0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-g con 0x7f179c09b7f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-h con 0x7f179c09bbd0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f179c09f8e0 con 0x7f179c09b7f0 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f179c0a1060 con 0x7f179c09bbd0 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f179c0a1340 con 0x7f179c09b410 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient(hunting): authenticate will time out at 1118.816291s 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45250/0 (socket says 172.21.15.105:45250) 2024-09-17T16:30:35.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 -- 172.21.15.105:0/3372683552 learned_addr learned my addr 172.21.15.105:0/3372683552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): get_auth_request con 0x7f179c09b410 auth_method 0 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1509640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179c09b7f0 0x7f179c0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a250b640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a250b640 10 monclient(hunting): get_auth_request con 0x7f179c09bbd0 auth_method 0 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a250b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a250b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a250b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1509640 10 monclient(hunting): get_auth_request con 0x7f179c09b7f0 auth_method 0 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1509640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1509640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1509640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_done global_id 23417 payload 274 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 -- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 msgr2=0x7f179c0a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c0a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 -- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179c09b7f0 msgr2=0x7f179c0a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179c09b7f0 0x7f179c0a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 -- 172.21.15.105:0/3372683552 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c0a5c50 con 0x7f179c09b410 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f17980098a0 tx=0x7f179801c470 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5044d7ef6e6752aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 1 -- 172.21.15.105:0/3372683552 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17980b2070 con 0x7f179c09b410 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: dump: 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.562 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 1 -- 172.21.15.105:0/3372683552 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17980ab7b0 con 0x7f179c09b410 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 5 monclient: authenticate success, global_id 23417 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 1 -- 172.21.15.105:0/3372683552 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17980abe50 con 0x7f179c09b410 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a0d08640 10 monclient: dump: 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 4 set_mon_vals no callback set 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 10 monclient: shutdown 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- 172.21.15.105:0/3372683552 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 msgr2=0x7f179c0a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f17980098a0 tx=0x7f179801c470 comp rx=0 tx=0).stop 2024-09-17T16:30:35.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- 172.21.15.105:0/3372683552 shutdown_connections 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c0a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179c09b7f0 0x7f179c0a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/3372683552 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c0a2300 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- 172.21.15.105:0/3372683552 >> 172.21.15.105:0/3372683552 conn(0x7f179c0a4e70 msgr2=0x7f179c0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- 172.21.15.105:0/3372683552 shutdown_connections 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.548+0000 7f17a3f6e640 1 -- 172.21.15.105:0/3372683552 wait complete. 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: monmap: 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.566 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 librados: starting msgr at 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 librados: starting objecter 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 Processor -- start 2024-09-17T16:30:35.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- start start 2024-09-17T16:30:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 librados: setting wanted keys 2024-09-17T16:30:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 librados: calling monclient init 2024-09-17T16:30:35.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: init 2024-09-17T16:30:35.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _add_conns ranks=[8,0,5,2,6,4,7,1,3] 2024-09-17T16:30:35.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c09a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-e con 0x7f179c09b410 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c09b7f0 0x7f179c09a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-a con 0x7f179c09b7f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c131400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): picked mon.noname-h con 0x7f179c09bbd0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f179c13ef40 con 0x7f179c09b7f0 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f179c0a4c10 con 0x7f179c09bbd0 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f179c13f0c0 con 0x7f179c09b410 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): authenticate will time out at 1118.821398s 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c09a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c09a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45262/0 (socket says 172.21.15.105:45262) 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 -- 172.21.15.105:0/1367451601 learned_addr learned my addr 172.21.15.105:0/1367451601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c09b7f0 0x7f179c09a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): get_auth_request con 0x7f179c09b410 auth_method 0 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): get_auth_request con 0x7f179c09b7f0 auth_method 0 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c131400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): get_auth_request con 0x7f179c09bbd0 auth_method 0 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient(hunting): handle_auth_done global_id 23426 payload 824 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 msgr2=0x7f179c131400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f179c09bbd0 0x7f179c131400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c09b7f0 msgr2=0x7f179c09a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c09b7f0 0x7f179c09a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c09dd60 con 0x7f179c09b410 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c09b7f0 0x7f179c09a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1d0a640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c09a300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f17980097c0 tx=0x7f1798019270 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 client.?.objecter ms_handle_connect 0x7f179c09b410 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17980b2070 con 0x7f179c09b410 2024-09-17T16:30:35.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: dump: 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.579 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17980abbf0 con 0x7f179c09b410 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 5 monclient: authenticate success, global_id 23426 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _renew_subs 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c09ea00 con 0x7f179c09b410 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17980abda0 con 0x7f179c09b410 2024-09-17T16:30:35.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f1793fff640 4 set_mon_vals no callback set 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 client.23426.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: dump: 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.581 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _renew_subs 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c13c100 con 0x7f179c09b410 2024-09-17T16:30:35.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 librados: init done 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 0x7f179c131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f179c051e80 con 0x7f179c131940 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 0x7f179c131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient: get_auth_request con 0x7f179c131940 auth_method 0 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f17980bf060 con 0x7f179c09b410 2024-09-17T16:30:35.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 10 monclient(hunting): handle_auth_done global_id 34263 payload 824 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f178003e550 0x7f1780040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f179c0a2300 con 0x7f179c131940 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(107..107 src has 1..107) ==== 8607+0+0 (secure 0 0 0) 0x7f17980ea5f0 con 0x7f179c09b410 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 client.23426.objecter ms_dispatch 0x7f179c000ba0 osd_map(107..107 src has 1..107) 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 3 client.23426.objecter handle_osd_map got epochs [107,107] > 0 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 3 client.23426.objecter handle_osd_map decoding full epoch 107 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 20 client.23426.objecter dump_active .. 0 homeless 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 client.23426.objecter ms_handle_connect 0x7f179c131940 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 client.23426.objecter resend_mon_ops 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 0x7f179c131d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c00e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f178003e550 0x7f1780040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17a250b640 10 monclient: get_auth_request con 0x7f178003e550 auth_method 0 2024-09-17T16:30:35.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f178c00b040 con 0x7f179c131940 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.552+0000 7f17927fc640 10 monclient: dump: 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.585 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f178c01b480 con 0x7f179c131940 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:35.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 msgr2=0x7f179c131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 0x7f179c131d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c00e6a0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 msgr2=0x7f179c131d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c131940 0x7f179c131d20 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c00e6a0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17a250b640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f178003e550 0x7f1780040a10 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f179c070b50 tx=0x7f1794009780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.556+0000 7f17927fc640 10 client.23426.objecter ms_handle_connect 0x7f178003e550 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 0x7f179c12d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f179c06a1c0 con 0x7f179c132260 2024-09-17T16:30:35.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 0x7f179c12d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient: get_auth_request con 0x7f179c132260 auth_method 0 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 10 monclient(hunting): handle_auth_done global_id 34272 payload 824 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 1 -- 172.21.15.105:0/1367451601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f179c09c560 con 0x7f179c132260 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a1509640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 0x7f179c12d430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 client.23426.objecter ms_handle_connect 0x7f179c132260 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 client.23426.objecter resend_mon_ops 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f178c012430 con 0x7f179c132260 2024-09-17T16:30:35.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 monclient: dump: 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.589 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f178c01bb70 con 0x7f179c132260 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 msgr2=0x7f179c12d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 0x7f179c12d430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 msgr2=0x7f179c12d430 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17927fc640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f179c132260 0x7f179c12d430 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f179c06d390 tx=0x7f178c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 26363, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:35.591 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.592 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.593 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:35.594 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.595 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.596 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:35.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:35.600 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.604 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.606 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:35.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.608 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.609 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.611 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.612 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:35.613 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 10 librados: watch_flush enter 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 10 librados: watch_flush exit 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 20 client.23426.objecter shutdown clearing up homeless session... 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.560+0000 7f17a3f6e640 10 client.23426.objecter successfully canceled tick 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f178003e550 msgr2=0x7f1780040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f178003e550 0x7f1780040a10 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7f179c070b50 tx=0x7f1794009780 comp rx=0 tx=0).stop 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 10 monclient: shutdown 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 msgr2=0x7f179c09a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 --2- 172.21.15.105:0/1367451601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f179c09b410 0x7f179c09a300 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f17980097c0 tx=0x7f1798019270 comp rx=0 tx=0).stop 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a250b640 1 -- 172.21.15.105:0/1367451601 reap_dead start 2024-09-17T16:30:35.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a250b640 1 -- 172.21.15.105:0/1367451601 reap_dead start 2024-09-17T16:30:35.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 shutdown_connections 2024-09-17T16:30:35.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 >> 172.21.15.105:0/1367451601 conn(0x7f179c0a4e70 msgr2=0x7f179c0a6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 shutdown_connections 2024-09-17T16:30:35.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 -- 172.21.15.105:0/1367451601 wait complete. 2024-09-17T16:30:35.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.564+0000 7f17a3f6e640 1 librados: shutdown 2024-09-17T16:30:35.616 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:35.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:30:35.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: get_monmap_and_config 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: monmap: 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:35.728355+0000 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:35.728355+0000 2024-09-17T16:30:35.730 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:35.731 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 Processor -- start 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 -- start start 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: init 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient: _add_conns ranks=[8,7,1,5,2,3,4,0,10,9,6] 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-e con 0x7fa4380a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 0x7fa4380a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-i con 0x7fa4380a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa4381396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43806ebc0 con 0x7fa4381396a0 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa4380a3fc0 con 0x7fa4380a4910 2024-09-17T16:30:35.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa4380a63c0 con 0x7fa4380a68e0 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43fff6640 10 monclient(hunting): authenticate will time out at 1118.994195s 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45286/0 (socket says 172.21.15.105:45286) 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 -- 172.21.15.105:0/1629762048 learned_addr learned my addr 172.21.15.105:0/1629762048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): get_auth_request con 0x7fa4380a68e0 auth_method 0 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient(hunting): handle_auth_done global_id 23435 payload 274 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43d591640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 0x7fa4380a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 -- 172.21.15.105:0/1629762048 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 msgr2=0x7fa4380a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 0x7fa4380a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 -- 172.21.15.105:0/1629762048 >> v1:172.21.15.160:6789/0 conn(0x7fa4381396a0 legacy=0x7fa438139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43dd92640 1 -- 172.21.15.105:0/1629762048 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4380a3690 con 0x7fa4380a68e0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43d591640 10 monclient: get_auth_request con 0x7fa4380a4910 auth_method 0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.724+0000 7fa43d591640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 0x7fa4380a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098a0 tx=0x7fa43401c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5aaebfb3a29de9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 1 -- 172.21.15.105:0/1629762048 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4340b0070 con 0x7fa4380a68e0 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: dump: 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.739 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 5 monclient: authenticate success, global_id 23435 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 1 -- 172.21.15.105:0/1629762048 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4340aa810 con 0x7fa4380a68e0 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:35.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 4 set_mon_vals no callback set 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 1 -- 172.21.15.105:0/1629762048 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4340a9300 con 0x7fa4380a68e0 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43cd90640 10 monclient: dump: 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.741 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: shutdown 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- 172.21.15.105:0/1629762048 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 msgr2=0x7fa4380a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098a0 tx=0x7fa43401c440 comp rx=0 tx=0).stop 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- 172.21.15.105:0/1629762048 shutdown_connections 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa4380a4910 0x7fa4380a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- 172.21.15.105:0/1629762048 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a68e0 0x7fa4380a6cc0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- 172.21.15.105:0/1629762048 >> 172.21.15.105:0/1629762048 conn(0x7fa438068ba0 msgr2=0x7fa438068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- 172.21.15.105:0/1629762048 shutdown_connections 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- 172.21.15.105:0/1629762048 wait complete. 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: monmap: 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:35.744 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.745 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 librados: starting msgr at 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 librados: starting objecter 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 Processor -- start 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- start start 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 librados: setting wanted keys 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 librados: calling monclient init 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: init 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient: _add_conns ranks=[8,0,5,3,4,7,6,2,1] 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 0x7fa4381d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-e con 0x7fa4380a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4380a68e0 0x7fa4381d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-a con 0x7fa4380a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4381396a0 0x7fa43813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): picked mon.noname-h con 0x7fa4381396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4381444b0 con 0x7fa4380a68e0 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa438144330 con 0x7fa4381396a0 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa438144630 con 0x7fa4380a4910 2024-09-17T16:30:35.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43fff6640 10 monclient(hunting): authenticate will time out at 1118.999550s 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 0x7fa4381d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 0x7fa4381d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45296/0 (socket says 172.21.15.105:45296) 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4380a68e0 0x7fa4381d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 -- 172.21.15.105:0/2391139810 learned_addr learned my addr 172.21.15.105:0/2391139810 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): get_auth_request con 0x7fa4380a4910 auth_method 0 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): get_auth_request con 0x7fa4380a68e0 auth_method 0 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43e593640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4381396a0 0x7fa43813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43e593640 10 monclient(hunting): get_auth_request con 0x7fa4381396a0 auth_method 0 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43e593640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43e593640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43e593640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient(hunting): handle_auth_done global_id 23444 payload 824 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4381396a0 msgr2=0x7fa43813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4381396a0 0x7fa43813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4380a68e0 msgr2=0x7fa4381d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4380a68e0 0x7fa4381d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.728+0000 7fa43dd92640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43dd92640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43dd92640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43dd92640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa438140bc0 con 0x7fa4380a4910 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43dd92640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 0x7fa4381d17d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097c0 tx=0x7fa4340191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.?.objecter ms_handle_connect 0x7fa4380a4910 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4340b9070 con 0x7fa4380a4910 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: dump: 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.756 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4340044b0 con 0x7fa4380a4910 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 5 monclient: authenticate success, global_id 23444 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 monclient: _renew_subs 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa438122660 con 0x7fa4380a4910 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4340b0040 con 0x7fa4380a4910 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 client.23444.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42ffff640 4 set_mon_vals no callback set 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: dump: 2024-09-17T16:30:35.758 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.759 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 monclient: _renew_subs 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43813eed0 con 0x7fa4380a4910 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 1 librados: init done 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 0x7fa43813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa438051e80 con 0x7fa43813dba0 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa434004040 con 0x7fa4380a4910 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 0x7fa43813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient: get_auth_request con 0x7fa43813dba0 auth_method 0 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa41c03e550 0x7fa41c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(107..107 src has 1..107) ==== 8607+0+0 (secure 0 0 0) 0x7fa4340e9b50 con 0x7fa4380a4910 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.23444.objecter ms_dispatch 0x7fa438000ba0 osd_map(107..107 src has 1..107) 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 3 client.23444.objecter handle_osd_map got epochs [107,107] > 0 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 3 client.23444.objecter handle_osd_map decoding full epoch 107 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 10 monclient(hunting): handle_auth_done global_id 23708 payload 824 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa438140990 con 0x7fa43813dba0 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 20 client.23444.objecter dump_active .. 0 homeless 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.23444.objecter ms_handle_connect 0x7fa43813dba0 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.23444.objecter resend_mon_ops 2024-09-17T16:30:35.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43d591640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 0x7fa43813df80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa428006fd0 tx=0x7fa42800fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa42800de60 con 0x7fa43813dba0 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43e593640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa41c03e550 0x7fa41c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: dump: 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43e593640 10 monclient: get_auth_request con 0x7fa41c03e550 auth_method 0 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa42800d9f0 con 0x7fa43813dba0 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 msgr2=0x7fa43813df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 0x7fa43813df80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa428006fd0 tx=0x7fa42800fdb0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 msgr2=0x7fa43813df80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813dba0 0x7fa43813df80 secure :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa428006fd0 tx=0x7fa42800fdb0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa43e593640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa41c03e550 0x7fa41c040a10 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 crypto rx=0x7fa438141460 tx=0x7fa430009410 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.732+0000 7fa42e7fc640 10 client.23444.objecter ms_handle_connect 0x7fa41c03e550 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 0x7fa4380a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa438139ae0 con 0x7fa43813e4c0 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 0x7fa4380a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient: get_auth_request con 0x7fa43813e4c0 auth_method 0 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 10 monclient(hunting): handle_auth_done global_id 23717 payload 824 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 1 -- 172.21.15.105:0/2391139810 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa438122b80 con 0x7fa43813e4c0 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43d591640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 0x7fa4380a0a20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa42801a000 tx=0x7fa4280046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 client.23444.objecter ms_handle_connect 0x7fa43813e4c0 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 client.23444.objecter resend_mon_ops 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4280064b0 con 0x7fa43813e4c0 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 monclient: dump: 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.765 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fa428005fa0 con 0x7fa43813e4c0 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 msgr2=0x7fa4380a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 0x7fa4380a0a20 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa42801a000 tx=0x7fa4280046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 msgr2=0x7fa4380a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa42e7fc640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa43813e4c0 0x7fa4380a0a20 secure :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa42801a000 tx=0x7fa4280046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 183383, 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:35.767 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.769 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:35.770 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.771 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:35.772 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.773 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.774 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.775 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:35.777 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.779 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.780 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.782 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.783 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.785 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:35.788 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:35.789 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 librados: watch_flush enter 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 librados: watch_flush exit 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 20 client.23444.objecter shutdown clearing up homeless session... 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 client.23444.objecter successfully canceled tick 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa41c03e550 msgr2=0x7fa41c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa41c03e550 0x7fa41c040a10 secure :-1 s=READY pgs=3356 cs=0 l=1 rev1=1 crypto rx=0x7fa438141460 tx=0x7fa430009410 comp rx=0 tx=0).stop 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 10 monclient: shutdown 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 msgr2=0x7fa4381d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 --2- 172.21.15.105:0/2391139810 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa4380a4910 0x7fa4381d17d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097c0 tx=0x7fa4340191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43e593640 1 -- 172.21.15.105:0/2391139810 reap_dead start 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43e593640 1 -- 172.21.15.105:0/2391139810 reap_dead start 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 shutdown_connections 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 >> 172.21.15.105:0/2391139810 conn(0x7fa438068ba0 msgr2=0x7fa4380a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 shutdown_connections 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 -- 172.21.15.105:0/2391139810 wait complete. 2024-09-17T16:30:35.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.740+0000 7fa43fff6640 1 librados: shutdown 2024-09-17T16:30:35.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:35.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:30:35.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:30:35.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: get_monmap_and_config 2024-09-17T16:30:35.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: monmap: 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:35.899931+0000 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:35.899931+0000 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:35.903 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 Processor -- start 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 -- start start 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: init 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient: _add_conns ranks=[8,9,0,4,1,7,5,3,6,2,10] 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-e con 0x7fa3a80a7060 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa3a80a5da0 0x7fa3a80a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-a con 0x7fa3a80a5da0 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa3a8139a00 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a80a7440 con 0x7fa3a8139a00 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a80a66e0 con 0x7fa3a80a5da0 2024-09-17T16:30:35.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa3a806d290 con 0x7fa3a80a7060 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3aedff640 10 monclient(hunting): authenticate will time out at 1119.165652s 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa3a80a5da0 0x7fa3a80a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45302/0 (socket says 172.21.15.105:45302) 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 1 -- 172.21.15.105:0/3287582557 learned_addr learned my addr 172.21.15.105:0/3287582557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): get_auth_request con 0x7fa3a80a7060 auth_method 0 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request con 0x7fa3a80a5da0 auth_method 0 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_done global_id 34290 payload 274 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 -- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 msgr2=0x7fa3a80a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 -- 172.21.15.105:0/3287582557 >> v1:172.21.15.105:6789/0 conn(0x7fa3a8139a00 legacy=0x7fa3a8139e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 monclient: found mon.noname-a 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3a7fff640 1 -- 172.21.15.105:0/3287582557 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a80a0e50 con 0x7fa3a80a5da0 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.896+0000 7fa3acb9b640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a80a5da0 0x7fa3a80a61a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0097c0 tx=0x7fa39c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e64df750798bad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 1 -- 172.21.15.105:0/3287582557 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa39c0b0070 con 0x7fa3a80a5da0 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: dump: 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.910 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 1 -- 172.21.15.105:0/3287582557 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c0aa810 con 0x7fa3a80a5da0 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 1 -- 172.21.15.105:0/3287582557 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa39c0a9300 con 0x7fa3a80a5da0 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 5 monclient: authenticate success, global_id 34290 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 4 set_mon_vals no callback set 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a77fe640 10 monclient: dump: 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.912 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: shutdown 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a80a5da0 msgr2=0x7fa3a80a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a80a5da0 0x7fa3a80a61a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0097c0 tx=0x7fa39c01c360 comp rx=0 tx=0).stop 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3287582557 shutdown_connections 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a80a5da0 0x7fa3a80a61a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3287582557 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a7060 0x7fa3a80a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3287582557 >> 172.21.15.105:0/3287582557 conn(0x7fa3a80aa450 msgr2=0x7fa3a80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3287582557 shutdown_connections 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3287582557 wait complete. 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: build_initial_monmap 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: monmap: 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:35.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:35.915 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 librados: starting msgr at 2024-09-17T16:30:35.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 librados: starting objecter 2024-09-17T16:30:35.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 Processor -- start 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- start start 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 librados: setting wanted keys 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 librados: calling monclient init 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: init 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _add_conns ranks=[8,2,5,6,1,7,0,4,3] 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-e con 0x7fa3a80a5da0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3a80a7060 0x7fa3a8138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-b con 0x7fa3a80a7060 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3a8139a00 0x7fa3a8139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): picked mon.noname-h con 0x7fa3a8139a00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:35.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a8144480 con 0x7fa3a80a7060 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa3a806db60 con 0x7fa3a8139a00 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa3a8144600 con 0x7fa3a80a5da0 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): authenticate will time out at 1119.170496s 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45308/0 (socket says 172.21.15.105:45308) 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 -- 172.21.15.105:0/3136876450 learned_addr learned my addr 172.21.15.105:0/3136876450 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): get_auth_request con 0x7fa3a80a5da0 auth_method 0 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3a80a7060 0x7fa3a8138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request con 0x7fa3a80a7060 auth_method 0 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient(hunting): handle_auth_done global_id 23462 payload 824 2024-09-17T16:30:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3a8139a00 msgr2=0x7fa3a8139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3a8139a00 0x7fa3a8139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3a80a7060 msgr2=0x7fa3a8138f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3a80a7060 0x7fa3a8138f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient: _finish_hunting 0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 monclient: found mon.noname-e 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a813d420 con 0x7fa3a80a5da0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3acb9b640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9d0 tx=0x7fa39800ee90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 client.?.objecter ms_handle_connect 0x7fa3a80a5da0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa398017070 con 0x7fa3a80a5da0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: dump: 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.927 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: _finish_auth 0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 5 monclient: authenticate success, global_id 23462 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa398013470 con 0x7fa3a80a5da0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _renew_subs 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a813f780 con 0x7fa3a80a5da0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa398013a90 con 0x7fa3a80a5da0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 client.23462.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a6ffd640 4 set_mon_vals no callback set 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3a57fa640 10 monclient: dump: 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _renew_subs 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a813df80 con 0x7fa3a80a5da0 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 librados: init done 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 0x7fa3a813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.900+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa3a8051e80 con 0x7fa3a813ff30 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 0x7fa3a813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient: get_auth_request con 0x7fa3a813ff30 auth_method 0 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa39801a070 con 0x7fa3a80a5da0 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_done global_id 22585 payload 824 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3a80a5360 con 0x7fa3a813ff30 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa39403e580 0x7fa394040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(107..107 src has 1..107) ==== 8607+0+0 (secure 0 0 0) 0x7fa39800f040 con 0x7fa3a80a5da0 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 client.23462.objecter ms_dispatch 0x7fa3a8000ba0 osd_map(107..107 src has 1..107) 2024-09-17T16:30:35.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 3 client.23462.objecter handle_osd_map got epochs [107,107] > 0 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 3 client.23462.objecter handle_osd_map decoding full epoch 107 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 20 client.23462.objecter dump_active .. 0 homeless 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 client.23462.objecter ms_handle_connect 0x7fa3a813ff30 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 client.23462.objecter resend_mon_ops 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 0x7fa3a813b930 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa3a806f220 tx=0x7fa39c019100 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa39c0b0040 con 0x7fa3a813ff30 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 monclient: dump: 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.932 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3ad39c640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa39403e580 0x7fa394040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3ad39c640 10 monclient: get_auth_request con 0x7fa39403e580 auth_method 0 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa39c003620 con 0x7fa3a813ff30 2024-09-17T16:30:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 msgr2=0x7fa3a813b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 0x7fa3a813b930 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa3a806f220 tx=0x7fa39c019100 comp rx=0 tx=0).stop 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 msgr2=0x7fa3a813b930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813ff30 0x7fa3a813b930 secure :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa3a806f220 tx=0x7fa39c019100 comp rx=0 tx=0).stop 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3a57fa640 10 client.23462.objecter ms_handle_connect 0x7fa39403e580 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.904+0000 7fa3ad39c640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa39403e580 0x7fa394040a40 secure :-1 s=READY pgs=3357 cs=0 l=1 rev1=1 crypto rx=0x7fa3a813dca0 tx=0x7fa3a0007d00 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3aedff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 0x7fa3a813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3aedff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa3a80a4fa0 con 0x7fa3a813be70 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 0x7fa3a813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient: get_auth_request con 0x7fa3a813be70 auth_method 0 2024-09-17T16:30:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 10 monclient(hunting): handle_auth_done global_id 22594 payload 824 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 1 -- 172.21.15.105:0/3136876450 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa3a813c790 con 0x7fa3a813be70 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a7fff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 0x7fa3a813c250 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa39c01c3a0 tx=0x7fa39c0a8000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 client.23462.objecter ms_handle_connect 0x7fa3a813be70 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 client.23462.objecter resend_mon_ops 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa39c0bd400 con 0x7fa3a813be70 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 monclient: dump: 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fa39c0b5b20 con 0x7fa3a813be70 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 msgr2=0x7fa3a813c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 0x7fa3a813c250 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa39c01c3a0 tx=0x7fa39c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 msgr2=0x7fa3a813c250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.908+0000 7fa3a57fa640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3a813be70 0x7fa3a813c250 secure :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa39c01c3a0 tx=0x7fa39c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:35.937 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 183547, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:35.938 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.939 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.940 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:35.941 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:35.942 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:35.943 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:35.945 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:35.947 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:35.948 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.949 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.950 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.951 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.952 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.953 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:35.954 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.955 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:35.956 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.957 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:35.958 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:35.959 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 10 librados: watch_flush enter 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 10 librados: watch_flush exit 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 20 client.23462.objecter shutdown clearing up homeless session... 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 10 client.23462.objecter successfully canceled tick 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa39403e580 msgr2=0x7fa394040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa39403e580 0x7fa394040a40 secure :-1 s=READY pgs=3357 cs=0 l=1 rev1=1 crypto rx=0x7fa3a813dca0 tx=0x7fa3a0007d00 comp rx=0 tx=0).stop 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 10 monclient: shutdown 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 msgr2=0x7fa3a81389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9d0 tx=0x7fa39800ee90 comp rx=0 tx=0).stop 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3ad39c640 1 -- 172.21.15.105:0/3136876450 reap_dead start 2024-09-17T16:30:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 shutdown_connections 2024-09-17T16:30:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 --2- 172.21.15.105:0/3136876450 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3a80a5da0 0x7fa3a81389f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 >> 172.21.15.105:0/3136876450 conn(0x7fa3a80aa450 msgr2=0x7fa3a80a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 shutdown_connections 2024-09-17T16:30:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 -- 172.21.15.105:0/3136876450 wait complete. 2024-09-17T16:30:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:35.912+0000 7fa3aedff640 1 librados: shutdown 2024-09-17T16:30:35.962 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:35.962 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:30:35.962 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: get_monmap_and_config 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: monmap: 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:36.072383+0000 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:36.072383+0000 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:36.075 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 Processor -- start 2024-09-17T16:30:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 -- start start 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: init 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient: _add_conns ranks=[6,2,8,0,3,1,9,4,7,10,5] 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5640a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-d con 0x7fa5640a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-b con 0x7fa5640a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 0x7fa564139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-e con 0x7fa5641396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa5640a3fc0 con 0x7fa5640a4910 2024-09-17T16:30:36.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa5640a63c0 con 0x7fa5640a68e0 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa5640a7740 con 0x7fa5641396a0 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa56a50e640 10 monclient(hunting): authenticate will time out at 1119.338084s 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 0x7fa564139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56574/0 (socket says 172.21.15.105:56574) 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 1 -- 172.21.15.105:0/2240534824 learned_addr learned my addr 172.21.15.105:0/2240534824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request con 0x7fa5640a4910 auth_method 0 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5640a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): get_auth_request con 0x7fa5641396a0 auth_method 0 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): get_auth_request con 0x7fa5640a68e0 auth_method 0 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient(hunting): handle_auth_done global_id 23460 payload 274 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 -- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 msgr2=0x7fa564139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 0x7fa564139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 -- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 msgr2=0x7fa5640a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 monclient: found mon.noname-d 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:36.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa563fff640 1 -- 172.21.15.105:0/2240534824 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56406ec10 con 0x7fa5640a68e0 2024-09-17T16:30:36.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:36.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.068+0000 7fa568aab640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 0x7fa564139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:36.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5640a6cc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa5540097f0 tx=0x7fa55401ce00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2895cfc1649ebf5d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 1 -- 172.21.15.105:0/2240534824 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5540b0070 con 0x7fa5640a68e0 2024-09-17T16:30:36.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:36.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: dump: 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.089 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 1 -- 172.21.15.105:0/2240534824 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5540aa810 con 0x7fa5640a68e0 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 1 -- 172.21.15.105:0/2240534824 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5540a4e60 con 0x7fa5640a68e0 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 5 monclient: authenticate success, global_id 23460 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 4 set_mon_vals no callback set 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:36.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa5637fe640 10 monclient: dump: 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.091 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: shutdown 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 msgr2=0x7fa5640a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5640a6cc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa5540097f0 tx=0x7fa55401ce00 comp rx=0 tx=0).stop 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- 172.21.15.105:0/2240534824 shutdown_connections 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5641396a0 0x7fa564139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5640a4910 0x7fa5640a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2240534824 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5640a6cc0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- 172.21.15.105:0/2240534824 >> 172.21.15.105:0/2240534824 conn(0x7fa564068ba0 msgr2=0x7fa564068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- 172.21.15.105:0/2240534824 shutdown_connections 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- 172.21.15.105:0/2240534824 wait complete. 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: monmap: 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.093 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 librados: starting msgr at 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 librados: starting objecter 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 Processor -- start 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- start start 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 librados: setting wanted keys 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 librados: calling monclient init 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: init 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient: _add_conns ranks=[8,6,1,0,4,2,5,7,3] 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 0x7fa5641d1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-e con 0x7fa5640a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5641d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-d con 0x7fa5640a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5641396a0 0x7fa56413d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): picked mon.noname-f con 0x7fa5641396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa564143320 con 0x7fa5641396a0 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa56406e630 con 0x7fa5640a68e0 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa5641434a0 con 0x7fa5640a4910 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa56a50e640 10 monclient(hunting): authenticate will time out at 1119.343766s 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 0x7fa5641d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5641d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 0x7fa5641d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45322/0 (socket says 172.21.15.105:45322) 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 1 -- 172.21.15.105:0/2670679884 learned_addr learned my addr 172.21.15.105:0/2670679884 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): get_auth_request con 0x7fa5640a4910 auth_method 0 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request con 0x7fa5640a68e0 auth_method 0 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.072+0000 7fa563fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 10 monclient(hunting): handle_auth_done global_id 23480 payload 824 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 msgr2=0x7fa5641d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5641d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5641396a0 msgr2=0x7fa56413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5641396a0 0x7fa56413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 monclient: found mon.noname-e 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564123cf0 con 0x7fa5640a4910 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5640a68e0 0x7fa5641d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa563fff640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 0x7fa5641d1680 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa55401c1d0 tx=0x7fa554019140 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.?.objecter ms_handle_connect 0x7fa5640a4910 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5540b0040 con 0x7fa5640a4910 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: dump: 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.098 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5540b9070 con 0x7fa5640a4910 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5540aae30 con 0x7fa5640a4910 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 5 monclient: authenticate success, global_id 23480 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 monclient: _renew_subs 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640abab0 con 0x7fa5640a4910 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa562ffd640 4 set_mon_vals no callback set 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: dump: 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.100 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 client.23480.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 monclient: _renew_subs 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa564123a10 con 0x7fa5640a4910 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 1 librados: init done 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 0x7fa56413e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa564051e80 con 0x7fa564124180 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 0x7fa56413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient: get_auth_request con 0x7fa564124180 auth_method 0 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa5540aa7e0 con 0x7fa5640a4910 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_done global_id 23478 payload 824 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5640a9ec0 con 0x7fa564124180 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa54403e580 0x7fa544040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(107..107 src has 1..107) ==== 8607+0+0 (secure 0 0 0) 0x7fa5540ea290 con 0x7fa5640a4910 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.23480.objecter ms_dispatch 0x7fa564000ba0 osd_map(107..107 src has 1..107) 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 3 client.23480.objecter handle_osd_map got epochs [107,107] > 0 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 3 client.23480.objecter handle_osd_map decoding full epoch 107 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 0x7fa56413e250 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa56413fa40 tx=0x7fa54c00f610 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 20 client.23480.objecter dump_active .. 0 homeless 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.23480.objecter ms_handle_connect 0x7fa564124180 2024-09-17T16:30:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.23480.objecter resend_mon_ops 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa54c011e70 con 0x7fa564124180 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: dump: 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.104 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa568aab640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa54403e580 0x7fa544040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa568aab640 10 monclient: get_auth_request con 0x7fa54403e580 auth_method 0 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa54c011e70 con 0x7fa564124180 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 msgr2=0x7fa56413e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 0x7fa56413e250 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa56413fa40 tx=0x7fa54c00f610 comp rx=0 tx=0).stop 2024-09-17T16:30:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 msgr2=0x7fa56413e250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa564124180 0x7fa56413e250 secure :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa56413fa40 tx=0x7fa54c00f610 comp rx=0 tx=0).stop 2024-09-17T16:30:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa5617fa640 10 client.23480.objecter ms_handle_connect 0x7fa54403e580 2024-09-17T16:30:36.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.076+0000 7fa568aab640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa54403e580 0x7fa544040a40 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fa564123910 tx=0x7fa55c00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 0x7fa56402d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa56413fbd0 con 0x7fa56402d450 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 0x7fa56402d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient: get_auth_request con 0x7fa56402d450 auth_method 0 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 10 monclient(hunting): handle_auth_done global_id 23487 payload 824 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 1 -- 172.21.15.105:0/2670679884 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa564139ae0 con 0x7fa56402d450 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 client.23480.objecter ms_handle_connect 0x7fa56402d450 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 client.23480.objecter resend_mon_ops 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa55b7fe640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 0x7fa56402d860 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa56413ecb0 tx=0x7fa54c0046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa54c014ba0 con 0x7fa56402d450 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 monclient: dump: 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fa54c022e50 con 0x7fa56402d450 2024-09-17T16:30:36.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 msgr2=0x7fa56402d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 0x7fa56402d860 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa56413ecb0 tx=0x7fa54c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 msgr2=0x7fa56402d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa5617fa640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa56402d450 0x7fa56402d860 secure :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa56413ecb0 tx=0x7fa54c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 183723, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:30:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:36.133 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:36.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 librados: watch_flush enter 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 librados: watch_flush exit 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 20 client.23480.objecter shutdown clearing up homeless session... 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 client.23480.objecter successfully canceled tick 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa54403e580 msgr2=0x7fa544040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa54403e580 0x7fa544040a40 secure :-1 s=READY pgs=3358 cs=0 l=1 rev1=1 crypto rx=0x7fa564123910 tx=0x7fa55c00b040 comp rx=0 tx=0).stop 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 10 monclient: shutdown 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 msgr2=0x7fa5641d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 --2- 172.21.15.105:0/2670679884 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa5640a4910 0x7fa5641d1680 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa55401c1d0 tx=0x7fa554019140 comp rx=0 tx=0).stop 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa568aab640 1 -- 172.21.15.105:0/2670679884 reap_dead start 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa568aab640 1 -- 172.21.15.105:0/2670679884 reap_dead start 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 shutdown_connections 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 >> 172.21.15.105:0/2670679884 conn(0x7fa564068ba0 msgr2=0x7fa56413fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 shutdown_connections 2024-09-17T16:30:36.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 -- 172.21.15.105:0/2670679884 wait complete. 2024-09-17T16:30:36.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.084+0000 7fa56a50e640 1 librados: shutdown 2024-09-17T16:30:36.137 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:36.137 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:30:36.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: get_monmap_and_config 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: monmap: 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:36.248388+0000 2024-09-17T16:30:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:36.248388+0000 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:36.252 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 Processor -- start 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 -- start start 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: init 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient: _add_conns ranks=[1,0,10,9,4,6,5,7,3,8,2] 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fccf40a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:36.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fccf40a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fccf40a73d0 0x7fccf4142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-f con 0x7fccf40a73d0 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf406ebc0 con 0x7fccf40a4910 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf40a3690 con 0x7fccf40a68e0 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fccf40a3fc0 con 0x7fccf40a73d0 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 10 monclient(hunting): authenticate will time out at 1119.514147s 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fccf40a4910 0x7fccf413eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43132/0 (socket says 172.21.15.105:43132) 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf37fe640 1 -- 172.21.15.105:0/2470250581 learned_addr learned my addr 172.21.15.105:0/2470250581 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58368506 0 0) 0x7fccf406ebc0 con 0x7fccf40a4910 2024-09-17T16:30:36.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): _negotiate 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): my global_id is 34308 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccdc003530 con 0x7fccf40a4910 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 1 --2- 172.21.15.105:0/2470250581 >> v2:172.21.15.160:3300/0 conn(0x7fccf40a73d0 0x7fccf4142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): get_auth_request con 0x7fccf40a73d0 auth_method 0 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1183948901 0 0) 0x7fccdc003530 con 0x7fccf40a4910 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 >> v2:172.21.15.160:3300/0 conn(0x7fccf40a73d0 msgr2=0x7fccf4142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 --2- 172.21.15.105:0/2470250581 >> v2:172.21.15.160:3300/0 conn(0x7fccf40a73d0 0x7fccf4142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 >> v1:172.21.15.160:6789/0 conn(0x7fccf40a68e0 legacy=0x7fccf40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf40a3970 con 0x7fccf40a4910 2024-09-17T16:30:36.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fcce8001cf0 con 0x7fccf40a4910 2024-09-17T16:30:36.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: dump: 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 >> v1:172.21.15.105:6789/0 conn(0x7fccf40a4910 legacy=0x7fccf413eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient: _add_conns ranks=[2,3,5,4,8,7,6,1,0] 2024-09-17T16:30:36.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 0x7fccdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): picked mon.b con 0x7fccdc004210 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fccdc004da0 0x7fccdc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): picked mon.g con 0x7fccdc004da0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccdc009c40 0x7fccdc00c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): picked mon.h con 0x7fccdc009c40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fccf40a3fc0 con 0x7fccdc004210 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc00c9b0 con 0x7fccdc004da0 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fccdc00cb60 con 0x7fccdc009c40 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf2ffd640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 0x7fccdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): get_auth_request con 0x7fccdc004210 auth_method 0 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fccdc004da0 0x7fccdc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccdc009c40 0x7fccdc00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient(hunting): handle_auth_done global_id 34308 payload 293 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 -- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccdc009c40 msgr2=0x7fccdc00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccdc009c40 0x7fccdc00c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 -- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fccdc004da0 msgr2=0x7fccdc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fccdc004da0 0x7fccdc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 monclient: found mon.b 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 1 -- 172.21.15.105:0/2470250581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc003530 con 0x7fccdc004210 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 5 monclient: authenticate success, global_id 34308 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccfa69e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf8c3b640 1 -- 172.21.15.105:0/2470250581 reap_dead start 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf37fe640 10 monclient: get_auth_request con 0x7fccdc004da0 auth_method 0 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.244+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fccdc004da0 0x7fccdc005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 0x7fccdc004810 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fccec007fd0 tx=0x7fccec01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf79496a69b4b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fccec0013e0 con 0x7fccdc004210 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: dump: 2024-09-17T16:30:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.273 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec0050d0 con 0x7fccdc004210 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2470250581 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fccec0052c0 con 0x7fccdc004210 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 4 set_mon_vals no callback set 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf2ffd640 10 monclient: dump: 2024-09-17T16:30:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.275 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: shutdown 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 msgr2=0x7fccdc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 0x7fccdc004810 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fccec007fd0 tx=0x7fccec01c350 comp rx=0 tx=0).stop 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- 172.21.15.105:0/2470250581 shutdown_connections 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2470250581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccdc004210 0x7fccdc004810 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- 172.21.15.105:0/2470250581 >> 172.21.15.105:0/2470250581 conn(0x7fccf4068ba0 msgr2=0x7fccf4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- 172.21.15.105:0/2470250581 shutdown_connections 2024-09-17T16:30:36.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- 172.21.15.105:0/2470250581 wait complete. 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: monmap: 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.277 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 librados: starting msgr at 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 librados: starting objecter 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 Processor -- start 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- start start 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 librados: setting wanted keys 2024-09-17T16:30:36.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 librados: calling monclient init 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: init 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient: _add_conns ranks=[4,5,1,6,3,2,8,7,0] 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 0x7fccf4138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-c con 0x7fccdc004210 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf41d1300 0x7fccf4138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-h con 0x7fccf41d1300 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf4124850 0x7fccf4122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): picked mon.noname-f con 0x7fccf4124850 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4139eb0 con 0x7fccf4124850 2024-09-17T16:30:36.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fccf406b5b0 con 0x7fccdc004210 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fccf413a030 con 0x7fccf41d1300 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccfa69e640 10 monclient(hunting): authenticate will time out at 1119.519750s 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 0x7fccf4138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 0x7fccf4138690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46438/0 (socket says 172.21.15.105:46438) 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 1 -- 172.21.15.105:0/2784072182 learned_addr learned my addr 172.21.15.105:0/2784072182 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): get_auth_request con 0x7fccdc004210 auth_method 0 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.248+0000 7fccf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf4124850 0x7fccf4122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf41d1300 0x7fccf4138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 10 monclient(hunting): get_auth_request con 0x7fccf4124850 auth_method 0 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 10 monclient(hunting): handle_auth_done global_id 22603 payload 824 2024-09-17T16:30:36.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf41d1300 msgr2=0x7fccf4138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf41d1300 0x7fccf4138bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf4124850 msgr2=0x7fccf4122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fccf4124850 0x7fccf4122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 monclient: found mon.noname-c 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf406dcb0 con 0x7fccdc004210 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf3fff640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 0x7fccf4138690 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fccec01aa50 tx=0x7fccdc005840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 client.?.objecter ms_handle_connect 0x7fccdc004210 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:36.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fccec003bb0 con 0x7fccdc004210 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: dump: 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.283 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec003d90 con 0x7fccdc004210 2024-09-17T16:30:36.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 5 monclient: authenticate success, global_id 22603 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 monclient: _renew_subs 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf4123cf0 con 0x7fccdc004210 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fccec0b48a0 con 0x7fccdc004210 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 monclient: dump: 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.286 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 client.22603.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 monclient: _renew_subs 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf41257b0 con 0x7fccdc004210 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 1 librados: init done 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf27fc640 4 set_mon_vals no callback set 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fccf4051e80 con 0x7fccf4122a70 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 10 monclient: get_auth_request con 0x7fccf4122a70 auth_method 0 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fccec0b4430 con 0x7fccdc004210 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 msgr2=0x7fccf4122e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send send error: (32) Broken pipe 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcce003e480 0x7fcce0040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(107..107 src has 1..107) ==== 8607+0+0 (secure 0 0 0) 0x7fccec0eaac0 con 0x7fccdc004210 2024-09-17T16:30:36.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 client.22603.objecter ms_dispatch 0x7fccf4000ba0 osd_map(107..107 src has 1..107) 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 3 client.22603.objecter handle_osd_map got epochs [107,107] > 0 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 3 client.22603.objecter handle_osd_map decoding full epoch 107 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 20 client.22603.objecter dump_active .. 0 homeless 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcce003e480 0x7fcce0040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 10 monclient: get_auth_request con 0x7fcce003e480 auth_method 0 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf8c3b640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcce003e480 0x7fcce0040940 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 crypto rx=0x7fccf406ed90 tx=0x7fcce4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.252+0000 7fccf0ff9640 10 client.22603.objecter ms_handle_connect 0x7fcce003e480 2024-09-17T16:30:36.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient: get_auth_request con 0x7fccf4122a70 auth_method 0 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 10 monclient(hunting): handle_auth_done global_id 23489 payload 824 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccf40a1f20 con 0x7fccf4122a70 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fccf40a1e40 tx=0x7fcce8007a60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 client.22603.objecter ms_handle_connect 0x7fccf4122a70 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 client.22603.objecter resend_mon_ops 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce8018020 con 0x7fccf4122a70 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 monclient: dump: 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.462 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.463 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcce800c070 con 0x7fccf4122a70 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 msgr2=0x7fccf4122e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fccf40a1e40 tx=0x7fcce8007a60 comp rx=0 tx=0).stop 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 msgr2=0x7fccf4122e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.456+0000 7fccf0ff9640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf4122a70 0x7fccf4122e50 secure :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fccf40a1e40 tx=0x7fcce8007a60 comp rx=0 tx=0).stop 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccfa69e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 0x7fccf409de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccfa69e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fccf4123390 con 0x7fccf409da00 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 0x7fccf409de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient: get_auth_request con 0x7fccf409da00 auth_method 0 2024-09-17T16:30:36.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 10 monclient(hunting): handle_auth_done global_id 23498 payload 824 2024-09-17T16:30:36.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 1 -- 172.21.15.105:0/2784072182 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fccf413a300 con 0x7fccf409da00 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf37fe640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 0x7fccf409de10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fccf40abda0 tx=0x7fccf40a3860 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf0ff9640 10 client.22603.objecter ms_handle_connect 0x7fccf409da00 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.460+0000 7fccf0ff9640 10 client.22603.objecter resend_mon_ops 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce8018040 con 0x7fccf409da00 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 10 monclient: dump: 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.474 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fcce80147d0 con 0x7fccf409da00 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 msgr2=0x7fccf409de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 0x7fccf409de10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fccf40abda0 tx=0x7fccf40a3860 comp rx=0 tx=0).stop 2024-09-17T16:30:36.475 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 184107, 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:36.476 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:36.477 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.478 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.479 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:36.480 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.481 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:36.482 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.483 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:36.484 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:36.486 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.487 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:36.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:36.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.495 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:36.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:36.499 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:36.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 msgr2=0x7fccf409de10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf0ff9640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf409da00 0x7fccf409de10 secure :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fccf40abda0 tx=0x7fccf40a3860 comp rx=0 tx=0).stop 2024-09-17T16:30:36.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 10 librados: watch_flush enter 2024-09-17T16:30:36.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 10 librados: watch_flush exit 2024-09-17T16:30:36.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 20 client.22603.objecter shutdown clearing up homeless session... 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 10 client.22603.objecter successfully canceled tick 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcce003e480 msgr2=0x7fcce0040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcce003e480 0x7fcce0040940 secure :-1 s=READY pgs=3359 cs=0 l=1 rev1=1 crypto rx=0x7fccf406ed90 tx=0x7fcce4008040 comp rx=0 tx=0).stop 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 10 monclient: shutdown 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 msgr2=0x7fccf4138690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 --2- 172.21.15.105:0/2784072182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fccdc004210 0x7fccf4138690 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fccec01aa50 tx=0x7fccdc005840 comp rx=0 tx=0).stop 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf8c3b640 1 -- 172.21.15.105:0/2784072182 reap_dead start 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccf8c3b640 1 -- 172.21.15.105:0/2784072182 reap_dead start 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 shutdown_connections 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 >> 172.21.15.105:0/2784072182 conn(0x7fccf4068ba0 msgr2=0x7fccf41c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 shutdown_connections 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 -- 172.21.15.105:0/2784072182 wait complete. 2024-09-17T16:30:36.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.464+0000 7fccfa69e640 1 librados: shutdown 2024-09-17T16:30:36.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:36.503 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:30:36.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: get_monmap_and_config 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: monmap: 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:36.636178+0000 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:36.636178+0000 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:36.639 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 Processor -- start 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 -- start start 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: init 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient: _add_conns ranks=[6,5,9,4,3,2,0,10,1,8,7] 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a200a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-d con 0x7f9a200a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 0x7f9a200a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-h con 0x7f9a200a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a201396a0 0x7f9a20139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-a con 0x7f9a201396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200a3fc0 con 0x7f9a201396a0 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9a200a63c0 con 0x7f9a200a4910 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9a200a7740 con 0x7f9a200a68e0 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a25a58640 10 monclient(hunting): authenticate will time out at 1119.901904s 2024-09-17T16:30:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a201396a0 0x7f9a20139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a201396a0 0x7f9a20139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52010/0 (socket says 172.21.15.105:52010) 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/872931388 learned_addr learned my addr 172.21.15.105:0/872931388 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a200a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): get_auth_request con 0x7f9a201396a0 auth_method 0 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): get_auth_request con 0x7f9a200a68e0 auth_method 0 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1e7fc640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 0x7f9a200a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient(hunting): handle_auth_done global_id 34344 payload 274 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 msgr2=0x7f9a200a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a200a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/872931388 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 msgr2=0x7f9a200a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 0x7f9a200a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 monclient: found mon.noname-a 2024-09-17T16:30:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/872931388 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2006ebc0 con 0x7f9a201396a0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1e7fc640 10 monclient: get_auth_request con 0x7f9a200a4910 auth_method 0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.632+0000 7f9a1e7fc640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 0x7f9a200a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a201396a0 0x7f9a20139ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009870 tx=0x7f9a1001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7cbb0c05c6fc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 1 -- 172.21.15.105:0/872931388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a100b0070 con 0x7f9a201396a0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: dump: 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 5 monclient: authenticate success, global_id 34344 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 1 -- 172.21.15.105:0/872931388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a100aa810 con 0x7f9a201396a0 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 4 set_mon_vals no callback set 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 1 -- 172.21.15.105:0/872931388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a100a9300 con 0x7f9a201396a0 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1dffb640 10 monclient: dump: 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.650 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: shutdown 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a201396a0 msgr2=0x7f9a20139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a201396a0 0x7f9a20139ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009870 tx=0x7f9a1001c3f0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- 172.21.15.105:0/872931388 shutdown_connections 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a201396a0 0x7f9a20139ae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9a200a4910 0x7f9a200a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- 172.21.15.105:0/872931388 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a200a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- 172.21.15.105:0/872931388 >> 172.21.15.105:0/872931388 conn(0x7f9a20068ba0 msgr2=0x7f9a20068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- 172.21.15.105:0/872931388 shutdown_connections 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- 172.21.15.105:0/872931388 wait complete. 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: monmap: 2024-09-17T16:30:36.652 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.653 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 librados: starting msgr at 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 librados: starting objecter 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 Processor -- start 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- start start 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 librados: setting wanted keys 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 librados: calling monclient init 2024-09-17T16:30:36.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: init 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient: _add_conns ranks=[3,6,7,1,5,4,2,0,8] 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200a4910 0x7f9a201d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-g con 0x7f9a200a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a201d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-d con 0x7f9a200a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9a201396a0 0x7f9a2013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): picked mon.noname-i con 0x7f9a201396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2006e630 con 0x7f9a200a4910 2024-09-17T16:30:36.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9a200a7af0 con 0x7f9a200a68e0 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9a20143320 con 0x7f9a201396a0 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a25a58640 10 monclient(hunting): authenticate will time out at 1119.907155s 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a201d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a201d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37592/0 (socket says 172.21.15.105:37592) 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 -- 172.21.15.105:0/799377213 learned_addr learned my addr 172.21.15.105:0/799377213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): get_auth_request con 0x7f9a200a68e0 auth_method 0 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9a201396a0 0x7f9a2013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 10 monclient(hunting): get_auth_request con 0x7f9a201396a0 auth_method 0 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient(hunting): handle_auth_done global_id 23514 payload 824 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9a201396a0 msgr2=0x7f9a2013d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1effd640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200a4910 0x7f9a201d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9a201396a0 0x7f9a2013d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200a4910 msgr2=0x7f9a201d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200a4910 0x7f9a201d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 monclient: found mon.noname-d 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201239a0 con 0x7f9a200a68e0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a1e7fc640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a201d1a10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c7f0 tx=0x7f9a1400ccb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 10 client.?.objecter ms_handle_connect 0x7f9a200a68e0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a140044a0 con 0x7f9a200a68e0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.636+0000 7f9a07fff640 10 monclient: dump: 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14007830 con 0x7f9a200a68e0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a14003620 con 0x7f9a200a68e0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 5 monclient: authenticate success, global_id 23514 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 monclient: _renew_subs 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a20123c80 con 0x7f9a200a68e0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1d7fa640 4 set_mon_vals no callback set 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 client.23514.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: dump: 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.667 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 monclient: _renew_subs 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2013fb20 con 0x7f9a200a68e0 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 1 librados: init done 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 0x7f9a2013dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20051e80 con 0x7f9a2013d930 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9a14004030 con 0x7f9a200a68e0 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 0x7f9a2013dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9a0c03e550 0x7f9a0c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient: get_auth_request con 0x7f9a2013d930 auth_method 0 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(108..108 src has 1..108) ==== 7481+0+0 (secure 0 0 0) 0x7f9a14050590 con 0x7f9a200a68e0 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 client.23514.objecter ms_dispatch 0x7f9a20000ba0 osd_map(108..108 src has 1..108) 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 3 client.23514.objecter handle_osd_map got epochs [108,108] > 0 2024-09-17T16:30:36.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 3 client.23514.objecter handle_osd_map decoding full epoch 108 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 20 client.23514.objecter dump_active .. 0 homeless 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9a0c03e550 0x7f9a0c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1f7fe640 10 monclient: get_auth_request con 0x7f9a0c03e550 auth_method 0 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 10 monclient(hunting): handle_auth_done global_id 24571 payload 824 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a20139ae0 con 0x7f9a2013d930 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 client.23514.objecter ms_handle_connect 0x7f9a2013d930 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1effd640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 0x7f9a2013dd10 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800f3b0 tx=0x7f9a1800f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 client.23514.objecter resend_mon_ops 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a18016890 con 0x7f9a2013d930 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: dump: 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.671 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 client.23514.objecter ms_handle_connect 0x7f9a0c03e550 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a1f7fe640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9a0c03e550 0x7f9a0c040a10 secure :-1 s=READY pgs=3362 cs=0 l=1 rev1=1 crypto rx=0x7f9a100ad000 tx=0x7f9a100a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9a1801b070 con 0x7f9a2013d930 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 msgr2=0x7f9a2013dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 0x7f9a2013dd10 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800f3b0 tx=0x7f9a1800f870 comp rx=0 tx=0).stop 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 msgr2=0x7f9a2013dd10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.640+0000 7f9a07fff640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013d930 0x7f9a2013dd10 secure :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800f3b0 tx=0x7f9a1800f870 comp rx=0 tx=0).stop 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a25a58640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a25a58640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 0x7f9a200a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a25a58640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2013f310 con 0x7f9a2013e250 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 0x7f9a200a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient: get_auth_request con 0x7f9a2013e250 auth_method 0 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 10 monclient(hunting): handle_auth_done global_id 24580 payload 824 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 1 -- 172.21.15.105:0/799377213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9a200008e0 con 0x7f9a2013e250 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a1effd640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 0x7f9a200a0a20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f9a18000f90 tx=0x7f9a18004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 10 client.23514.objecter ms_handle_connect 0x7f9a2013e250 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 10 client.23514.objecter resend_mon_ops 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a18016ba0 con 0x7f9a2013e250 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.648+0000 7f9a07fff640 10 monclient: dump: 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f9a18016730 con 0x7f9a2013e250 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 msgr2=0x7f9a200a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 0x7f9a200a0a20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f9a18000f90 tx=0x7f9a18004a50 comp rx=0 tx=0).stop 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 msgr2=0x7f9a200a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a07fff640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2013e250 0x7f9a200a0a20 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:36.676 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 184252, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.677 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.678 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.679 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:36.680 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.681 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.683 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.685 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:36.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:36.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:36.691 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:36.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:36.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.698 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:36.699 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 10 librados: watch_flush enter 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 10 librados: watch_flush exit 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 20 client.23514.objecter shutdown clearing up homeless session... 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 10 client.23514.objecter successfully canceled tick 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9a0c03e550 msgr2=0x7f9a0c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9a0c03e550 0x7f9a0c040a10 secure :-1 s=READY pgs=3362 cs=0 l=1 rev1=1 crypto rx=0x7f9a100ad000 tx=0x7f9a100a6000 comp rx=0 tx=0).stop 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 10 monclient: shutdown 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 msgr2=0x7f9a201d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 --2- 172.21.15.105:0/799377213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9a200a68e0 0x7f9a201d1a10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400c7f0 tx=0x7f9a1400ccb0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/799377213 reap_dead start 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a1f7fe640 1 -- 172.21.15.105:0/799377213 reap_dead start 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 shutdown_connections 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 >> 172.21.15.105:0/799377213 conn(0x7f9a20068ba0 msgr2=0x7f9a200a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 shutdown_connections 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 -- 172.21.15.105:0/799377213 wait complete. 2024-09-17T16:30:36.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.652+0000 7f9a25a58640 1 librados: shutdown 2024-09-17T16:30:36.702 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:36.702 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:30:36.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.816+0000 7f5df59ce640 10 monclient: get_monmap_and_config 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.816+0000 7f5df59ce640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.816+0000 7f5df59ce640 10 monclient: monmap: 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:36.822972+0000 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:36.822972+0000 2024-09-17T16:30:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:36.826 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:36.827 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 Processor -- start 2024-09-17T16:30:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 -- start start 2024-09-17T16:30:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient: init 2024-09-17T16:30:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient: _add_conns ranks=[4,5,2,9,8,3,7,10,1,6,0] 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-c con 0x7f5df00a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5df00a4ab0 0x7f5df00a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-h con 0x7f5df00a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 0x7f5df0139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-b con 0x7f5df01397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5df00a7c40 con 0x7f5df01397f0 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f5df00a7890 con 0x7f5df00a6a30 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5df00a4080 con 0x7f5df00a4ab0 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient(hunting): authenticate will time out at 1120.088835s 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 0x7f5df0139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46452/0 (socket says 172.21.15.105:46452) 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 -- 172.21.15.105:0/855991656 learned_addr learned my addr 172.21.15.105:0/855991656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): get_auth_request con 0x7f5df00a6a30 auth_method 0 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): get_auth_request con 0x7f5df01397f0 auth_method 0 2024-09-17T16:30:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5de7fff640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5df00a4ab0 0x7f5df00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5de7fff640 10 monclient(hunting): get_auth_request con 0x7f5df00a4ab0 auth_method 0 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5de7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5de7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5de7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient(hunting): handle_auth_done global_id 22630 payload 274 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 -- 172.21.15.105:0/855991656 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5df00a4ab0 msgr2=0x7f5df00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5df00a4ab0 0x7f5df00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 -- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 msgr2=0x7f5df0139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 0x7f5df0139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 monclient: found mon.noname-c 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 -- 172.21.15.105:0/855991656 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df00a37e0 con 0x7f5df00a6a30 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5def7fe640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 0x7f5df0139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5deeffd640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de001c410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a4d3822760c04a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 1 -- 172.21.15.105:0/855991656 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de00b1070 con 0x7f5df00a6a30 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: dump: 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 1 -- 172.21.15.105:0/855991656 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de00aa7b0 con 0x7f5df00a6a30 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 5 monclient: authenticate success, global_id 22630 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 1 -- 172.21.15.105:0/855991656 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de00aae50 con 0x7f5df00a6a30 2024-09-17T16:30:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5df59ce640 4 set_mon_vals no callback set 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.820+0000 7f5dee7fc640 10 monclient: dump: 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: shutdown 2024-09-17T16:30:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 msgr2=0x7f5df00a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de001c410 comp rx=0 tx=0).stop 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/855991656 shutdown_connections 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5df01397f0 0x7f5df0139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5df00a4ab0 0x7f5df00a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- 172.21.15.105:0/855991656 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5df00a6a30 0x7f5df00a6e10 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/855991656 >> 172.21.15.105:0/855991656 conn(0x7f5df00a8d00 msgr2=0x7f5df00a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/855991656 shutdown_connections 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/855991656 wait complete. 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: build_initial_monmap 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: monmap: 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 librados: starting msgr at 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 librados: starting objecter 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 Processor -- start 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- start start 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 librados: setting wanted keys 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 librados: calling monclient init 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: init 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _add_conns ranks=[0,1,8,4,3,6,2,7,5] 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 0x7f5df01381e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-a con 0x7f5df00a4ab0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5df00a6a30 0x7f5df0138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-f con 0x7f5df00a6a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5df01397f0 0x7f5df013d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): picked mon.noname-e con 0x7f5df01397f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5df006e770 con 0x7f5df00a4ab0 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5df013a5a0 con 0x7f5df00a6a30 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5df0143260 con 0x7f5df01397f0 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): authenticate will time out at 1120.094267s 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 0x7f5df01381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5df01397f0 0x7f5df013d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 0x7f5df01381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52026/0 (socket says 172.21.15.105:52026) 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 1 -- 172.21.15.105:0/440815814 learned_addr learned my addr 172.21.15.105:0/440815814 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): get_auth_request con 0x7f5df00a4ab0 auth_method 0 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): get_auth_request con 0x7f5df01397f0 auth_method 0 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5de7fff640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5df00a6a30 0x7f5df0138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient(hunting): handle_auth_done global_id 23507 payload 824 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5df00a6a30 msgr2=0x7f5df0138720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5df00a6a30 0x7f5df0138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 msgr2=0x7f5df01381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 0x7f5df01381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 monclient: found mon.noname-e 2024-09-17T16:30:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df013fd10 con 0x7f5df01397f0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5deeffd640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5df00a4ab0 0x7f5df01381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5def7fe640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5df01397f0 0x7f5df013d8a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5de800efc0 tx=0x7f5de800c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 client.?.objecter ms_handle_connect 0x7f5df01397f0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de8018020 con 0x7f5df01397f0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: dump: 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: _finish_auth 0 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: _check_auth_tickets 2024-09-17T16:30:36.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 5 monclient: authenticate success, global_id 23507 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de800cd30 con 0x7f5df01397f0 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _renew_subs 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df00aa330 con 0x7f5df01397f0 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de8014d60 con 0x7f5df01397f0 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5dedffb640 4 set_mon_vals no callback set 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 client.23507.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5decff9640 10 monclient: dump: 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.855 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _renew_subs 2024-09-17T16:30:36.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df0122810 con 0x7f5df01397f0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 librados: init done 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 0x7f5df013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.824+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f5df0051e80 con 0x7f5df013dde0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5de801b020 con 0x7f5df01397f0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 0x7f5df013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient: get_auth_request con 0x7f5df013dde0 auth_method 0 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5dd003e550 0x7f5dd0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(108..108 src has 1..108) ==== 7481+0+0 (secure 0 0 0) 0x7f5de8050ef0 con 0x7f5df01397f0 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 client.23507.objecter ms_dispatch 0x7f5df0000ba0 osd_map(108..108 src has 1..108) 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 3 client.23507.objecter handle_osd_map got epochs [108,108] > 0 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 3 client.23507.objecter handle_osd_map decoding full epoch 108 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 20 client.23507.objecter dump_active .. 0 homeless 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5de7fff640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5dd003e550 0x7f5dd0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5de7fff640 10 monclient: get_auth_request con 0x7f5dd003e550 auth_method 0 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 10 monclient(hunting): handle_auth_done global_id 19092 payload 824 2024-09-17T16:30:36.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5df00a9f00 con 0x7f5df013dde0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 client.23507.objecter ms_handle_connect 0x7f5df013dde0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 client.23507.objecter resend_mon_ops 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5deeffd640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 0x7f5df013e1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de00191b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 client.23507.objecter ms_handle_connect 0x7f5dd003e550 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5de7fff640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5dd003e550 0x7f5dd0040a10 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 crypto rx=0x7f5df013ef80 tx=0x7f5dd8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de00bb020 con 0x7f5df013dde0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 monclient: dump: 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.859 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.860 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5de00aa6f0 con 0x7f5df013dde0 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 msgr2=0x7f5df013e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 0x7f5df013e1c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098d0 tx=0x7f5de00191b0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 msgr2=0x7f5df013e1c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.828+0000 7f5decff9640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013dde0 0x7f5df013e1c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:36.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:36.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 0x7f5df00a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:36.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:36.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f5df006a200 con 0x7f5df013e700 2024-09-17T16:30:36.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 0x7f5df00a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient: get_auth_request con 0x7f5df013e700 auth_method 0 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 10 monclient(hunting): handle_auth_done global_id 19101 payload 824 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 1 -- 172.21.15.105:0/440815814 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5df0139c50 con 0x7f5df013e700 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5deeffd640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 0x7f5df00a0a20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b9000 tx=0x7f5de00a6000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:36.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 client.23507.objecter ms_handle_connect 0x7f5df013e700 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 client.23507.objecter resend_mon_ops 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5de00bb040 con 0x7f5df013e700 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 monclient: dump: 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:36.865 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f5de00c2020 con 0x7f5df013e700 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 msgr2=0x7f5df00a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 0x7f5df00a0a20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b9000 tx=0x7f5de00a6000 comp rx=0 tx=0).stop 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 msgr2=0x7f5df00a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5decff9640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5df013e700 0x7f5df00a0a20 secure :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5de00b9000 tx=0x7f5de00a6000 comp rx=0 tx=0).stop 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 184464, 2024-09-17T16:30:36.867 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:36.868 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.869 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.870 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:36.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:36.872 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:36.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.878 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:36.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:36.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:36.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:36.886 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.887 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:36.889 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.890 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:36.891 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 librados: watch_flush enter 2024-09-17T16:30:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 librados: watch_flush exit 2024-09-17T16:30:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 20 client.23507.objecter shutdown clearing up homeless session... 2024-09-17T16:30:36.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 client.23507.objecter successfully canceled tick 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5dd003e550 msgr2=0x7f5dd0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5dd003e550 0x7f5dd0040a10 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 crypto rx=0x7f5df013ef80 tx=0x7f5dd8008040 comp rx=0 tx=0).stop 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 10 monclient: shutdown 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5df01397f0 msgr2=0x7f5df013d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 --2- 172.21.15.105:0/440815814 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5df01397f0 0x7f5df013d8a0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5de800efc0 tx=0x7f5de800c490 comp rx=0 tx=0).stop 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5def7fe640 1 -- 172.21.15.105:0/440815814 reap_dead start 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5def7fe640 1 -- 172.21.15.105:0/440815814 reap_dead start 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 shutdown_connections 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 >> 172.21.15.105:0/440815814 conn(0x7f5df00a8d00 msgr2=0x7f5df00a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 shutdown_connections 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 -- 172.21.15.105:0/440815814 wait complete. 2024-09-17T16:30:36.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:36.836+0000 7f5df59ce640 1 librados: shutdown 2024-09-17T16:30:36.895 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:36.895 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:30:36.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:30:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: get_monmap_and_config 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: monmap: 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:37.007939+0000 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:37.007939+0000 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:37.011 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 Processor -- start 2024-09-17T16:30:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 -- start start 2024-09-17T16:30:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: init 2024-09-17T16:30:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient: _add_conns ranks=[9,4,2,8,0,7,1,5,6,10,3] 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-a con 0x7fe5cc0a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-c con 0x7fe5cc0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 0x7fe5cc139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-b con 0x7fe5cc1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe5cc0a3fc0 con 0x7fe5cc0a68e0 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe5cc0a63c0 con 0x7fe5cc1396a0 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe5cc0a7740 con 0x7fe5cc0a4910 2024-09-17T16:30:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5d0e06640 10 monclient(hunting): authenticate will time out at 1120.273670s 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 0x7fe5cc139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46466/0 (socket says 172.21.15.105:46466) 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52040/0 (socket says 172.21.15.105:52040) 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): get_auth_request con 0x7fe5cc1396a0 auth_method 0 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 1 -- 172.21.15.105:0/130843998 learned_addr learned my addr 172.21.15.105:0/130843998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a4910 auth_method 0 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a68e0 auth_method 0 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_done global_id 23753 payload 274 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 -- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 msgr2=0x7fe5cc0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 -- 172.21.15.105:0/130843998 >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 msgr2=0x7fe5cc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/130843998 >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 monclient: found mon.noname-b 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5cad9d640 1 -- 172.21.15.105:0/130843998 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc06ebc0 con 0x7fe5cc1396a0 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5c9d9b640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.004+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/130843998 >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 0x7fe5cc139b50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c401c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b755390d9d9409 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 1 -- 172.21.15.105:0/130843998 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40b2070 con 0x7fe5cc1396a0 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: dump: 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 1 -- 172.21.15.105:0/130843998 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c40a5ad0 con 0x7fe5cc1396a0 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 5 monclient: authenticate success, global_id 23753 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 1 -- 172.21.15.105:0/130843998 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40a5d00 con 0x7fe5cc1396a0 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c959a640 10 monclient: dump: 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 4 set_mon_vals no callback set 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: shutdown 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 msgr2=0x7fe5cc139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 0x7fe5cc139b50 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c401c800 comp rx=0 tx=0).stop 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- 172.21.15.105:0/130843998 shutdown_connections 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe5cc1396a0 0x7fe5cc139b50 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/130843998 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/130843998 >> v2:172.21.15.105:3300/0 conn(0x7fe5cc0a68e0 0x7fe5cc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- 172.21.15.105:0/130843998 >> 172.21.15.105:0/130843998 conn(0x7fe5cc068ba0 msgr2=0x7fe5cc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- 172.21.15.105:0/130843998 shutdown_connections 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- 172.21.15.105:0/130843998 wait complete. 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: monmap: 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 librados: starting msgr at 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 librados: starting objecter 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 Processor -- start 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- start start 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 librados: setting wanted keys 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 librados: calling monclient init 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: init 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient: _add_conns ranks=[4,5,0,1,2,3,6,7,8] 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc1d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-c con 0x7fe5cc0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc0a68e0 0x7fe5cc1d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-h con 0x7fe5cc0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:37.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 0x7fe5cc13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): picked mon.noname-a con 0x7fe5cc1396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc06e630 con 0x7fe5cc1396a0 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe5cc0a7af0 con 0x7fe5cc0a4910 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5cc143320 con 0x7fe5cc0a68e0 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5d0e06640 10 monclient(hunting): authenticate will time out at 1120.279297s 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc1d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 0x7fe5cc13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 0x7fe5cc13d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52044/0 (socket says 172.21.15.105:52044) 2024-09-17T16:30:37.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc1d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:46476/0 (socket says 172.21.15.105:46476) 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 learned_addr learned my addr 172.21.15.105:0/1476613564 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): get_auth_request con 0x7fe5cc1396a0 auth_method 0 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a4910 auth_method 0 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient(hunting): handle_auth_done global_id 34371 payload 824 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc0a68e0 msgr2=0x7fe5cc1d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c9d9b640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc0a68e0 0x7fe5cc1d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc0a68e0 0x7fe5cc1d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 msgr2=0x7fe5cc1d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc1d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 monclient: found mon.noname-a 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1236c0 con 0x7fe5cc1396a0 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c9d9b640 10 monclient: get_auth_request con 0x7fe5cc0a68e0 auth_method 0 2024-09-17T16:30:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe5cc0a4910 0x7fe5cc1d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5c9d9b640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc0a68e0 0x7fe5cc1d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5cad9d640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 0x7fe5cc13d630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401d0a0 tx=0x7fe5c40195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.008+0000 7fe5baffd640 10 client.?.objecter ms_handle_connect 0x7fe5cc1396a0 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40bb020 con 0x7fe5cc1396a0 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: dump: 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c40b3040 con 0x7fe5cc1396a0 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 5 monclient: authenticate success, global_id 34371 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 monclient: _renew_subs 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc13f360 con 0x7fe5cc1396a0 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40b2070 con 0x7fe5cc1396a0 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: dump: 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 client.34371.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 monclient: _renew_subs 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5bbfff640 4 set_mon_vals no callback set 2024-09-17T16:30:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1233e0 con 0x7fe5cc1396a0 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 1 librados: init done 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 0x7fe5cc13df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5cc051e80 con 0x7fe5cc13db70 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe5c40be020 con 0x7fe5cc1396a0 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 0x7fe5cc13df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient: get_auth_request con 0x7fe5cc13db70 auth_method 0 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5ac03e550 0x7fe5ac040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7481+0+0 (secure 0 0 0) 0x7fe5c40eac00 con 0x7fe5cc1396a0 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 client.34371.objecter ms_dispatch 0x7fe5cc000ba0 osd_map(108..108 src has 1..108) 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 3 client.34371.objecter handle_osd_map got epochs [108,108] > 0 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 3 client.34371.objecter handle_osd_map decoding full epoch 108 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 20 client.34371.objecter dump_active .. 0 homeless 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5c9d9b640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5ac03e550 0x7fe5ac040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5c9d9b640 10 monclient: get_auth_request con 0x7fe5ac03e550 auth_method 0 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_done global_id 19220 payload 824 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5cc13fcd0 con 0x7fe5cc13db70 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 client.34371.objecter ms_handle_connect 0x7fe5cc13db70 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 client.34371.objecter resend_mon_ops 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 0x7fe5cc13df50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c000f610 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c0019070 con 0x7fe5cc13db70 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: dump: 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 client.34371.objecter ms_handle_connect 0x7fe5ac03e550 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5c9d9b640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5ac03e550 0x7fe5ac040a10 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc13ec70 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe5c0004030 con 0x7fe5cc13db70 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 msgr2=0x7fe5cc13df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 0x7fe5cc13df50 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c000f610 comp rx=0 tx=0).stop 2024-09-17T16:30:37.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 msgr2=0x7fe5cc13df50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.012+0000 7fe5baffd640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13db70 0x7fe5cc13df50 secure :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c000f610 comp rx=0 tx=0).stop 2024-09-17T16:30:37.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 0x7fe5cc0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe5cc139b50 con 0x7fe5cc13e490 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 0x7fe5cc0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient: get_auth_request con 0x7fe5cc13e490 auth_method 0 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 10 monclient(hunting): handle_auth_done global_id 19229 payload 824 2024-09-17T16:30:37.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 1 -- 172.21.15.105:0/1476613564 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe5cc123900 con 0x7fe5cc13e490 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5ca59c640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 0x7fe5cc0a0a20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c00046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 client.34371.objecter ms_handle_connect 0x7fe5cc13e490 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 client.34371.objecter resend_mon_ops 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c0019040 con 0x7fe5cc13e490 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 monclient: dump: 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.046 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fe5c001e660 con 0x7fe5cc13e490 2024-09-17T16:30:37.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 msgr2=0x7fe5cc0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 0x7fe5cc0a0a20 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c00046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 msgr2=0x7fe5cc0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5baffd640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe5cc13e490 0x7fe5cc0a0a20 secure :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0011cf0 tx=0x7fe5c00046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 184624, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:37.049 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.050 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.051 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.051 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.051 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:37.053 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:37.054 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.055 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.056 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.058 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:37.060 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:37.064 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:37.066 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.068 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.069 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:37.070 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:37.071 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:37.072 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 librados: watch_flush enter 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 librados: watch_flush exit 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 20 client.34371.objecter shutdown clearing up homeless session... 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 client.34371.objecter successfully canceled tick 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5ac03e550 msgr2=0x7fe5ac040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe5ac03e550 0x7fe5ac040a10 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc13ec70 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 10 monclient: shutdown 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 msgr2=0x7fe5cc13d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 --2- 172.21.15.105:0/1476613564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5cc1396a0 0x7fe5cc13d630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401d0a0 tx=0x7fe5c40195a0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 reap_dead start 2024-09-17T16:30:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5cad9d640 1 -- 172.21.15.105:0/1476613564 reap_dead start 2024-09-17T16:30:37.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 shutdown_connections 2024-09-17T16:30:37.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 >> 172.21.15.105:0/1476613564 conn(0x7fe5cc068ba0 msgr2=0x7fe5cc0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 shutdown_connections 2024-09-17T16:30:37.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 -- 172.21.15.105:0/1476613564 wait complete. 2024-09-17T16:30:37.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.020+0000 7fe5d0e06640 1 librados: shutdown 2024-09-17T16:30:37.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:37.075 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:30:37.075 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:30:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: get_monmap_and_config 2024-09-17T16:30:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: monmap: 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:37.188833+0000 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:37.188833+0000 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:37.192 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 Processor -- start 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 -- start start 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: init 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient: _add_conns ranks=[10,0,6,5,4,9,7,3,1,8,2] 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f00780a6a30 0x7f00780a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): picked mon.noname-f con 0x7f00780a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f00780a4ab0 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 0x7f0078139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): picked mon.noname-d con 0x7f00780a7350 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00780a4080 con 0x7f00780a4ab0 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f00780a7c40 con 0x7f00780a6a30 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f00780a37e0 con 0x7f00780a7350 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007f579640 10 monclient(hunting): authenticate will time out at 1120.454555s 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007cb14640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f00780a4ab0 0x7f00781397f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43136/0 (socket says 172.21.15.105:43136) 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007cb14640 1 -- 172.21.15.105:0/3084195124 learned_addr learned my addr 172.21.15.105:0/3084195124 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 --2- 172.21.15.105:0/3084195124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 0x7f0078139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): get_auth_request con 0x7f00780a7350 auth_method 0 2024-09-17T16:30:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007d315640 1 --2- 172.21.15.105:0/3084195124 >> v2:172.21.15.160:3300/0 conn(0x7f00780a6a30 0x7f00780a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 1 -- 172.21.15.105:0/3084195124 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270703386 0 0) 0x7f00780a4080 con 0x7f00780a4ab0 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 10 monclient(hunting): _negotiate 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 10 monclient(hunting): my global_id is 34380 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f006ffff640 1 -- 172.21.15.105:0/3084195124 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0064003530 con 0x7f00780a4ab0 2024-09-17T16:30:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007d315640 10 monclient(hunting): get_auth_request con 0x7f00780a6a30 auth_method 0 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007d315640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007d315640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient(hunting): handle_auth_done global_id 23523 payload 274 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 -- 172.21.15.105:0/3084195124 >> v2:172.21.15.160:3300/0 conn(0x7f00780a6a30 msgr2=0x7f00780a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 --2- 172.21.15.105:0/3084195124 >> v2:172.21.15.160:3300/0 conn(0x7f00780a6a30 0x7f00780a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 -- 172.21.15.105:0/3084195124 >> v1:172.21.15.105:6789/0 conn(0x7f00780a4ab0 legacy=0x7f00781397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 monclient: found mon.noname-d 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.184+0000 7f007db16640 1 -- 172.21.15.105:0/3084195124 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00780a3ac0 con 0x7f00780a7350 2024-09-17T16:30:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007db16640 1 --2- 172.21.15.105:0/3084195124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 0x7f0078139f00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0070009d90 tx=0x7f007001c7f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ba289710950db40 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 1 -- 172.21.15.105:0/3084195124 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00700b2070 con 0x7f00780a7350 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: dump: 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.206 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 5 monclient: authenticate success, global_id 23523 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 1 -- 172.21.15.105:0/3084195124 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00700a5b80 con 0x7f00780a7350 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 4 set_mon_vals no callback set 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 1 -- 172.21.15.105:0/3084195124 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00700a5d70 con 0x7f00780a7350 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f006ffff640 10 monclient: dump: 2024-09-17T16:30:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: shutdown 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- 172.21.15.105:0/3084195124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 msgr2=0x7f0078139f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- 172.21.15.105:0/3084195124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 0x7f0078139f00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0070009d90 tx=0x7f007001c7f0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- 172.21.15.105:0/3084195124 shutdown_connections 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- 172.21.15.105:0/3084195124 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00780a7350 0x7f0078139f00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- 172.21.15.105:0/3084195124 >> v2:172.21.15.160:3300/0 conn(0x7f00780a6a30 0x7f00780a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- 172.21.15.105:0/3084195124 >> 172.21.15.105:0/3084195124 conn(0x7f00780a8d00 msgr2=0x7f00780a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- 172.21.15.105:0/3084195124 shutdown_connections 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- 172.21.15.105:0/3084195124 wait complete. 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: monmap: 2024-09-17T16:30:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 librados: starting msgr at 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 librados: starting objecter 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 Processor -- start 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- start start 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 librados: setting wanted keys 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 librados: calling monclient init 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: init 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient: _add_conns ranks=[3,0,7,4,2,5,8,6,1] 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00780a4ab0 0x7f00781d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): picked mon.noname-g con 0x7f00780a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 0x7f00781d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): picked mon.noname-a con 0x7f00780a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00780a7350 0x7f00781d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): picked mon.noname-i con 0x7f00780a7350 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f007813a440 con 0x7f00780a6a30 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f007806e770 con 0x7f00780a4ab0 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f007813a5c0 con 0x7f00780a7350 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007f579640 10 monclient(hunting): authenticate will time out at 1120.459698s 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 0x7f00781d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 0x7f00781d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52054/0 (socket says 172.21.15.105:52054) 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 1 -- 172.21.15.105:0/2776020556 learned_addr learned my addr 172.21.15.105:0/2776020556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): get_auth_request con 0x7f00780a6a30 auth_method 0 2024-09-17T16:30:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.188+0000 7f007cb14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00780a4ab0 0x7f00781d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00780a7350 0x7f00781d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): get_auth_request con 0x7f00780a4ab0 auth_method 0 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 10 monclient(hunting): handle_auth_done global_id 34389 payload 824 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00780a7350 msgr2=0x7f00781d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00780a7350 0x7f00781d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00780a4ab0 msgr2=0x7f00781d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00780a4ab0 0x7f00781d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 monclient: found mon.noname-a 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0078122890 con 0x7f00780a6a30 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 10 monclient: get_auth_request con 0x7f00780a7350 auth_method 0 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00780a7350 0x7f00781d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007cb14640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 0x7f00781d1540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0074001ec0 tx=0x7f007400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.?.objecter ms_handle_connect 0x7f00780a6a30 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0074017070 con 0x7f00780a6a30 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: dump: 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0074007070 con 0x7f00780a6a30 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0074007710 con 0x7f00780a6a30 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006f7fe640 4 set_mon_vals no callback set 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: dump: 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 5 monclient: authenticate success, global_id 34389 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 monclient: _renew_subs 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00780a1e00 con 0x7f00780a6a30 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 client.34389.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 monclient: _renew_subs 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007813f790 con 0x7f00780a6a30 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 1 librados: init done 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 0x7f007813de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f00781394f0 con 0x7f00781233b0 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f00740072a0 con 0x7f00780a6a30 2024-09-17T16:30:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 0x7f007813de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient: get_auth_request con 0x7f00781233b0 auth_method 0 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f005c03e580 0x7f005c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7481+0+0 (secure 0 0 0) 0x7f007404f580 con 0x7f00780a6a30 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.34389.objecter ms_dispatch 0x7f0078000ba0 osd_map(108..108 src has 1..108) 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 3 client.34389.objecter handle_osd_map got epochs [108,108] > 0 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 3 client.34389.objecter handle_osd_map decoding full epoch 108 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 20 client.34389.objecter dump_active .. 0 homeless 2024-09-17T16:30:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f005c03e580 0x7f005c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 10 monclient(hunting): handle_auth_done global_id 34225 payload 824 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00780a6e10 con 0x7f00781233b0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 10 monclient: get_auth_request con 0x7f005c03e580 auth_method 0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.34389.objecter ms_handle_connect 0x7f00781233b0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.34389.objecter resend_mon_ops 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007d315640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 0x7f007813de00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f006800f4b0 tx=0x7f006800f970 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0068019020 con 0x7f00781233b0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: dump: 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 client.34389.objecter ms_handle_connect 0x7f005c03e580 2024-09-17T16:30:37.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f007db16640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f005c03e580 0x7f005c040a40 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 crypto rx=0x7f00700af000 tx=0x7f00700a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0068014670 con 0x7f00781233b0 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 msgr2=0x7f007813de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 0x7f007813de00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f006800f4b0 tx=0x7f006800f970 comp rx=0 tx=0).stop 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 msgr2=0x7f007813de00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.192+0000 7f006dffb640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00781233b0 0x7f007813de00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007f579640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:37.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007f579640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 0x7f007813e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007f579640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0078051e80 con 0x7f007813e340 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 0x7f007813e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient: get_auth_request con 0x7f007813e340 auth_method 0 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 10 monclient(hunting): handle_auth_done global_id 34234 payload 824 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 1 -- 172.21.15.105:0/2776020556 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f007813ec90 con 0x7f007813e340 2024-09-17T16:30:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f007d315640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 0x7f007813e750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f006801a000 tx=0x7f00680046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 10 client.34389.objecter ms_handle_connect 0x7f007813e340 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 10 client.34389.objecter resend_mon_ops 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0068019040 con 0x7f007813e340 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.200+0000 7f006dffb640 10 monclient: dump: 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f00680157e0 con 0x7f007813e340 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 msgr2=0x7f007813e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 0x7f007813e750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f006801a000 tx=0x7f00680046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 msgr2=0x7f007813e750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f006dffb640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f007813e340 0x7f007813e750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 28, 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28003, 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:37.227 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.228 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.229 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:37.230 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:37.231 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.232 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:37.233 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.234 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:37.235 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.236 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:37.237 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.239 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.240 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.245 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:37.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:37.248 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 10 librados: watch_flush enter 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 10 librados: watch_flush exit 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 20 client.34389.objecter shutdown clearing up homeless session... 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 10 client.34389.objecter successfully canceled tick 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f005c03e580 msgr2=0x7f005c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f005c03e580 0x7f005c040a40 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 crypto rx=0x7f00700af000 tx=0x7f00700a7070 comp rx=0 tx=0).stop 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 10 monclient: shutdown 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 msgr2=0x7f00781d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 --2- 172.21.15.105:0/2776020556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00780a6a30 0x7f00781d1540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0074001ec0 tx=0x7f007400c960 comp rx=0 tx=0).stop 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007db16640 1 -- 172.21.15.105:0/2776020556 reap_dead start 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007db16640 1 -- 172.21.15.105:0/2776020556 reap_dead start 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 shutdown_connections 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 >> 172.21.15.105:0/2776020556 conn(0x7f00780a8d00 msgr2=0x7f00780aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 shutdown_connections 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 -- 172.21.15.105:0/2776020556 wait complete. 2024-09-17T16:30:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.204+0000 7f007f579640 1 librados: shutdown 2024-09-17T16:30:37.251 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:30:37.251 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-17T16:30:37.252 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['d'] 2024-09-17T16:30:37.252 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-17T16:30:37.252 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-09-17T16:30:37.252 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-09-17T16:30:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.360+0000 7fc21e01a640 10 monclient: get_monmap_and_config 2024-09-17T16:30:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.360+0000 7fc21e01a640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.360+0000 7fc21e01a640 10 monclient: monmap: 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:37.366837+0000 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:37.366837+0000 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:37.370 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 Processor -- start 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 -- start start 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient: init 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient: _add_conns ranks=[4,8,6,5,7,3,2,0,10,1,9] 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 0x7fc2180a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-c con 0x7fc2180a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-e con 0x7fc2180a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-d con 0x7fc2181396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc2180a3fc0 con 0x7fc2180a68e0 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc2180a63c0 con 0x7fc2181396a0 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc2180a7740 con 0x7fc2180a4910 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc21e01a640 10 monclient(hunting): authenticate will time out at 1120.633083s 2024-09-17T16:30:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45348/0 (socket says 172.21.15.105:45348) 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37616/0 (socket says 172.21.15.105:37616) 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 -- 172.21.15.105:0/3945124607 learned_addr learned my addr 172.21.15.105:0/3945124607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): get_auth_request con 0x7fc2180a4910 auth_method 0 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 0x7fc2180a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): get_auth_request con 0x7fc2181396a0 auth_method 0 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): get_auth_request con 0x7fc2180a68e0 auth_method 0 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient(hunting): handle_auth_done global_id 23516 payload 274 2024-09-17T16:30:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 -- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 msgr2=0x7fc218139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 -- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 msgr2=0x7fc2180a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 0x7fc2180a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 monclient: found mon.noname-e 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 -- 172.21.15.105:0/3945124607 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21806ec10 con 0x7fc2180a4910 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc2177fe640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 0x7fc2180a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc217fff640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.364+0000 7fc216ffd640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc2080098d0 tx=0x7fc20801cea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1c7a7372e47be60 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 1 -- 172.21.15.105:0/3945124607 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2080b0070 con 0x7fc2180a4910 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: dump: 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 1 -- 172.21.15.105:0/3945124607 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080aa810 con 0x7fc2180a4910 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 5 monclient: authenticate success, global_id 23516 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 1 -- 172.21.15.105:0/3945124607 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2080a9300 con 0x7fc2180a4910 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 4 set_mon_vals no callback set 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2167fc640 10 monclient: dump: 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.385 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.386 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: shutdown 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 msgr2=0x7fc2180a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc2080098d0 tx=0x7fc20801cea0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/3945124607 shutdown_connections 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc2181396a0 0x7fc218139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a4910 0x7fc2180a7200 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- 172.21.15.105:0/3945124607 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc2180a68e0 0x7fc2180a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/3945124607 >> 172.21.15.105:0/3945124607 conn(0x7fc218068ba0 msgr2=0x7fc218068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/3945124607 shutdown_connections 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/3945124607 wait complete. 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: build_initial_monmap 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: monmap: 2024-09-17T16:30:37.387 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:37.388 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 librados: starting msgr at 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 librados: starting objecter 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 Processor -- start 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- start start 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 librados: setting wanted keys 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 librados: calling monclient init 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: init 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _add_conns ranks=[7,8,5,4,6,0,1,2,3] 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc2180a4910 0x7fc218122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-i con 0x7fc2180a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 0x7fc2181d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-e con 0x7fc2180a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc2181396a0 0x7fc21813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): picked mon.noname-h con 0x7fc2181396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc218143320 con 0x7fc2181396a0 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc21806e630 con 0x7fc2180a4910 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc2181434a0 con 0x7fc2180a68e0 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): authenticate will time out at 1120.638183s 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 0x7fc2181d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 0x7fc2181d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:45360/0 (socket says 172.21.15.105:45360) 2024-09-17T16:30:37.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 -- 172.21.15.105:0/1036271638 learned_addr learned my addr 172.21.15.105:0/1036271638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): get_auth_request con 0x7fc2180a68e0 auth_method 0 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient(hunting): handle_auth_done global_id 23525 payload 824 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc2180a4910 msgr2=0x7fc218122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc2180a4910 0x7fc218122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc217fff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc2181396a0 0x7fc21813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc2181396a0 msgr2=0x7fc21813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc2181396a0 0x7fc21813d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 monclient: found mon.noname-e 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180aa070 con 0x7fc2180a68e0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc216ffd640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 0x7fc2181d1a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc2080098d0 tx=0x7fc2080191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 client.?.objecter ms_handle_connect 0x7fc2180a68e0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2080b9070 con 0x7fc2180a68e0 2024-09-17T16:30:37.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: dump: 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.393 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: _finish_auth 0 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080044b0 con 0x7fc2180a68e0 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 5 monclient: authenticate success, global_id 23525 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _renew_subs 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc218077ba0 con 0x7fc2180a68e0 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2080b0040 con 0x7fc2180a68e0 2024-09-17T16:30:37.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc215ffb640 4 set_mon_vals no callback set 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 client.23525.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 10 monclient: dump: 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _renew_subs 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fc218123a90 con 0x7fc2180a68e0 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 librados: init done 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 0x7fc21813da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc218051e80 con 0x7fc2181d1f60 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2177fe640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 0x7fc21813da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2177fe640 10 monclient: get_auth_request con 0x7fc2181d1f60 auth_method 0 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2177fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc2177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.368+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc2080aac00 con 0x7fc2180a68e0 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 10 monclient(hunting): handle_auth_done global_id 23762 payload 824 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc218123660 con 0x7fc2181d1f60 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc20403e540 0x7fc204040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(108..108 src has 1..108) ==== 7481+0+0 (secure 0 0 0) 0x7fc2080e9500 con 0x7fc2180a68e0 2024-09-17T16:30:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 client.23525.objecter ms_dispatch 0x7fc218000ba0 osd_map(108..108 src has 1..108) 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 3 client.23525.objecter handle_osd_map got epochs [108,108] > 0 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 3 client.23525.objecter handle_osd_map decoding full epoch 108 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc2177fe640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 0x7fc21813da20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c00eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 20 client.23525.objecter dump_active .. 0 homeless 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 client.23525.objecter ms_handle_connect 0x7fc2181d1f60 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 client.23525.objecter resend_mon_ops 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc20c017070 con 0x7fc2181d1f60 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc217fff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc20403e540 0x7fc204040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 monclient: dump: 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc217fff640 10 monclient: get_auth_request con 0x7fc20403e540 auth_method 0 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc20c00c720 con 0x7fc2181d1f60 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 msgr2=0x7fc21813da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 0x7fc21813da20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c00eb70 comp rx=0 tx=0).stop 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 msgr2=0x7fc21813da20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2181d1f60 0x7fc21813da20 secure :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c00eb70 comp rx=0 tx=0).stop 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc1fffff640 10 client.23525.objecter ms_handle_connect 0x7fc20403e540 2024-09-17T16:30:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.372+0000 7fc217fff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc20403e540 0x7fc204040a00 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7fc2181241d0 tx=0x7fc21000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.376+0000 7fc21e01a640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.376+0000 7fc21e01a640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 0x7fc2180a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.376+0000 7fc21e01a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.376+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc21813e340 con 0x7fc21813df60 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 0x7fc2180a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient: get_auth_request con 0x7fc21813df60 auth_method 0 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 10 monclient(hunting): handle_auth_done global_id 23771 payload 824 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 1 -- 172.21.15.105:0/1036271638 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fc218139ae0 con 0x7fc21813df60 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 client.23525.objecter ms_handle_connect 0x7fc21813df60 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 client.23525.objecter resend_mon_ops 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc2177fe640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 0x7fc2180a0a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc20c02b080 con 0x7fc21813df60 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 monclient: dump: 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fc20c006440 con 0x7fc21813df60 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:30:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 msgr2=0x7fc2180a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 0x7fc2180a0a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 msgr2=0x7fc2180a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc1fffff640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc21813df60 0x7fc2180a0a20 secure :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00e770 tx=0x7fc20c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:30:37.405 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 10 librados: watch_flush enter 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 10 librados: watch_flush exit 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 20 client.23525.objecter shutdown clearing up homeless session... 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 10 client.23525.objecter successfully canceled tick 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc20403e540 msgr2=0x7fc204040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc20403e540 0x7fc204040a00 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7fc2181241d0 tx=0x7fc21000b040 comp rx=0 tx=0).stop 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 10 monclient: shutdown 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 msgr2=0x7fc2181d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 --2- 172.21.15.105:0/1036271638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc2180a68e0 0x7fc2181d1a20 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc2080098d0 tx=0x7fc2080191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc217fff640 1 -- 172.21.15.105:0/1036271638 reap_dead start 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc217fff640 1 -- 172.21.15.105:0/1036271638 reap_dead start 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 shutdown_connections 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 >> 172.21.15.105:0/1036271638 conn(0x7fc218068ba0 msgr2=0x7fc2180a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 shutdown_connections 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 -- 172.21.15.105:0/1036271638 wait complete. 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:37.380+0000 7fc21e01a640 1 librados: shutdown 2024-09-17T16:30:37.408 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-17T16:30:37.408 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T16:30:37.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:30:37.420 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T16:30:37.420 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-17T16:30:37.421 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-17T16:30:37.421 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:30:51.691 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: Running main() from gmock_main.cc 2024-09-17T16:30:51.691 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-09-17T16:30:51.691 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] Global test environment set-up. 2024-09-17T16:30:51.691 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2351 ms) 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3024 ms) 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3018 ms) 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3033 ms) 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3395 ms) 2024-09-17T16:30:51.692 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3047 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3045 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2887 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3014 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3063 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2921 ms) 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-17T16:30:51.693 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3091 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2933 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2974 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.Flush (3032 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2976 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3005 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3002 ms) 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-17T16:30:51.694 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3003 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3027 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2958 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3021 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (8114 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2865 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (7210 ms) 2024-09-17T16:30:51.695 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-17T16:30:51.696 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (5876 ms) 2024-09-17T16:30:51.696 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 26 tests from LibRadosAio (89887 ms total) 2024-09-17T16:30:51.696 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: 2024-09-17T16:30:51.696 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-17T16:30:51.696 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-17T16:30:52.421 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-17T16:30:52.422 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-17T16:30:52.422 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-17T16:30:52.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:30:52.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: get_monmap_and_config 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: build_initial_monmap 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: monmap: 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:52.548830+0000 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:52.548830+0000 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:52.551 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:52.552 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 Processor -- start 2024-09-17T16:30:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 -- start start 2024-09-17T16:30:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: init 2024-09-17T16:30:52.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient: _add_conns ranks=[8,10,7,5,6,0,2,1,4,9,3] 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-e con 0x7f04f00a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-f con 0x7f04f00a4ab0 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-i con 0x7f04f01397f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f04f00a7c40 con 0x7f04f00a4ab0 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f04f00a7890 con 0x7f04f01397f0 2024-09-17T16:30:52.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f04f00a4080 con 0x7f04f00a6a30 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04f5782640 10 monclient(hunting): authenticate will time out at 1135.814484s 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:60704/0 (socket says 172.21.15.105:60704) 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 learned_addr learned my addr 172.21.15.105:0/1741777968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04f00a6a30 auth_method 0 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04f01397f0 auth_method 0 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04f00a4ab0 auth_method 0 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04f00a6a30 auth_method 0 2024-09-17T16:30:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04f00a4ab0 auth_method 0 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04f01397f0 auth_method 0 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:52.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04f00a6a30 auth_method 0 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04f00a4ab0 auth_method 0 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04f01397f0 auth_method 0 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:30:53.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:30:53.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:53.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04f00a6a30 auth_method 0 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04f00a4ab0 auth_method 0 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04f01397f0 auth_method 0 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:30:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:53.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:53.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:30:55.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): tick 2024-09-17T16:30:55.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 msgr2=0x7f04f00a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f04f00a6a30 0x7f04f00a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 msgr2=0x7f04f0139ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04f01397f0 0x7f04f0139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 msgr2=0x7f04f00a7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 --2- 172.21.15.105:0/1741777968 >> v2:172.21.15.160:3300/0 conn(0x7f04f00a4ab0 0x7f04f00a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient: _add_conns ranks=[1,4,5,2,9,6,3,0] 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f04d8003830 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): picked mon.noname-c con 0x7f04d8007420 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:55.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): picked mon.noname-h con 0x7f04d8009f10 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f00a37e0 con 0x7f04d8003830 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f04f00a7c40 con 0x7f04d8007420 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f04f00a7890 con 0x7f04d8009f10 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ecff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04d8007420 auth_method 0 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:55.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04d8009f10 auth_method 0 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:55.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.544+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:55.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:55.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04d8007420 auth_method 0 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04d8009f10 auth_method 0 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:55.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:55.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:55.748+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:30:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04d8007420 auth_method 0 2024-09-17T16:30:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04d8009f10 auth_method 0 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:56.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:56.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.152+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04d8007420 auth_method 0 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:56.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04d8009f10 auth_method 0 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:30:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:56.956+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:30:57.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819201206 0 0) 0x7f04f00a37e0 con 0x7f04d8003830 2024-09-17T16:30:57.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 10 monclient(hunting): _negotiate 2024-09-17T16:30:57.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 10 monclient(hunting): my global_id is 24598 2024-09-17T16:30:57.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.452+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04dc003530 con 0x7f04d8003830 2024-09-17T16:30:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 664095446 0 0) 0x7f04dc003530 con 0x7f04d8003830 2024-09-17T16:30:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 msgr2=0x7f04d800c3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04d8009f10 0x7f04d800c3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 msgr2=0x7f04d80098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04d8007420 0x7f04d80098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 monclient: found mon.noname-f-legacy 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 reap_dead start 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.160:6789/0 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d800dc80 con 0x7f04d8003830 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f04e000f040 con 0x7f04d8003830 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: dump: 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] but i'm connected to v1:172.21.15.160:6789/0, reconnecting 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 >> v1:172.21.15.160:6789/0 conn(0x7f04d8003830 legacy=0x7f04d8005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient: _add_conns ranks=[1,5,6,2,3,8,0,7,4] 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004210 0x7f04dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): picked mon.f con 0x7f04dc004210 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04dc004e70 0x7f04dc0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): picked mon.h con 0x7f04dc004e70 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 0x7f04dc00c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): picked mon.d con 0x7f04dc009f00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f04f00a7c40 con 0x7f04dc004210 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f04f00a7890 con 0x7f04dc004e70 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f04dc00c990 con 0x7f04dc009f00 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04edffb640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:30:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04dc004e70 0x7f04dc0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 0x7f04dc00c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04dc004e70 auth_method 0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04dc009f00 auth_method 0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004210 0x7f04dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04dc004210 auth_method 0 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_done global_id 24598 payload 293 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04dc004e70 msgr2=0x7f04dc0099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04dc004e70 0x7f04dc0099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004210 msgr2=0x7f04dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:57.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004210 0x7f04dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 monclient: found mon.d 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 1 -- 172.21.15.105:0/1741777968 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc003530 con 0x7f04dc009f00 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04f5782640 5 monclient: authenticate success, global_id 24598 2024-09-17T16:30:57.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.464+0000 7f04f5782640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:57.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.468+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 0x7f04dc00c2f0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f04f00a7bd0 tx=0x7f04e801e350 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8fb231e6390d7d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e80b4070 con 0x7f04dc009f00 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: dump: 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e80ad780 con 0x7f04dc009f00 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 4 set_mon_vals no callback set 2024-09-17T16:30:57.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 1 -- 172.21.15.105:0/1741777968 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e80099d0 con 0x7f04dc009f00 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04edffb640 10 monclient: dump: 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.486 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.487 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 10 monclient: shutdown 2024-09-17T16:30:57.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 -- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 msgr2=0x7f04dc00c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 0x7f04dc00c2f0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f04f00a7bd0 tx=0x7f04e801e350 comp rx=0 tx=0).stop 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 -- 172.21.15.105:0/1741777968 shutdown_connections 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04dc009f00 0x7f04dc00c2f0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04dc004e70 0x7f04dc0099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 --2- 172.21.15.105:0/1741777968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004210 0x7f04dc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 -- 172.21.15.105:0/1741777968 >> 172.21.15.105:0/1741777968 conn(0x7f04f00a8d00 msgr2=0x7f04f00a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 -- 172.21.15.105:0/1741777968 shutdown_connections 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.476+0000 7f04f5782640 1 -- 172.21.15.105:0/1741777968 wait complete. 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient: build_initial_monmap 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient: monmap: 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:57.491 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 librados: starting msgr at 2024-09-17T16:30:57.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 librados: starting objecter 2024-09-17T16:30:57.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 Processor -- start 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 -- start start 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 librados: setting wanted keys 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 librados: calling monclient init 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient: init 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient: _add_conns ranks=[3,0,1,7,6,8,5,4,2] 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-g con 0x7f04d8003830 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-a con 0x7f04dc004210 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 0x7f04f006f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): picked mon.noname-f con 0x7f04dc004e70 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04f006e680 con 0x7f04dc004210 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f04f006b5b0 con 0x7f04dc004e70 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f04f006e800 con 0x7f04d8003830 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04f5782640 10 monclient(hunting): authenticate will time out at 1140.751078s 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56490/0 (socket says 172.21.15.105:56490) 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 -- 172.21.15.105:0/4013795792 learned_addr learned my addr 172.21.15.105:0/4013795792 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:59756/0 (socket says 172.21.15.105:59756) 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request con 0x7f04dc004210 auth_method 0 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): get_auth_request con 0x7f04d8003830 auth_method 0 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 0x7f04f006f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request con 0x7f04dc004e70 auth_method 0 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient(hunting): handle_auth_done global_id 34416 payload 824 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 -- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 msgr2=0x7f04f00a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 -- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 msgr2=0x7f04f006f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 0x7f04f006f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 monclient: found mon.noname-a 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ee7fc640 1 -- 172.21.15.105:0/4013795792 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f00a9f20 con 0x7f04dc004210 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04ef7fe640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 0x7f04f006f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.480+0000 7f04eeffd640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04ee7fc640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003690 tx=0x7f04e4003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 client.?.objecter ms_handle_connect 0x7f04dc004210 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e4017e90 con 0x7f04dc004210 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: dump: 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 5 monclient: authenticate success, global_id 34416 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 10 monclient: _renew_subs 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f0138a90 con 0x7f04dc004210 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4028070 con 0x7f04dc004210 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 10 client.34416.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 10 monclient: _renew_subs 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f0071270 con 0x7f04dc004210 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04f5782640 1 librados: init done 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e4023400 con 0x7f04dc004210 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 monclient: dump: 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f04e4022070 con 0x7f04dc004210 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 0x7f04d8041e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8261+0+0 (secure 0 0 0) 0x7f04e400f040 con 0x7f04dc004210 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 10 client.34416.objecter ms_dispatch 0x7f04f0000ba0 osd_map(109..109 src has 1..109) 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 3 client.34416.objecter handle_osd_map got epochs [109,109] > 0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 3 client.34416.objecter handle_osd_map decoding full epoch 109 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04d77fe640 20 client.34416.objecter dump_active .. 0 homeless 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04eeffd640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 0x7f04d8041e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.484+0000 7f04eeffd640 10 monclient: get_auth_request con 0x7f04d803f9a0 auth_method 0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04ecff9640 4 set_mon_vals no callback set 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04f5782640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04d77fe640 10 client.34416.objecter ms_handle_connect 0x7f04d803f9a0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04f5782640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04eeffd640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 0x7f04d8041e60 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7f04e0001ec0 tx=0x7f04e0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04f5782640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.488+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d0005090 con 0x7f04dc004210 2024-09-17T16:30:57.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.492+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04e4031060 con 0x7f04dc004210 2024-09-17T16:30:57.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.492+0000 7f04d77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:57.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.492+0000 7f04d77fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:57.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.760+0000 7f04f5782640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.760+0000 7f04f5782640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f04d0005650 con 0x7f04dc004210 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04d77fe640 1 -- 172.21.15.105:0/4013795792 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3206 (secure 0 0 0) 0x7f04e4020040 con 0x7f04dc004210 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04d77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:30:57.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04d77fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:57.771 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":38,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 10 librados: watch_flush enter 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 10 librados: watch_flush exit 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 20 client.34416.objecter shutdown clearing up homeless session... 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 10 client.34416.objecter successfully canceled tick 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 msgr2=0x7f04d8041e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 0x7f04d8041e60 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7f04e0001ec0 tx=0x7f04e0008040 comp rx=0 tx=0).stop 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 10 monclient: shutdown 2024-09-17T16:30:57.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 msgr2=0x7f04f0073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04e4003690 tx=0x7f04e4003b50 comp rx=0 tx=0).stop 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 shutdown_connections 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f04d803f9a0 0x7f04d8041e60 unknown :-1 s=CLOSED pgs=3368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04dc004e70 0x7f04f006f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04dc004210 0x7f04f0073010 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 --2- 172.21.15.105:0/4013795792 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04d8003830 0x7f04f00a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 >> 172.21.15.105:0/4013795792 conn(0x7f04f00a8d00 msgr2=0x7f04f00a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 shutdown_connections 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 -- 172.21.15.105:0/4013795792 wait complete. 2024-09-17T16:30:57.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.764+0000 7f04f5782640 1 librados: shutdown 2024-09-17T16:30:57.790 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-17T16:30:57.790 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-17T16:30:57.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: get_monmap_and_config 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: build_initial_monmap 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: monmap: 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:57.908206+0000 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:57.908206+0000 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:57.911 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 Processor -- start 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 -- start start 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: init 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient: _add_conns ranks=[2,0,3,4,8,5,10,7,9,6,1] 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1ce00a4e00 0x7f1ce0137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-b con 0x7f1ce00a4e00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ce0137b50 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:57.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1ce00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-g con 0x7f1ce0137f90 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce0138440 con 0x7f1ce0137b50 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce00a1c30 con 0x7f1ce00a4e00 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce00a18f0 con 0x7f1ce0137f90 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 10 monclient(hunting): authenticate will time out at 1141.174140s 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1ce00a4e00 msgr2=0x7f1ce0137610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1ce00a4e00 0x7f1ce0137610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1ce0137b50 0x7f1ce00a51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37056/0 (socket says 172.21.15.105:37056) 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3888142790 learned_addr learned my addr 172.21.15.105:0/3888142790 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801432063 0 0) 0x7f1cc80014a0 con 0x7f1ce0137b50 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): _negotiate 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): my global_id is 34425 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1ce00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cc0003530 con 0x7f1ce0137b50 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): get_auth_request con 0x7f1ce0137f90 auth_method 0 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 509395184 0 0) 0x7f1cc0003530 con 0x7f1ce0137b50 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 msgr2=0x7f1ce00a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1ce00a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1ce00a4e00 msgr2=0x7f1ce0137610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1ce00a4e00 0x7f1ce0137610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00a2c40 con 0x7f1ce0137b50 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f1cc8001ec0 con 0x7f1ce0137b50 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: dump: 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 >> v1:172.21.15.105:6789/0 conn(0x7f1ce0137b50 legacy=0x7f1ce00a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient: _add_conns ranks=[3,0,5,8,2,6,7,4,1] 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1cc0004210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): picked mon.g con 0x7f1ce0137f90 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): picked mon.a con 0x7f1cc0004810 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1cc00099c0 0x7f1cc000bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): picked mon.h con 0x7f1cc00099c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00a1c30 con 0x7f1cc0004810 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce00a18f0 con 0x7f1ce0137f90 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1cc000c480 con 0x7f1cc00099c0 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cde7fc640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): get_auth_request con 0x7f1cc0004810 auth_method 0 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1cc0004210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f1ce0137f90 auth_method 0 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1cc00099c0 0x7f1cc000bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_done global_id 34425 payload 293 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1cc00099c0 msgr2=0x7f1cc000bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1cc00099c0 0x7f1cc000bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 msgr2=0x7f1cc0004210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce0137f90 0x7f1cc0004210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 monclient: found mon.a 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3888142790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc0003530 con 0x7f1cc0004810 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 5 monclient: authenticate success, global_id 34425 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1ce5d1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 10 monclient: get_auth_request con 0x7f1cc00099c0 auth_method 0 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.904+0000 7f1cdffff640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1cc00099c0 0x7f1cc000bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aae10 tx=0x7f1cc801cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a33cfb43644db36 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cc800a480 con 0x7f1cc0004810 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: dump: 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc80a9070 con 0x7f1cc0004810 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 1 -- 172.21.15.105:0/3888142790 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cc80aed50 con 0x7f1cc0004810 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 4 set_mon_vals no callback set 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cde7fc640 10 monclient: dump: 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.934 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 10 monclient: shutdown 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc0004810 msgr2=0x7f1cc0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3888142790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cc0004810 0x7f1cc0004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00aae10 tx=0x7f1cc801cfc0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1cdffff640 1 -- 172.21.15.105:0/3888142790 reap_dead start 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3888142790 shutdown_connections 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3888142790 >> 172.21.15.105:0/3888142790 conn(0x7f1ce00aa250 msgr2=0x7f1ce00aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3888142790 shutdown_connections 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3888142790 wait complete. 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 10 monclient: build_initial_monmap 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 10 monclient: monmap: 2024-09-17T16:30:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 librados: starting msgr at 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 librados: starting objecter 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 Processor -- start 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 -- start start 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 librados: setting wanted keys 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 1 librados: calling monclient init 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.908+0000 7f1ce5d1c640 10 monclient: init 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _add_conns ranks=[2,0,3,5,7,6,4,1,8] 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1cc0004810 0x7f1ce013ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-b con 0x7f1cc0004810 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:57.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-a con 0x7f1ce0136c90 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce006d220 0x7f1ce006d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): picked mon.noname-g con 0x7f1ce006d220 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce013f870 con 0x7f1ce0136c90 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce006b5b0 con 0x7f1cc0004810 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce013f9f0 con 0x7f1ce006d220 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): authenticate will time out at 1141.180404s 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1cc0004810 msgr2=0x7f1ce013ca30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1cc0004810 0x7f1ce013ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54472/0 (socket says 172.21.15.105:54472) 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3641575187 learned_addr learned my addr 172.21.15.105:0/3641575187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): get_auth_request con 0x7f1ce0136c90 auth_method 0 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient(hunting): handle_auth_done global_id 34434 payload 824 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce006d220 msgr2=0x7f1ce006d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce006d220 0x7f1ce006d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1cc0004810 msgr2=0x7f1ce013ca30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1cc0004810 0x7f1ce013ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 monclient: found mon.noname-a 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce009ffd0 con 0x7f1ce0136c90 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdeffd640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cc801d000 tx=0x7f1cc0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.?.objecter ms_handle_connect 0x7f1ce0136c90 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cc80013d0 con 0x7f1ce0136c90 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: dump: 2024-09-17T16:30:57.940 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8007580 con 0x7f1ce0136c90 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 5 monclient: authenticate success, global_id 34434 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _renew_subs 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce01253a0 con 0x7f1ce0136c90 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cc80ae470 con 0x7f1ce0136c90 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cddffb640 4 set_mon_vals no callback set 2024-09-17T16:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 client.34434.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: dump: 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _renew_subs 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01d1d70 con 0x7f1ce0136c90 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 librados: init done 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 0x7f1ce0138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0051e80 con 0x7f1ce006db80 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 0x7f1ce0138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient: get_auth_request con 0x7f1ce006db80 auth_method 0 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1cc8007110 con 0x7f1ce0136c90 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1cc403e550 0x7f1cc4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_done global_id 34443 payload 824 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ce009faf0 con 0x7f1ce006db80 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8505+0+0 (secure 0 0 0) 0x7f1cc80b5030 con 0x7f1ce0136c90 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.34434.objecter ms_dispatch 0x7f1ce0000ba0 osd_map(110..110 src has 1..110) 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 3 client.34434.objecter handle_osd_map got epochs [110,110] > 0 2024-09-17T16:30:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 3 client.34434.objecter handle_osd_map decoding full epoch 110 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 20 client.34434.objecter dump_active .. 0 homeless 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.34434.objecter ms_handle_connect 0x7f1ce006db80 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.34434.objecter resend_mon_ops 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdf7fe640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 0x7f1ce0138620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000d420 tx=0x7f1cd000d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cd0016020 con 0x7f1ce006db80 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdffff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1cc403e550 0x7f1cc4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: dump: 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.946 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdffff640 10 monclient: get_auth_request con 0x7f1cc403e550 auth_method 0 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1cd0003bd0 con 0x7f1ce006db80 2024-09-17T16:30:57.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 msgr2=0x7f1ce0138620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 0x7f1ce0138620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000d420 tx=0x7f1cd000d8e0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 msgr2=0x7f1ce0138620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce006db80 0x7f1ce0138620 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000d420 tx=0x7f1cd000d8e0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cd7fff640 10 client.34434.objecter ms_handle_connect 0x7f1cc403e550 2024-09-17T16:30:57.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.912+0000 7f1cdffff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1cc403e550 0x7f1cc4040a10 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00a3c60 tx=0x7f1cd8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 0x7f1ce0138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce0137610 con 0x7f1ce0138b60 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 0x7f1ce0138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient: get_auth_request con 0x7f1ce0138b60 auth_method 0 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:57.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 10 monclient(hunting): handle_auth_done global_id 34452 payload 824 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 1 -- 172.21.15.105:0/3641575187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1ce01248b0 con 0x7f1ce0138b60 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 client.34434.objecter ms_handle_connect 0x7f1ce0138b60 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 client.34434.objecter resend_mon_ops 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdf7fe640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 0x7f1ce0138f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0000c10 tx=0x7f1cd0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cd0016040 con 0x7f1ce0138b60 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 monclient: dump: 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:57.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:57.952 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7f1cd0027440 con 0x7f1ce0138b60 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 msgr2=0x7f1ce0138f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 0x7f1ce0138f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0000c10 tx=0x7f1cd0005a90 comp rx=0 tx=0).stop 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 msgr2=0x7f1ce0138f70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cd7fff640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0138b60 0x7f1ce0138f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:57.953 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 48723, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:57.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:57.955 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:57.956 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:57.957 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:57.958 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:57.960 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.962 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.963 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:57.965 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.967 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:57.974 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:57.975 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 librados: watch_flush enter 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 librados: watch_flush exit 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 20 client.34434.objecter shutdown clearing up homeless session... 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 client.34434.objecter successfully canceled tick 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1cc403e550 msgr2=0x7f1cc4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1cc403e550 0x7f1cc4040a10 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00a3c60 tx=0x7f1cd8008040 comp rx=0 tx=0).stop 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 10 monclient: shutdown 2024-09-17T16:30:57.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 msgr2=0x7f1ce0137070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cc801d000 tx=0x7f1cc0005840 comp rx=0 tx=0).stop 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1cdffff640 1 -- 172.21.15.105:0/3641575187 reap_dead start 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 shutdown_connections 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 --2- 172.21.15.105:0/3641575187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ce0136c90 0x7f1ce0137070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 >> 172.21.15.105:0/3641575187 conn(0x7f1ce00aa250 msgr2=0x7f1ce009e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 shutdown_connections 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 -- 172.21.15.105:0/3641575187 wait complete. 2024-09-17T16:30:57.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:57.920+0000 7f1ce5d1c640 1 librados: shutdown 2024-09-17T16:30:57.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:57.979 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-17T16:30:57.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: get_monmap_and_config 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: monmap: 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:58.096325+0000 2024-09-17T16:30:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:58.096325+0000 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:58.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 Processor -- start 2024-09-17T16:30:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 -- start start 2024-09-17T16:30:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: init 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient: _add_conns ranks=[6,8,0,5,3,7,1,10,9,2,4] 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 0x7fc05c0a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): picked mon.noname-d con 0x7fc05c0a2620 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 0x7fc05c137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): picked mon.noname-e con 0x7fc05c0a2f40 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc05c137940 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05c06f2e0 con 0x7fc05c137940 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc05c06e580 con 0x7fc05c0a2620 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc05c0a15e0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc062827640 10 monclient(hunting): authenticate will time out at 1141.362115s 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 0x7fc05c0a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 0x7fc05c137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc060dc4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc05c137940 0x7fc05c13e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37066/0 (socket says 172.21.15.105:37066) 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc060dc4640 1 -- 172.21.15.105:0/3326148265 learned_addr learned my addr 172.21.15.105:0/3326148265 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): get_auth_request con 0x7fc05c0a2620 auth_method 0 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): get_auth_request con 0x7fc05c0a2f40 auth_method 0 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_done global_id 23543 payload 274 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 -- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 msgr2=0x7fc05c0a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 0x7fc05c0a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05affd640 1 -- 172.21.15.105:0/3326148265 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717595597 0 0) 0x7fc05c06f2e0 con 0x7fc05c137940 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 -- 172.21.15.105:0/3326148265 >> v1:172.21.15.105:6789/0 conn(0x7fc05c137940 legacy=0x7fc05c13e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 monclient: found mon.noname-e 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05b7fe640 1 -- 172.21.15.105:0/3326148265 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c0a3c30 con 0x7fc05c0a2f40 2024-09-17T16:30:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05bfff640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 0x7fc05c0a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.092+0000 7fc05affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:30:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 0x7fc05c137400 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc05000f3a0 tx=0x7fc050020c50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cdfc2d94963b8e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 1 -- 172.21.15.105:0/3326148265 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc05000d0d0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: dump: 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.112 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 1 -- 172.21.15.105:0/3326148265 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05000d290 con 0x7fc05c0a2f40 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 1 -- 172.21.15.105:0/3326148265 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc05000d440 con 0x7fc05c0a2f40 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 5 monclient: authenticate success, global_id 23543 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 4 set_mon_vals no callback set 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05affd640 10 monclient: dump: 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.114 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: shutdown 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 msgr2=0x7fc05c137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 0x7fc05c137400 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc05000f3a0 tx=0x7fc050020c50 comp rx=0 tx=0).stop 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- 172.21.15.105:0/3326148265 shutdown_connections 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc05c0a2f40 0x7fc05c137400 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- 172.21.15.105:0/3326148265 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2620 0x7fc05c0a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- 172.21.15.105:0/3326148265 >> 172.21.15.105:0/3326148265 conn(0x7fc05c068ba0 msgr2=0x7fc05c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- 172.21.15.105:0/3326148265 shutdown_connections 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- 172.21.15.105:0/3326148265 wait complete. 2024-09-17T16:30:58.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: monmap: 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.116 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 librados: starting msgr at 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 librados: starting objecter 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 Processor -- start 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- start start 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 librados: setting wanted keys 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 librados: calling monclient init 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: init 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient: _add_conns ranks=[1,6,4,7,5,3,2,0,8] 2024-09-17T16:30:58.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c0a2620 0x7fc05c11e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): picked mon.noname-f con 0x7fc05c0a2620 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): picked mon.noname-d con 0x7fc05c0a2f40 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c137940 0x7fc05c11ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): picked mon.noname-c con 0x7fc05c137940 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c069ad0 con 0x7fc05c0a2620 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc05c069950 con 0x7fc05c137940 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc05c0a1dd0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc062827640 10 monclient(hunting): authenticate will time out at 1141.367329s 2024-09-17T16:30:58.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39722/0 (socket says 172.21.15.105:39722) 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 -- 172.21.15.105:0/856702492 learned_addr learned my addr 172.21.15.105:0/856702492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): get_auth_request con 0x7fc05c0a2f40 auth_method 0 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c137940 0x7fc05c11ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): get_auth_request con 0x7fc05c137940 auth_method 0 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc060dc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient(hunting): handle_auth_done global_id 23550 payload 824 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c137940 msgr2=0x7fc05c11ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c137940 0x7fc05c11ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c0a2620 msgr2=0x7fc05c11e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05bfff640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c0a2620 0x7fc05c11e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c0a2620 0x7fc05c11e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 monclient: found mon.noname-d 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.096+0000 7fc05b7fe640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c0726e0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05b7fe640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc05000f3a0 tx=0x7fc05001d9f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.?.objecter ms_handle_connect 0x7fc05c0a2f40 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0500c0020 con 0x7fc05c0a2f40 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: dump: 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc050003e10 con 0x7fc05c0a2f40 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 5 monclient: authenticate success, global_id 23550 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 monclient: _renew_subs 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c120a80 con 0x7fc05c0a2f40 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc05000bbb0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: dump: 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05a7fc640 4 set_mon_vals no callback set 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 client.23550.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 monclient: _renew_subs 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c13fcf0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 1 librados: init done 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 0x7fc05c13aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc05c051e80 con 0x7fc05c13a6a0 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 0x7fc05c13aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient: get_auth_request con 0x7fc05c13a6a0 auth_method 0 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc0500c3020 con 0x7fc05c0a2f40 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 10 monclient(hunting): handle_auth_done global_id 22684 payload 824 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc05c1201e0 con 0x7fc05c13a6a0 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc04803e580 0x7fc048040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(110..110 src has 1..110) ==== 8505+0+0 (secure 0 0 0) 0x7fc05c0a11a0 con 0x7fc05c0a2f40 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.23550.objecter ms_dispatch 0x7fc05c000ba0 osd_map(110..110 src has 1..110) 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 3 client.23550.objecter handle_osd_map got epochs [110,110] > 0 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc05bfff640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 0x7fc05c13aa80 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc0540107c0 tx=0x7fc054010c80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 3 client.23550.objecter handle_osd_map decoding full epoch 110 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 20 client.23550.objecter dump_active .. 0 homeless 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.23550.objecter ms_handle_connect 0x7fc05c13a6a0 2024-09-17T16:30:58.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.23550.objecter resend_mon_ops 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc05400f040 con 0x7fc05c13a6a0 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: dump: 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.127 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc060dc4640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc04803e580 0x7fc048040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc060dc4640 10 monclient: get_auth_request con 0x7fc04803e580 auth_method 0 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc05401c400 con 0x7fc05c13a6a0 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 msgr2=0x7fc05c13aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 0x7fc05c13aa80 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc0540107c0 tx=0x7fc054010c80 comp rx=0 tx=0).stop 2024-09-17T16:30:58.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 msgr2=0x7fc05c13aa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13a6a0 0x7fc05c13aa80 secure :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc0540107c0 tx=0x7fc054010c80 comp rx=0 tx=0).stop 2024-09-17T16:30:58.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc058ff9640 10 client.23550.objecter ms_handle_connect 0x7fc04803e580 2024-09-17T16:30:58.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.100+0000 7fc060dc4640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc04803e580 0x7fc048040a40 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fc04c007bc0 tx=0x7fc04c007b10 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:58.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 0x7fc05c1d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc05c0a2a00 con 0x7fc05c13afc0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 0x7fc05c1d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient: get_auth_request con 0x7fc05c13afc0 auth_method 0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 10 monclient(hunting): handle_auth_done global_id 22693 payload 824 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 1 -- 172.21.15.105:0/856702492 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc05c06cfa0 con 0x7fc05c13afc0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc05bfff640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 0x7fc05c1d75e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc054009fd0 tx=0x7fc0540046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 client.23550.objecter ms_handle_connect 0x7fc05c13afc0 2024-09-17T16:30:58.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 client.23550.objecter resend_mon_ops 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc05401cda0 con 0x7fc05c13afc0 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 monclient: dump: 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.132 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9276 (secure 0 0 0) 0x7fc05401c930 con 0x7fc05c13afc0 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 msgr2=0x7fc05c1d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 0x7fc05c1d75e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc054009fd0 tx=0x7fc0540046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 msgr2=0x7fc05c1d75e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc058ff9640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc05c13afc0 0x7fc05c1d75e0 secure :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc054009fd0 tx=0x7fc0540046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 205747, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:58.134 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.135 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.136 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:58.137 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.138 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:58.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.140 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:58.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.147 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.149 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:58.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:58.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.154 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.157 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.158 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:58.158 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:58.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 librados: watch_flush enter 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 librados: watch_flush exit 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 20 client.23550.objecter shutdown clearing up homeless session... 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 client.23550.objecter successfully canceled tick 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc04803e580 msgr2=0x7fc048040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc04803e580 0x7fc048040a40 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fc04c007bc0 tx=0x7fc04c007b10 comp rx=0 tx=0).stop 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 10 monclient: shutdown 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 msgr2=0x7fc05c11e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc05000f3a0 tx=0x7fc05001d9f0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc060dc4640 1 -- 172.21.15.105:0/856702492 reap_dead start 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 shutdown_connections 2024-09-17T16:30:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 --2- 172.21.15.105:0/856702492 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc05c0a2f40 0x7fc05c11e7b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 >> 172.21.15.105:0/856702492 conn(0x7fc05c068ba0 msgr2=0x7fc05c0a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 shutdown_connections 2024-09-17T16:30:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 -- 172.21.15.105:0/856702492 wait complete. 2024-09-17T16:30:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.108+0000 7fc062827640 1 librados: shutdown 2024-09-17T16:30:58.161 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:58.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-17T16:30:58.161 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:30:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.272+0000 7fe39ddd2640 10 monclient: get_monmap_and_config 2024-09-17T16:30:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.272+0000 7fe39ddd2640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient: monmap: 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:58.279794+0000 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:58.279794+0000 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:58.283 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 Processor -- start 2024-09-17T16:30:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 -- start start 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient: init 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient: _add_conns ranks=[4,10,8,0,2,9,7,1,6,5,3] 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-c con 0x7fe3980a2de0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3980a31c0 0x7fe3980a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-f con 0x7fe3980a31c0 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 0x7fe3981399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-e con 0x7fe3981395f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe3980a0160 con 0x7fe3980a31c0 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe39809f8e0 con 0x7fe3980a2de0 2024-09-17T16:30:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe398074a20 con 0x7fe3981395f0 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe39ddd2640 10 monclient(hunting): authenticate will time out at 1141.545483s 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 0x7fe3981399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52066/0 (socket says 172.21.15.105:52066) 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 1 -- 172.21.15.105:0/3170042747 learned_addr learned my addr 172.21.15.105:0/3170042747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): get_auth_request con 0x7fe3980a2de0 auth_method 0 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): get_auth_request con 0x7fe3981395f0 auth_method 0 2024-09-17T16:30:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe396ffd640 1 --2- 172.21.15.105:0/3170042747 >> v2:172.21.15.160:3300/0 conn(0x7fe3980a31c0 0x7fe3980a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe396ffd640 10 monclient(hunting): get_auth_request con 0x7fe3980a31c0 auth_method 0 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe396ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe396ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe396ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient(hunting): handle_auth_done global_id 23552 payload 274 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 -- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 msgr2=0x7fe3980a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 -- 172.21.15.105:0/3170042747 >> v2:172.21.15.160:3300/0 conn(0x7fe3980a31c0 msgr2=0x7fe3980a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 --2- 172.21.15.105:0/3170042747 >> v2:172.21.15.160:3300/0 conn(0x7fe3980a31c0 0x7fe3980a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 monclient: found mon.noname-e 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe397fff640 1 -- 172.21.15.105:0/3170042747 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398074c80 con 0x7fe3981395f0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.276+0000 7fe3977fe640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe397fff640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 0x7fe3981399d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b703dab9e00d30af server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 1 -- 172.21.15.105:0/3170042747 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3900ac070 con 0x7fe3981395f0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: dump: 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 1 -- 172.21.15.105:0/3170042747 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3900a68f0 con 0x7fe3981395f0 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 1 -- 172.21.15.105:0/3170042747 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3900a7460 con 0x7fe3981395f0 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3967fc640 10 monclient: dump: 2024-09-17T16:30:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 5 monclient: authenticate success, global_id 23552 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 4 set_mon_vals no callback set 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: shutdown 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 msgr2=0x7fe3981399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 0x7fe3981399d0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c830 comp rx=0 tx=0).stop 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/3170042747 shutdown_connections 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/3170042747 >> v2:172.21.15.160:3300/0 conn(0x7fe3980a31c0 0x7fe3980a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe3981395f0 0x7fe3981399d0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/3170042747 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3980a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/3170042747 >> 172.21.15.105:0/3170042747 conn(0x7fe3980a83e0 msgr2=0x7fe3980a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/3170042747 shutdown_connections 2024-09-17T16:30:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/3170042747 wait complete. 2024-09-17T16:30:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: monmap: 2024-09-17T16:30:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 librados: starting msgr at 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 librados: starting objecter 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 Processor -- start 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- start start 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 librados: setting wanted keys 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 librados: calling monclient init 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: init 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _add_conns ranks=[4,5,7,2,0,3,6,8,1] 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3981d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-c con 0x7fe3980a2de0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe3980a31c0 0x7fe3981d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-h con 0x7fe3980a31c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe3981395f0 0x7fe39812d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): picked mon.noname-i con 0x7fe3981395f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe398143460 con 0x7fe3980a2de0 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe3981432e0 con 0x7fe3980a31c0 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe3981435e0 con 0x7fe3981395f0 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient(hunting): authenticate will time out at 1141.550591s 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3981d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3981d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52076/0 (socket says 172.21.15.105:52076) 2024-09-17T16:30:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 -- 172.21.15.105:0/1948421920 learned_addr learned my addr 172.21.15.105:0/1948421920 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): get_auth_request con 0x7fe3980a2de0 auth_method 0 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient(hunting): handle_auth_done global_id 22711 payload 824 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe3981395f0 msgr2=0x7fe39812d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe396ffd640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe3980a31c0 0x7fe3981d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe3981395f0 0x7fe39812d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe3980a31c0 msgr2=0x7fe3981d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe3980a31c0 0x7fe3981d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 monclient: found mon.noname-c 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398130b40 con 0x7fe3980a2de0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe3977fe640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3981d1800 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ea00 tx=0x7fe38800eec0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 client.?.objecter ms_handle_connect 0x7fe3980a2de0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe388017070 con 0x7fe3980a2de0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: dump: 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe388013470 con 0x7fe3980a2de0 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 5 monclient: authenticate success, global_id 22711 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _renew_subs 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39812f620 con 0x7fe3980a2de0 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe388013a90 con 0x7fe3980a2de0 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe37ffff640 10 monclient: dump: 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.309 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 client.22711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _renew_subs 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe398130860 con 0x7fe3980a2de0 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe39ddd2640 1 librados: init done 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.280+0000 7fe395ffb640 4 set_mon_vals no callback set 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe39ddd2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 0x7fe39812ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe398051e80 con 0x7fe39812d9e0 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 0x7fe39812ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient: get_auth_request con 0x7fe39812d9e0 auth_method 0 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 10 monclient(hunting): handle_auth_done global_id 23559 payload 824 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe38801a070 con 0x7fe3980a2de0 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3980a87c0 con 0x7fe39812d9e0 2024-09-17T16:30:58.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe38403e580 0x7fe384040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe396ffd640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 0x7fe39812ddc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c00e9a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 client.22711.objecter ms_handle_connect 0x7fe39812d9e0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 client.22711.objecter resend_mon_ops 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(110..110 src has 1..110) ==== 8505+0+0 (secure 0 0 0) 0x7fe38800f040 con 0x7fe3980a2de0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 client.22711.objecter ms_dispatch 0x7fe398000ba0 osd_map(110..110 src has 1..110) 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 3 client.22711.objecter handle_osd_map got epochs [110,110] > 0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 3 client.22711.objecter handle_osd_map decoding full epoch 110 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 20 client.22711.objecter dump_active .. 0 homeless 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe38c017070 con 0x7fe39812d9e0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 monclient: dump: 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.312 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.313 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe397fff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe38403e580 0x7fe384040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe397fff640 10 monclient: get_auth_request con 0x7fe38403e580 auth_method 0 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe38c01c450 con 0x7fe39812d9e0 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 msgr2=0x7fe39812ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 0x7fe39812ddc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c00e9a0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 msgr2=0x7fe39812ddc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812d9e0 0x7fe39812ddc0 secure :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c00e9a0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe37ffff640 10 client.22711.objecter ms_handle_connect 0x7fe38403e580 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.284+0000 7fe397fff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe38403e580 0x7fe384040a40 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ae000 tx=0x7fe390004ac0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe39ddd2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:58.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 0x7fe39813b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe39ddd2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe398130580 con 0x7fe39812e300 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 0x7fe39813b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient: get_auth_request con 0x7fe39812e300 auth_method 0 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 10 monclient(hunting): handle_auth_done global_id 23568 payload 824 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.288+0000 7fe396ffd640 1 -- 172.21.15.105:0/1948421920 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe398130f40 con 0x7fe39812e300 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe396ffd640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 0x7fe39813b890 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c0046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 client.22711.objecter ms_handle_connect 0x7fe39812e300 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 client.22711.objecter resend_mon_ops 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe38c02a020 con 0x7fe39812e300 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 monclient: dump: 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.316 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7fe38c013a30 con 0x7fe39812e300 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 msgr2=0x7fe39813b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 0x7fe39813b890 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 msgr2=0x7fe39813b890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe37ffff640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39812e300 0x7fe39813b890 secure :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e4e0 tx=0x7fe38c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 205931, 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:58.318 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.319 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:58.320 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:58.321 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.322 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.323 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.325 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.326 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.328 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.330 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:58.335 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.338 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.339 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.340 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:58.341 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:58.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 10 librados: watch_flush enter 2024-09-17T16:30:58.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 10 librados: watch_flush exit 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 20 client.22711.objecter shutdown clearing up homeless session... 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 10 client.22711.objecter successfully canceled tick 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe38403e580 msgr2=0x7fe384040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe38403e580 0x7fe384040a40 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ae000 tx=0x7fe390004ac0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 10 monclient: shutdown 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 msgr2=0x7fe3981d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 --2- 172.21.15.105:0/1948421920 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe3980a2de0 0x7fe3981d1800 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fe38800ea00 tx=0x7fe38800eec0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe397fff640 1 -- 172.21.15.105:0/1948421920 reap_dead start 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe397fff640 1 -- 172.21.15.105:0/1948421920 reap_dead start 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 shutdown_connections 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 >> 172.21.15.105:0/1948421920 conn(0x7fe3980a83e0 msgr2=0x7fe3980a62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 shutdown_connections 2024-09-17T16:30:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 -- 172.21.15.105:0/1948421920 wait complete. 2024-09-17T16:30:58.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.292+0000 7fe39ddd2640 1 librados: shutdown 2024-09-17T16:30:58.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:58.344 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-17T16:30:58.344 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:30:58.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: get_monmap_and_config 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: monmap: 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:58.456880+0000 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:58.456880+0000 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:58.459 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:58.460 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 Processor -- start 2024-09-17T16:30:58.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 -- start start 2024-09-17T16:30:58.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: init 2024-09-17T16:30:58.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient: _add_conns ranks=[4,8,10,1,7,5,0,6,3,9,2] 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): picked mon.noname-c con 0x7f17040a7060 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): picked mon.noname-e con 0x7f17040a5da0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 0x7f1704139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): picked mon.noname-f con 0x7f1704139a00 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f17040a66e0 con 0x7f1704139a00 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f17040a7470 con 0x7f17040a7060 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f170406d260 con 0x7f17040a5da0 2024-09-17T16:30:58.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17092db640 10 monclient(hunting): authenticate will time out at 1141.722596s 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59536/0 (socket says 172.21.15.105:59536) 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 -- 172.21.15.105:0/326575422 learned_addr learned my addr 172.21.15.105:0/326575422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52086/0 (socket says 172.21.15.105:52086) 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): get_auth_request con 0x7f17040a5da0 auth_method 0 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): get_auth_request con 0x7f17040a7060 auth_method 0 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17037fe640 1 --2- 172.21.15.105:0/326575422 >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 0x7f1704139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient(hunting): handle_auth_done global_id 23561 payload 274 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 -- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 msgr2=0x7f17040a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 -- 172.21.15.105:0/326575422 >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 msgr2=0x7f1704139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 --2- 172.21.15.105:0/326575422 >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 0x7f1704139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 monclient: found mon.noname-e 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f16fbfff640 1 -- 172.21.15.105:0/326575422 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040a0e20 con 0x7f17040a5da0 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17037fe640 10 monclient: get_auth_request con 0x7f1704139a00 auth_method 0 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f1702ffd640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:58.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.452+0000 7f17037fe640 1 --2- 172.21.15.105:0/326575422 >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 0x7f1704139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:58.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f16fbfff640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec01c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ce12f90c46f08556 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 1 -- 172.21.15.105:0/326575422 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec0b0070 con 0x7f17040a5da0 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: dump: 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.476 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 5 monclient: authenticate success, global_id 23561 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 1 -- 172.21.15.105:0/326575422 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16ec0aa810 con 0x7f17040a5da0 2024-09-17T16:30:58.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 4 set_mon_vals no callback set 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 1 -- 172.21.15.105:0/326575422 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec0a9300 con 0x7f17040a5da0 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17027fc640 10 monclient: dump: 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.478 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 10 monclient: shutdown 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 msgr2=0x7f17040a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec01c400 comp rx=0 tx=0).stop 2024-09-17T16:30:58.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- 172.21.15.105:0/326575422 shutdown_connections 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 --2- 172.21.15.105:0/326575422 >> v2:172.21.15.160:3300/0 conn(0x7f1704139a00 0x7f1704139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f17040a61a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 --2- 172.21.15.105:0/326575422 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f17040a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- 172.21.15.105:0/326575422 >> 172.21.15.105:0/326575422 conn(0x7f17040aa450 msgr2=0x7f17040a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- 172.21.15.105:0/326575422 shutdown_connections 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- 172.21.15.105:0/326575422 wait complete. 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 10 monclient: monmap: 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.480 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 librados: starting msgr at 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 librados: starting objecter 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 Processor -- start 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 -- start start 2024-09-17T16:30:58.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 librados: setting wanted keys 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 1 librados: calling monclient init 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.456+0000 7f17092db640 10 monclient: init 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _add_conns ranks=[8,4,0,3,2,7,5,6,1] 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f170413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): picked mon.noname-e con 0x7f17040a5da0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f170413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): picked mon.noname-c con 0x7f17040a7060 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1704139a00 0x7f17041d1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): picked mon.noname-a con 0x7f1704139a00 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f170406db60 con 0x7f1704139a00 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f170413a7c0 con 0x7f17040a7060 2024-09-17T16:30:58.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1704143470 con 0x7f17040a5da0 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): authenticate will time out at 1141.728134s 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f170413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f170413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f170413ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59540/0 (socket says 172.21.15.105:59540) 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f170413f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52096/0 (socket says 172.21.15.105:52096) 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 -- 172.21.15.105:0/2969571386 learned_addr learned my addr 172.21.15.105:0/2969571386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): get_auth_request con 0x7f17040a5da0 auth_method 0 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1704139a00 0x7f17041d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): get_auth_request con 0x7f17040a7060 auth_method 0 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): get_auth_request con 0x7f1704139a00 auth_method 0 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient(hunting): handle_auth_done global_id 23570 payload 824 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 msgr2=0x7f170413f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f170413f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1704139a00 msgr2=0x7f17041d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1704139a00 0x7f17041d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 monclient: found mon.noname-e 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040a6180 con 0x7f17040a5da0 2024-09-17T16:30:58.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17040a7060 0x7f170413f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1704139a00 0x7f17041d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1702ffd640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f170413ed60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9b0 tx=0x7f16f400ee70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 client.?.objecter ms_handle_connect 0x7f17040a5da0 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16f4017070 con 0x7f17040a5da0 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: dump: 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.486 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f400ce70 con 0x7f17040a5da0 2024-09-17T16:30:58.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16f40138e0 con 0x7f17040a5da0 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: dump: 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.488 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f1701ffb640 4 set_mon_vals no callback set 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 5 monclient: authenticate success, global_id 23570 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _renew_subs 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17040a0700 con 0x7f17040a5da0 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 client.23570.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:58.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _renew_subs 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f170413d270 con 0x7f17040a5da0 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 librados: init done 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 0x7f17041d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1704051e80 con 0x7f17041d19b0 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 0x7f17041d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient: get_auth_request con 0x7f17041d19b0 auth_method 0 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f16f401a070 con 0x7f17040a5da0 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 10 monclient(hunting): handle_auth_done global_id 23579 payload 824 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17041d22d0 con 0x7f17041d19b0 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f16e403e550 0x7f16e4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(110..110 src has 1..110) ==== 8505+0+0 (secure 0 0 0) 0x7f16f400f040 con 0x7f17040a5da0 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 client.23570.objecter ms_dispatch 0x7f1704000ba0 osd_map(110..110 src has 1..110) 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 3 client.23570.objecter handle_osd_map got epochs [110,110] > 0 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 3 client.23570.objecter handle_osd_map decoding full epoch 110 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fbfff640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 0x7f17041d1d90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f17040a1910 tx=0x7f16ec0191a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 20 client.23570.objecter dump_active .. 0 homeless 2024-09-17T16:30:58.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 client.23570.objecter ms_handle_connect 0x7f17041d19b0 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 client.23570.objecter resend_mon_ops 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec0ba030 con 0x7f17041d19b0 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: dump: 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.492 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f16e403e550 0x7f16e4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f17037fe640 10 monclient: get_auth_request con 0x7f16e403e550 auth_method 0 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f16ec0aa750 con 0x7f17041d19b0 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 msgr2=0x7f17041d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 0x7f17041d1d90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f17040a1910 tx=0x7f16ec0191a0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 msgr2=0x7f17041d1d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.460+0000 7f16fb7fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17041d19b0 0x7f17041d1d90 secure :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f17040a1910 tx=0x7f16ec0191a0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.464+0000 7f16fb7fe640 10 client.23570.objecter ms_handle_connect 0x7f16e403e550 2024-09-17T16:30:58.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.464+0000 7f17037fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f16e403e550 0x7f16e4040a10 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f170413c520 tx=0x7f16fc007780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 0x7f1704138090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f17040a5860 con 0x7f1704137c80 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 0x7f1704138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient: get_auth_request con 0x7f1704137c80 auth_method 0 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 10 monclient(hunting): handle_auth_done global_id 23588 payload 824 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 1 -- 172.21.15.105:0/2969571386 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f17040a4320 con 0x7f1704137c80 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 client.23570.objecter ms_handle_connect 0x7f1704137c80 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fbfff640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 0x7f1704138090 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16ec01c440 tx=0x7f16ec0a8000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 client.23570.objecter resend_mon_ops 2024-09-17T16:30:58.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16ec0ba040 con 0x7f1704137c80 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 monclient: dump: 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.498 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7f16ec0c0ae0 con 0x7f1704137c80 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 msgr2=0x7f1704138090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 0x7f1704138090 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16ec01c440 tx=0x7f16ec0a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 msgr2=0x7f1704138090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f16fb7fe640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1704137c80 0x7f1704138090 secure :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16ec01c440 tx=0x7f16ec0a8000 comp rx=0 tx=0).stop 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 206111, 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:58.500 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.501 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.502 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:58.503 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.504 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:58.505 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.511 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:58.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.518 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.519 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.521 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:58.522 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 librados: watch_flush enter 2024-09-17T16:30:58.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 librados: watch_flush exit 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 20 client.23570.objecter shutdown clearing up homeless session... 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 client.23570.objecter successfully canceled tick 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f16e403e550 msgr2=0x7f16e4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f16e403e550 0x7f16e4040a10 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f170413c520 tx=0x7f16fc007780 comp rx=0 tx=0).stop 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 10 monclient: shutdown 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 msgr2=0x7f170413ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 --2- 172.21.15.105:0/2969571386 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f17040a5da0 0x7f170413ed60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9b0 tx=0x7f16f400ee70 comp rx=0 tx=0).stop 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17037fe640 1 -- 172.21.15.105:0/2969571386 reap_dead start 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17037fe640 1 -- 172.21.15.105:0/2969571386 reap_dead start 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 shutdown_connections 2024-09-17T16:30:58.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 >> 172.21.15.105:0/2969571386 conn(0x7f17040aa450 msgr2=0x7f17040a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 shutdown_connections 2024-09-17T16:30:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 -- 172.21.15.105:0/2969571386 wait complete. 2024-09-17T16:30:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.468+0000 7f17092db640 1 librados: shutdown 2024-09-17T16:30:58.525 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:58.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-17T16:30:58.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.644+0000 7f639bf9f640 10 monclient: get_monmap_and_config 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.644+0000 7f639bf9f640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.644+0000 7f639bf9f640 10 monclient: monmap: 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:58.651152+0000 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:58.651152+0000 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:58.654 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 Processor -- start 2024-09-17T16:30:58.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 -- start start 2024-09-17T16:30:58.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient: init 2024-09-17T16:30:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient: _add_conns ranks=[10,3,5,7,0,2,9,8,1,6,4] 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f63940a4750 0x7f63940a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-f con 0x7f63940a4750 addr v2:172.21.15.160:3300/0 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f639406c2e0 0x7f63940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-g con 0x7f639406c2e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 0x7f6394138980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-h con 0x7f6394138520 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f63940a55b0 con 0x7f63940a4750 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f63940a6610 con 0x7f639406c2e0 2024-09-17T16:30:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f63940a61d0 con 0x7f6394138520 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient(hunting): authenticate will time out at 1141.916659s 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f63940a4750 0x7f63940a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 0x7f6394138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 0x7f6394138980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:41528/0 (socket says 172.21.15.105:41528) 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f63940a4750 0x7f63940a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:52434/0 (socket says 172.21.15.105:52434) 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 1 -- 172.21.15.105:0/1757850800 learned_addr learned my addr 172.21.15.105:0/1757850800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): get_auth_request con 0x7f6394138520 auth_method 0 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f639406c2e0 0x7f63940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): get_auth_request con 0x7f63940a4750 auth_method 0 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): get_auth_request con 0x7f639406c2e0 auth_method 0 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639a53c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient(hunting): handle_auth_done global_id 24643 payload 274 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 -- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 msgr2=0x7f6394138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 0x7f6394138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 -- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f639406c2e0 msgr2=0x7f63940a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f639406c2e0 0x7f63940a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 monclient: found mon.noname-f 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 -- 172.21.15.105:0/1757850800 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63940a5f90 con 0x7f63940a4750 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6399d3b640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63940a4750 0x7f63940a4b30 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f63900097c0 tx=0x7f639001c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c84c2fd6e28d727 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 1 -- 172.21.15.105:0/1757850800 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63900b0070 con 0x7f63940a4750 2024-09-17T16:30:58.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: dump: 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 1 -- 172.21.15.105:0/1757850800 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900aa810 con 0x7f63940a4750 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 5 monclient: authenticate success, global_id 24643 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:58.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 1 -- 172.21.15.105:0/1757850800 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63900a9300 con 0x7f63940a4750 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f639bf9f640 4 set_mon_vals no callback set 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.648+0000 7f6398d39640 10 monclient: dump: 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.670 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: shutdown 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63940a4750 msgr2=0x7f63940a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63940a4750 0x7f63940a4b30 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f63900097c0 tx=0x7f639001c360 comp rx=0 tx=0).stop 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/1757850800 shutdown_connections 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6394138520 0x7f6394138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f639406c2e0 0x7f63940a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- 172.21.15.105:0/1757850800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63940a4750 0x7f63940a4b30 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/1757850800 >> 172.21.15.105:0/1757850800 conn(0x7f63940a94d0 msgr2=0x7f63940a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/1757850800 shutdown_connections 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/1757850800 wait complete. 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: monmap: 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:58.672 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 librados: starting msgr at 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 librados: starting objecter 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 Processor -- start 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- start start 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 librados: setting wanted keys 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 librados: calling monclient init 2024-09-17T16:30:58.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: init 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _add_conns ranks=[6,7,1,5,8,2,4,3,0] 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 0x7f6394121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-d con 0x7f639406c2e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63940a4750 0x7f63941d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-i con 0x7f63940a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6394138520 0x7f639411f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): picked mon.noname-f con 0x7f6394138520 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6394143370 con 0x7f6394138520 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f639409ede0 con 0x7f639406c2e0 2024-09-17T16:30:58.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f63941434f0 con 0x7f63940a4750 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): authenticate will time out at 1141.921964s 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 0x7f6394121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 0x7f6394121480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39758/0 (socket says 172.21.15.105:39758) 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 -- 172.21.15.105:0/3518291152 learned_addr learned my addr 172.21.15.105:0/3518291152 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): get_auth_request con 0x7f639406c2e0 auth_method 0 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63940a4750 0x7f63941d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639a53c640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6394138520 0x7f639411f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): get_auth_request con 0x7f63940a4750 auth_method 0 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient(hunting): handle_auth_done global_id 23595 payload 824 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63940a4750 msgr2=0x7f63941d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63940a4750 0x7f63941d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6394138520 msgr2=0x7f639411f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6394138520 0x7f639411f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 monclient: found mon.noname-d 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6394122c80 con 0x7f639406c2e0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639a53c640 10 monclient: get_auth_request con 0x7f6394138520 auth_method 0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639a53c640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6394138520 0x7f639411f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f6399d3b640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 0x7f6394121480 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f639001cc00 tx=0x7f6390019100 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 client.?.objecter ms_handle_connect 0x7f639406c2e0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63900b0040 con 0x7f639406c2e0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: dump: 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.679 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900b9070 con 0x7f639406c2e0 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 5 monclient: authenticate success, global_id 23595 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _renew_subs 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639413fc90 con 0x7f639406c2e0 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63900044b0 con 0x7f639406c2e0 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 10 monclient: dump: 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.681 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 client.23595.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638bfff640 4 set_mon_vals no callback set 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _renew_subs 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f63940a6bf0 con 0x7f639406c2e0 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 librados: init done 2024-09-17T16:30:58.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 0x7f639411f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6394051e80 con 0x7f63941d1f80 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 0x7f639411f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f63900aac80 con 0x7f639406c2e0 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient: get_auth_request con 0x7f63941d1f80 auth_method 0 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.652+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f637803e5b0 0x7f6378040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639953a640 10 monclient(hunting): handle_auth_done global_id 24652 payload 824 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639953a640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6394138980 con 0x7f63941d1f80 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(111..111 src has 1..111) ==== 7879+0+0 (secure 0 0 0) 0x7f63900ea150 con 0x7f639406c2e0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 client.23595.objecter ms_dispatch 0x7f6394000ba0 osd_map(111..111 src has 1..111) 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 3 client.23595.objecter handle_osd_map got epochs [111,111] > 0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 3 client.23595.objecter handle_osd_map decoding full epoch 111 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639a53c640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f637803e5b0 0x7f6378040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 20 client.23595.objecter dump_active .. 0 homeless 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639a53c640 10 monclient: get_auth_request con 0x7f637803e5b0 auth_method 0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 client.23595.objecter ms_handle_connect 0x7f63941d1f80 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 client.23595.objecter resend_mon_ops 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639953a640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 0x7f639411f8d0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f639413fa30 tx=0x7f6384011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6384015ae0 con 0x7f63941d1f80 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:58.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 monclient: dump: 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.685 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 client.23595.objecter ms_handle_connect 0x7f637803e5b0 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f639a53c640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f637803e5b0 0x7f6378040a70 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7f638c009440 tx=0x7f638c00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6384015670 con 0x7f63941d1f80 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 msgr2=0x7f639411f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 0x7f639411f8d0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f639413fa30 tx=0x7f6384011990 comp rx=0 tx=0).stop 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 msgr2=0x7f639411f8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.656+0000 7f638a7fc640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63941d1f80 0x7f639411f8d0 secure :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f639413fa30 tx=0x7f6384011990 comp rx=0 tx=0).stop 2024-09-17T16:30:58.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639bf9f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:58.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639bf9f640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 0x7f639413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639bf9f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6394120220 con 0x7f639411fe10 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 0x7f639413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient: get_auth_request con 0x7f639411fe10 auth_method 0 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 10 monclient(hunting): handle_auth_done global_id 24661 payload 824 2024-09-17T16:30:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 1 -- 172.21.15.105:0/3518291152 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6394122a00 con 0x7f639411fe10 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f639953a640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 0x7f639413a8a0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f639413fa30 tx=0x7f6384004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f638a7fc640 10 client.23595.objecter ms_handle_connect 0x7f639411fe10 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.660+0000 7f638a7fc640 10 client.23595.objecter resend_mon_ops 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6384022bf0 con 0x7f639411fe10 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 10 monclient: dump: 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.690 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9276 (secure 0 0 0) 0x7f6384022780 con 0x7f639411fe10 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 msgr2=0x7f639413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 0x7f639413a8a0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f639413fa30 tx=0x7f6384004740 comp rx=0 tx=0).stop 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 msgr2=0x7f639413a8a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f638a7fc640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f639411fe10 0x7f639413a8a0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 206264, 2024-09-17T16:30:58.692 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.693 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.694 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:58.696 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.697 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:58.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.705 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.707 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:58.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.712 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.713 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:58.714 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 10 librados: watch_flush enter 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 10 librados: watch_flush exit 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 20 client.23595.objecter shutdown clearing up homeless session... 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 10 client.23595.objecter successfully canceled tick 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f637803e5b0 msgr2=0x7f6378040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f637803e5b0 0x7f6378040a70 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7f638c009440 tx=0x7f638c00b040 comp rx=0 tx=0).stop 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 10 monclient: shutdown 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 msgr2=0x7f6394121480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 --2- 172.21.15.105:0/3518291152 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f639406c2e0 0x7f6394121480 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f639001cc00 tx=0x7f6390019100 comp rx=0 tx=0).stop 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639a53c640 1 -- 172.21.15.105:0/3518291152 reap_dead start 2024-09-17T16:30:58.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639a53c640 1 -- 172.21.15.105:0/3518291152 reap_dead start 2024-09-17T16:30:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 shutdown_connections 2024-09-17T16:30:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 >> 172.21.15.105:0/3518291152 conn(0x7f63940a94d0 msgr2=0x7f63940a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 shutdown_connections 2024-09-17T16:30:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 -- 172.21.15.105:0/3518291152 wait complete. 2024-09-17T16:30:58.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.664+0000 7f639bf9f640 1 librados: shutdown 2024-09-17T16:30:58.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:58.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-17T16:30:58.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: get_monmap_and_config 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: monmap: 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:58.833283+0000 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:58.833283+0000 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:58.836 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 Processor -- start 2024-09-17T16:30:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 -- start start 2024-09-17T16:30:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: init 2024-09-17T16:30:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient: _add_conns ranks=[0,1,3,10,8,9,2,4,7,5,6] 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7ff00a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7ff00a4910 addr v1:172.21.15.160:6789/0 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7ff0142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-g con 0x7f7ff00a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff006ebc0 con 0x7f7ff00a68e0 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff00a3690 con 0x7f7ff00a4910 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00a3fc0 con 0x7f7ff00a73d0 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7ff5844640 10 monclient(hunting): authenticate will time out at 1142.099116s 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7feeffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7ff00a68e0 0x7f7ff00a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37082/0 (socket says 172.21.15.105:37082) 2024-09-17T16:30:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7feeffd640 1 -- 172.21.15.105:0/1970932378 learned_addr learned my addr 172.21.15.105:0/1970932378 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7ff0142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request con 0x7f7ff00a73d0 auth_method 0 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1669532950 0 0) 0x7f7ff006ebc0 con 0x7f7ff00a68e0 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient(hunting): _negotiate 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient(hunting): my global_id is 34488 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd8003530 con 0x7f7ff00a68e0 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2564486214 0 0) 0x7f7fd8003530 con 0x7f7ff00a68e0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 msgr2=0x7f7ff0142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7ff0142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 >> v1:172.21.15.160:6789/0 conn(0x7f7ff00a4910 legacy=0x7f7ff013eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.828+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff00a3970 con 0x7f7ff00a68e0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f7fe8001cf0 con 0x7f7ff00a68e0 2024-09-17T16:30:58.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 5 monclient: authenticate success, global_id 34488 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: dump: 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.851 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 >> v1:172.21.15.105:6789/0 conn(0x7f7ff00a68e0 legacy=0x7f7ff00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: _add_conns ranks=[2,8,3,7,5,6,4,1,0] 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fd8004240 0x7f7fd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): picked mon.b con 0x7f7fd8004240 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7fd8004da0 0x7f7fd8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): picked mon.e con 0x7f7fd8004da0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7fd800bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): picked mon.g con 0x7f7ff00a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7feeffd640 1 -- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fd8004240 msgr2=0x7f7fd8004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00a3fc0 con 0x7f7fd8004240 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7feeffd640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fd8004240 0x7f7fd8004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd800c3e0 con 0x7f7ff00a73d0 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f7fd800c590 con 0x7f7fd8004da0 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:30:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7fd8004da0 0x7f7fd8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): get_auth_request con 0x7f7fd8004da0 auth_method 0 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7fd800bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request con 0x7f7ff00a73d0 auth_method 0 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient(hunting): handle_auth_done global_id 34488 payload 293 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 -- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 msgr2=0x7f7fd800bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff00a73d0 0x7f7fd800bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 -- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fd8004240 msgr2=0x7f7fd8004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7fd8004240 0x7f7fd8004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 monclient: found mon.e 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 -- 172.21.15.105:0/1970932378 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd8003530 con 0x7f7fd8004da0 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7fd8004da0 0x7f7fd8005220 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4001790 tx=0x7f7fe401c3d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d4de695658352158 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe40013e0 con 0x7f7fd8004da0 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: dump: 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4003600 con 0x7f7fd8004da0 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 1 -- 172.21.15.105:0/1970932378 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe4003c20 con 0x7f7fd8004da0 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 4 set_mon_vals no callback set 2024-09-17T16:30:58.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fedffb640 10 monclient: dump: 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 10 monclient: shutdown 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 -- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7fd8004da0 msgr2=0x7f7fd8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 --2- 172.21.15.105:0/1970932378 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7fd8004da0 0x7f7fd8005220 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4001790 tx=0x7f7fe401c3d0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7fef7fe640 1 -- 172.21.15.105:0/1970932378 reap_dead start 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 -- 172.21.15.105:0/1970932378 shutdown_connections 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 -- 172.21.15.105:0/1970932378 >> 172.21.15.105:0/1970932378 conn(0x7f7ff0068ba0 msgr2=0x7f7ff0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 -- 172.21.15.105:0/1970932378 shutdown_connections 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.832+0000 7f7ff5844640 1 -- 172.21.15.105:0/1970932378 wait complete. 2024-09-17T16:30:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: build_initial_monmap 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: monmap: 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 librados: starting msgr at 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 librados: starting objecter 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 Processor -- start 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- start start 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 librados: setting wanted keys 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 librados: calling monclient init 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: init 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _add_conns ranks=[7,3,6,1,5,4,8,0,2] 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7fd8004da0 0x7f7ff0137230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-i con 0x7f7fd8004da0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff0137770 0x7f7ff013c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-g con 0x7f7ff0137770 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 0x7f7ff013cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): picked mon.noname-d con 0x7f7ff0137b70 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff0142c40 con 0x7f7ff0137770 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f7ff006b5b0 con 0x7f7ff0137b70 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f7ff0142dc0 con 0x7f7fd8004da0 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): authenticate will time out at 1142.105215s 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 0x7f7ff013cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 0x7f7ff013cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39764/0 (socket says 172.21.15.105:39764) 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 learned_addr learned my addr 172.21.15.105:0/2776304745 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request con 0x7f7ff0137b70 auth_method 0 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff0137770 0x7f7ff013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 10 monclient(hunting): get_auth_request con 0x7f7ff0137770 auth_method 0 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7fd8004da0 0x7f7ff0137230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient(hunting): handle_auth_done global_id 23604 payload 824 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7fd8004da0 msgr2=0x7f7ff0137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7fd8004da0 0x7f7ff0137230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff0137770 msgr2=0x7f7ff013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff0137770 0x7f7ff013c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 monclient: found mon.noname-d 2024-09-17T16:30:58.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff0122c60 con 0x7f7ff0137b70 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient: get_auth_request con 0x7f7fd8004da0 auth_method 0 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7fd8004da0 0x7f7ff0137230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff0137770 0x7f7ff013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 0x7f7ff013cc90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00acf60 tx=0x7f7fe0011920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 client.?.objecter ms_handle_connect 0x7f7ff0137b70 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe0019020 con 0x7f7ff0137b70 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: dump: 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.865 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.866 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: _finish_auth 0 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 5 monclient: authenticate success, global_id 23604 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0004300 con 0x7f7ff0137b70 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _renew_subs 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff013ff30 con 0x7f7ff0137b70 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe00044b0 con 0x7f7ff0137b70 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 monclient: dump: 2024-09-17T16:30:58.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.868 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 client.23604.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _renew_subs 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fed7fa640 4 set_mon_vals no callback set 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff013ed20 con 0x7f7ff0137b70 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 librados: init done 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 0x7f7ff013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff0051e80 con 0x7f7ff013d1d0 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7fe001c020 con 0x7f7ff0137b70 2024-09-17T16:30:58.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 0x7f7ff013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient: get_auth_request con 0x7f7ff013d1d0 auth_method 0 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7fdc03e480 0x7f7fdc040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(111..111 src has 1..111) ==== 7879+0+0 (secure 0 0 0) 0x7f7fe0050770 con 0x7f7ff0137b70 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 10 client.23604.objecter ms_dispatch 0x7f7ff0000ba0 osd_map(111..111 src has 1..111) 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 3 client.23604.objecter handle_osd_map got epochs [111,111] > 0 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 3 client.23604.objecter handle_osd_map decoding full epoch 111 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fd7fff640 20 client.23604.objecter dump_active .. 0 homeless 2024-09-17T16:30:58.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7fdc03e480 0x7f7fdc040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7fee7fc640 10 monclient: get_auth_request con 0x7f7fdc03e480 auth_method 0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 10 monclient(hunting): handle_auth_done global_id 19164 payload 824 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.836+0000 7f7feeffd640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ff006dd20 con 0x7f7ff013d1d0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 client.23604.objecter ms_handle_connect 0x7f7ff013d1d0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 client.23604.objecter resend_mon_ops 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 0x7f7ff013d5b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe800def0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 client.23604.objecter ms_handle_connect 0x7f7fdc03e480 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fee7fc640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7fdc03e480 0x7f7fdc040940 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40ac000 tx=0x7f7fe40011a0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe8018070 con 0x7f7ff013d1d0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 monclient: dump: 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.871 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.872 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7fe8004d80 con 0x7f7ff013d1d0 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 msgr2=0x7f7ff013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 0x7f7ff013d5b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe800def0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 msgr2=0x7f7ff013d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.840+0000 7f7fd7fff640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff013d1d0 0x7f7ff013d5b0 secure :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe800def0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7ff5844640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7ff5844640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 0x7f7ff0138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7ff5844640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff013daf0 con 0x7f7ff01388a0 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 0x7f7ff0138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7feeffd640 10 monclient: get_auth_request con 0x7f7ff01388a0 auth_method 0 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7feeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7feeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.844+0000 7f7feeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 10 monclient(hunting): handle_auth_done global_id 19173 payload 824 2024-09-17T16:30:58.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 1 -- 172.21.15.105:0/2776304745 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7ff013e1c0 con 0x7f7ff01388a0 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 0x7f7ff0138c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe8002020 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 client.23604.objecter ms_handle_connect 0x7f7ff01388a0 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 client.23604.objecter resend_mon_ops 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe8018070 con 0x7f7ff01388a0 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 monclient: dump: 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:58.876 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9445 (secure 0 0 0) 0x7f7fe8019040 con 0x7f7ff01388a0 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 msgr2=0x7f7ff0138c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 0x7f7ff0138c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe8002020 comp rx=0 tx=0).stop 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 msgr2=0x7f7ff0138c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fd7fff640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ff01388a0 0x7f7ff0138c80 secure :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ff0 tx=0x7f7fe8002020 comp rx=0 tx=0).stop 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 206472, 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:58.878 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.879 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.880 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:58.881 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:58.882 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:58.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:58.885 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:58.891 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:58.893 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.894 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:58.895 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.896 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:58.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:58.900 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:58.901 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 10 librados: watch_flush enter 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 10 librados: watch_flush exit 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 20 client.23604.objecter shutdown clearing up homeless session... 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 10 client.23604.objecter successfully canceled tick 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7fdc03e480 msgr2=0x7f7fdc040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7fdc03e480 0x7f7fdc040940 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40ac000 tx=0x7f7fe40011a0 comp rx=0 tx=0).stop 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 10 monclient: shutdown 2024-09-17T16:30:58.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 msgr2=0x7f7ff013cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 --2- 172.21.15.105:0/2776304745 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7ff0137b70 0x7f7ff013cc90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00acf60 tx=0x7f7fe0011920 comp rx=0 tx=0).stop 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 reap_dead start 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2776304745 reap_dead start 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 shutdown_connections 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 >> 172.21.15.105:0/2776304745 conn(0x7f7ff0068ba0 msgr2=0x7f7ff00a9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 shutdown_connections 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 -- 172.21.15.105:0/2776304745 wait complete. 2024-09-17T16:30:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:58.848+0000 7f7ff5844640 1 librados: shutdown 2024-09-17T16:30:58.904 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:58.905 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-17T16:30:58.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 10 monclient: get_monmap_and_config 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 10 monclient: build_initial_monmap 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 10 monclient: monmap: 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:59.022833+0000 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:59.022833+0000 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.025 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:59.026 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 1 Processor -- start 2024-09-17T16:30:59.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 1 -- start start 2024-09-17T16:30:59.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.016+0000 7efdb2627640 10 monclient: init 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient: _add_conns ranks=[2,3,6,4,5,9,8,10,1,0,7] 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 0x7efdac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-b con 0x7efdac0a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 0x7efdac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-g con 0x7efdac0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-d con 0x7efdac1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efdac0a3fc0 con 0x7efdac0a68e0 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efdac0a63c0 con 0x7efdac0a4910 2024-09-17T16:30:59.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efdac0a7740 con 0x7efdac1396a0 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient(hunting): authenticate will time out at 1142.288614s 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdabfff640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 msgr2=0x7efdac0a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdabfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 0x7efdac0a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39768/0 (socket says 172.21.15.105:39768) 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 -- 172.21.15.105:0/3714902361 learned_addr learned my addr 172.21.15.105:0/3714902361 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): get_auth_request con 0x7efdac1396a0 auth_method 0 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdab7fe640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 0x7efdac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_done global_id 23613 payload 274 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 -- 172.21.15.105:0/3714902361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 msgr2=0x7efdac0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 0x7efdac0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 -- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 msgr2=0x7efdac0a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 0x7efdac0a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient: _finish_hunting 0 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 monclient: found mon.noname-d 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 -- 172.21.15.105:0/3714902361 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac06ec10 con 0x7efdac1396a0 2024-09-17T16:30:59.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdab7fe640 10 monclient: get_auth_request con 0x7efdac0a4910 auth_method 0 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdab7fe640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 0x7efdac0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efda4009eb0 tx=0x7efda401d370 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b2be0fd46d0d1576 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 1 -- 172.21.15.105:0/3714902361 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda40090d0 con 0x7efdac1396a0 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: dump: 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.034 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 1 -- 172.21.15.105:0/3714902361 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda40a5a70 con 0x7efdac1396a0 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 5 monclient: authenticate success, global_id 23613 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 1 -- 172.21.15.105:0/3714902361 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda40a5c60 con 0x7efdac1396a0 2024-09-17T16:30:59.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 4 set_mon_vals no callback set 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdaaffd640 10 monclient: dump: 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 10 monclient: shutdown 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 -- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 msgr2=0x7efdac139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.020+0000 7efdb2627640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efda4009eb0 tx=0x7efda401d370 comp rx=0 tx=0).stop 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/3714902361 shutdown_connections 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdac0a4910 0x7efdac0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac139ae0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- 172.21.15.105:0/3714902361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efdac0a68e0 0x7efdac0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/3714902361 >> 172.21.15.105:0/3714902361 conn(0x7efdac068ba0 msgr2=0x7efdac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/3714902361 shutdown_connections 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/3714902361 wait complete. 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: build_initial_monmap 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: monmap: 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:59.038 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 librados: starting msgr at 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 librados: starting objecter 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 Processor -- start 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- start start 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 librados: setting wanted keys 2024-09-17T16:30:59.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 librados: calling monclient init 2024-09-17T16:30:59.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: init 2024-09-17T16:30:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _add_conns ranks=[5,4,6,8,3,7,0,1,2] 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a4910 0x7efdac13d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-h con 0x7efdac0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efdac0a68e0 0x7efdac13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-c con 0x7efdac0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): picked mon.noname-d con 0x7efdac1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7efdac143320 con 0x7efdac0a68e0 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7efdac06e630 con 0x7efdac0a4910 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efdac1434a0 con 0x7efdac1396a0 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): authenticate will time out at 1142.293651s 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac13e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39778/0 (socket says 172.21.15.105:39778) 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 learned_addr learned my addr 172.21.15.105:0/155509620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): get_auth_request con 0x7efdac1396a0 auth_method 0 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efdac0a68e0 0x7efdac13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): get_auth_request con 0x7efdac0a68e0 auth_method 0 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a4910 0x7efdac13d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): get_auth_request con 0x7efdac0a4910 auth_method 0 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient(hunting): handle_auth_done global_id 23622 payload 824 2024-09-17T16:30:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a4910 msgr2=0x7efdac13d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a4910 0x7efdac13d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efdac0a68e0 msgr2=0x7efdac13dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efdac0a68e0 0x7efdac13dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient: _finish_hunting 0 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 monclient: found mon.noname-d 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac123b80 con 0x7efdac1396a0 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdab7fe640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efdac0a68e0 0x7efdac13dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a4910 0x7efdac13d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb0bc4640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac13e1e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efda4009040 tx=0x7efda40196b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 client.?.objecter ms_handle_connect 0x7efdac1396a0 2024-09-17T16:30:59.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda40bc020 con 0x7efdac1396a0 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: dump: 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.051 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: _finish_auth 0 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 5 monclient: authenticate success, global_id 23622 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4003bb0 con 0x7efdac1396a0 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _renew_subs 2024-09-17T16:30:59.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda4003da0 con 0x7efdac1396a0 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac0a9e70 con 0x7efdac1396a0 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdaa7fc640 4 set_mon_vals no callback set 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 10 monclient: dump: 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 client.23622.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _renew_subs 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac0aba00 con 0x7efdac1396a0 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 librados: init done 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 0x7efdac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7efdac123520 con 0x7efdac0a0a20 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 0x7efdac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient: get_auth_request con 0x7efdac0a0a20 auth_method 0 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.024+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efda40bf020 con 0x7efdac1396a0 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdabfff640 10 monclient(hunting): handle_auth_done global_id 19292 payload 824 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdabfff640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdac13edd0 con 0x7efdac0a0a20 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd9803e580 0x7efd98040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(111..111 src has 1..111) ==== 7879+0+0 (secure 0 0 0) 0x7efda40eb760 con 0x7efdac1396a0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 client.23622.objecter ms_dispatch 0x7efdac000ba0 osd_map(111..111 src has 1..111) 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 3 client.23622.objecter handle_osd_map got epochs [111,111] > 0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 3 client.23622.objecter handle_osd_map decoding full epoch 111 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 20 client.23622.objecter dump_active .. 0 homeless 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 client.23622.objecter ms_handle_connect 0x7efdac0a0a20 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 client.23622.objecter resend_mon_ops 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 0x7efdac0a0e00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda000dbc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdab7fe640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd9803e580 0x7efd98040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdab7fe640 10 monclient: get_auth_request con 0x7efd9803e580 auth_method 0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda00039c0 con 0x7efdac0a0a20 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 monclient: dump: 2024-09-17T16:30:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 client.23622.objecter ms_handle_connect 0x7efd9803e580 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efdab7fe640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd9803e580 0x7efd98040a40 secure :-1 s=READY pgs=3377 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006fd0 tx=0x7efd9c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efda0005a50 con 0x7efdac0a0a20 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 msgr2=0x7efdac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 0x7efdac0a0e00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda000dbc0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 msgr2=0x7efdac0a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.028+0000 7efda8ff9640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a0a20 0x7efdac0a0e00 secure :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda000dbc0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.032+0000 7efdb2627640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.032+0000 7efdb2627640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 0x7efdac1db410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.032+0000 7efdb2627640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.032+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7efdac051e80 con 0x7efdac0a1340 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 0x7efdac1db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient: get_auth_request con 0x7efdac0a1340 auth_method 0 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 10 monclient(hunting): handle_auth_done global_id 19301 payload 824 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 1 -- 172.21.15.105:0/155509620 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efdac139ae0 con 0x7efdac0a1340 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdabfff640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 0x7efdac1db410 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda00059e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 client.23622.objecter ms_handle_connect 0x7efdac0a1340 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 client.23622.objecter resend_mon_ops 2024-09-17T16:30:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda001bbd0 con 0x7efdac0a1340 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 monclient: dump: 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7efda001b760 con 0x7efdac0a1340 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 msgr2=0x7efdac1db410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 0x7efdac1db410 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda00059e0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 msgr2=0x7efdac1db410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efda8ff9640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdac0a1340 0x7efdac1db410 secure :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7efda000d700 tx=0x7efda00059e0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 206640, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:59.062 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.063 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.064 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:59.065 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.066 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.068 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:59.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:59.071 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:59.072 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:59.074 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:59.075 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.076 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:59.077 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.078 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.079 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.080 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.081 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:30:59.082 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:59.083 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 10 librados: watch_flush enter 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 10 librados: watch_flush exit 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 20 client.23622.objecter shutdown clearing up homeless session... 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 10 client.23622.objecter successfully canceled tick 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd9803e580 msgr2=0x7efd98040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efd9803e580 0x7efd98040a40 secure :-1 s=READY pgs=3377 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006fd0 tx=0x7efd9c008040 comp rx=0 tx=0).stop 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 10 monclient: shutdown 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 msgr2=0x7efdac13e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 --2- 172.21.15.105:0/155509620 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdac1396a0 0x7efdac13e1e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efda4009040 tx=0x7efda40196b0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 reap_dead start 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb0bc4640 1 -- 172.21.15.105:0/155509620 reap_dead start 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 shutdown_connections 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 >> 172.21.15.105:0/155509620 conn(0x7efdac068ba0 msgr2=0x7efdac0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 shutdown_connections 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 -- 172.21.15.105:0/155509620 wait complete. 2024-09-17T16:30:59.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.036+0000 7efdb2627640 1 librados: shutdown 2024-09-17T16:30:59.087 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:59.087 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-17T16:30:59.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:30:59.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: get_monmap_and_config 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: build_initial_monmap 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: monmap: 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:30:59.199932+0000 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:30:59.199932+0000 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:30:59.203 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 Processor -- start 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 -- start start 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: init 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient: _add_conns ranks=[8,7,5,4,0,3,10,2,1,6,9] 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-e con 0x7fced80a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced80a4b30 0x7fced80a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-i con 0x7fced80a4b30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fced8136500 0x7fced8136960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-h con 0x7fced8136500 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fced809de80 con 0x7fced8136500 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fced809e850 con 0x7fced80a4b30 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fced809f570 con 0x7fced80a4750 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcede2ca640 10 monclient(hunting): authenticate will time out at 1142.465689s 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59566/0 (socket says 172.21.15.105:59566) 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 -- 172.21.15.105:0/2363448465 learned_addr learned my addr 172.21.15.105:0/2363448465 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): get_auth_request con 0x7fced80a4750 auth_method 0 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcedc867640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fced8136500 0x7fced8136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcedc867640 10 monclient(hunting): get_auth_request con 0x7fced8136500 auth_method 0 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcedc867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcedc867640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fcedc867640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced77fe640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced80a4b30 0x7fced80a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient(hunting): handle_auth_done global_id 23615 payload 274 2024-09-17T16:30:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 -- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced80a4b30 msgr2=0x7fced80a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced80a4b30 0x7fced80a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 -- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fced8136500 msgr2=0x7fced8136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fced8136500 0x7fced8136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 monclient: found mon.noname-e 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.196+0000 7fced7fff640 1 -- 172.21.15.105:0/2363448465 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced809f850 con 0x7fced80a4750 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced7fff640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcec00098d0 tx=0x7fcec001c0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=261021fbc20e2f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 1 -- 172.21.15.105:0/2363448465 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcec00ab070 con 0x7fced80a4750 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: dump: 2024-09-17T16:30:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.210 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 5 monclient: authenticate success, global_id 23615 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 1 -- 172.21.15.105:0/2363448465 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec00a5910 con 0x7fced80a4750 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 1 -- 172.21.15.105:0/2363448465 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcec00a6480 con 0x7fced80a4750 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: get_monmap_and_config success 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 4 set_mon_vals no callback set 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced6ffd640 10 monclient: dump: 2024-09-17T16:30:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.213 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: shutdown 2024-09-17T16:30:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- 172.21.15.105:0/2363448465 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 msgr2=0x7fced80a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcec00098d0 tx=0x7fcec001c0d0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- 172.21.15.105:0/2363448465 shutdown_connections 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fced8136500 0x7fced8136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced80a4b30 0x7fced80a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- 172.21.15.105:0/2363448465 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fced80a4750 0x7fced80a0490 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- 172.21.15.105:0/2363448465 >> 172.21.15.105:0/2363448465 conn(0x7fced80a93d0 msgr2=0x7fced80a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- 172.21.15.105:0/2363448465 shutdown_connections 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- 172.21.15.105:0/2363448465 wait complete. 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: build_initial_monmap 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: monmap: 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:30:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 librados: starting msgr at 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 librados: starting objecter 2024-09-17T16:30:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 Processor -- start 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- start start 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 librados: setting wanted keys 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 librados: calling monclient init 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: init 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: _reopen_session rank -1 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient: _add_conns ranks=[2,3,7,6,0,4,8,5,1] 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fced80a4750 0x7fced8134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-b con 0x7fced80a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fced80a4b30 0x7fced8134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-g con 0x7fced80a4b30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 0x7fced8135500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): picked mon.noname-i con 0x7fced8136500 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fced8142180 con 0x7fced80a4750 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fced8142300 con 0x7fced80a4b30 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fced8142480 con 0x7fced8136500 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): _renew_subs 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcede2ca640 10 monclient(hunting): authenticate will time out at 1142.470899s 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced7fff640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fced80a4750 msgr2=0x7fced8134a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fced80a4750 0x7fced8134a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 0x7fced8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 0x7fced8135500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:49030/0 (socket says 172.21.15.105:49030) 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 learned_addr learned my addr 172.21.15.105:0/712166772 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:30:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): get_auth_request con 0x7fced8136500 auth_method 0 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fced80a4b30 0x7fced8134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): get_auth_request con 0x7fced80a4b30 auth_method 0 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fced77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient(hunting): handle_auth_done global_id 34252 payload 824 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fced80a4b30 msgr2=0x7fced8134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fced80a4b30 0x7fced8134fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fced80a4750 msgr2=0x7fced8134a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fced80a4750 0x7fced8134a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient: _finish_hunting 0 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 monclient: found mon.noname-i 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.200+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced80777c0 con 0x7fced8136500 2024-09-17T16:30:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcedc867640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 0x7fced8135500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fced0006e90 tx=0x7fced000d9e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.?.objecter ms_handle_connect 0x7fced8136500 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fced0012a60 con 0x7fced8136500 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: dump: 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: _finish_auth 0 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:30:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced0019070 con 0x7fced8136500 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 5 monclient: authenticate success, global_id 34252 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 monclient: _renew_subs 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced8120410 con 0x7fced8136500 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fced0012c00 con 0x7fced8136500 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: dump: 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced67fc640 4 set_mon_vals no callback set 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 client.34252.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 monclient: _renew_subs 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fced80a0620 con 0x7fced8136500 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 1 librados: init done 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 0x7fced8072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fced806cf90 con 0x7fced8072150 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 0x7fced8072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient: get_auth_request con 0x7fced8072150 auth_method 0 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fced00125f0 con 0x7fced8136500 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fceb803e580 0x7fceb8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 10 monclient(hunting): handle_auth_done global_id 34261 payload 824 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(111..111 src has 1..111) ==== 7879+0+0 (secure 0 0 0) 0x7fced000f040 con 0x7fced8136500 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.34252.objecter ms_dispatch 0x7fced8000ba0 osd_map(111..111 src has 1..111) 2024-09-17T16:30:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 3 client.34252.objecter handle_osd_map got epochs [111,111] > 0 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 3 client.34252.objecter handle_osd_map decoding full epoch 111 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fced813c250 con 0x7fced8072150 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 20 client.34252.objecter dump_active .. 0 homeless 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced77fe640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fceb803e580 0x7fceb8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced77fe640 10 monclient: get_auth_request con 0x7fceb803e580 auth_method 0 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.34252.objecter ms_handle_connect 0x7fced8072150 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.34252.objecter resend_mon_ops 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced7fff640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 0x7fced8072530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad000 tx=0x7fcec001a960 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcec00ab030 con 0x7fced8072150 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: dump: 2024-09-17T16:30:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 client.34252.objecter ms_handle_connect 0x7fceb803e580 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced77fe640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fceb803e580 0x7fceb8040a40 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7fced80a6670 tx=0x7fcec80077d0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcec00a54a0 con 0x7fced8072150 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 msgr2=0x7fced8072530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 0x7fced8072530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad000 tx=0x7fcec001a960 comp rx=0 tx=0).stop 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 msgr2=0x7fced8072530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.204+0000 7fced4ff9640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072150 0x7fced8072530 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad000 tx=0x7fcec001a960 comp rx=0 tx=0).stop 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fcede2ca640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fcede2ca640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 0x7fced813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fcede2ca640 10 monclient(hunting): start opening mon connection 2024-09-17T16:30:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fced8051e80 con 0x7fced8072a70 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 0x7fced813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient: get_auth_request con 0x7fced8072a70 auth_method 0 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 10 monclient(hunting): handle_auth_done global_id 34270 payload 824 2024-09-17T16:30:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 1 -- 172.21.15.105:0/712166772 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fced80a4f10 con 0x7fced8072a70 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced7fff640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 0x7fced813a440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcec001c110 tx=0x7fcec00191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 10 client.34252.objecter ms_handle_connect 0x7fced8072a70 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 10 client.34252.objecter resend_mon_ops 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcec00a6b10 con 0x7fced8072a70 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:30:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.212+0000 7fced4ff9640 10 monclient: dump: 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:30:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7fcec0003620 con 0x7fced8072a70 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 msgr2=0x7fced813a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 0x7fced813a440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcec001c110 tx=0x7fcec00191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 msgr2=0x7fced813a440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:30:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fced4ff9640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8072a70 0x7fced813a440 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcec001c110 tx=0x7fcec00191e0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 38, 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 50015, 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:30:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.241 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:30:59.243 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:30:59.244 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.245 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:30:59.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:30:59.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.252 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.253 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:30:59.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.257 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.258 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:30:59.260 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:30:59.261 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:30:59.262 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 10 librados: watch_flush enter 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 10 librados: watch_flush exit 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 20 client.34252.objecter shutdown clearing up homeless session... 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 10 client.34252.objecter successfully canceled tick 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fceb803e580 msgr2=0x7fceb8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fceb803e580 0x7fceb8040a40 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7fced80a6670 tx=0x7fcec80077d0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 10 monclient: shutdown 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 msgr2=0x7fced8135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:30:59.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 --2- 172.21.15.105:0/712166772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fced8136500 0x7fced8135500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fced0006e90 tx=0x7fced000d9e0 comp rx=0 tx=0).stop 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 reap_dead start 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcedc867640 1 -- 172.21.15.105:0/712166772 reap_dead start 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 shutdown_connections 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 >> 172.21.15.105:0/712166772 conn(0x7fced80a93d0 msgr2=0x7fced80a2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 shutdown_connections 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 -- 172.21.15.105:0/712166772 wait complete. 2024-09-17T16:30:59.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:30:59.216+0000 7fcede2ca640 1 librados: shutdown 2024-09-17T16:30:59.265 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:30:59.265 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:31:02.495 INFO:tasks.workunit.client.0.smithi160.stdout: pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1859 ms) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (13992 ms total) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4128 ms) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3907 ms) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (24080 ms) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (32115 ms total) 2024-09-17T16:31:02.496 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: 2024-09-17T16:31:02.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-17T16:31:02.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (121720 ms total) 2024-09-17T16:31:02.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-17T16:31:02.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-09-17T16:31:02.497 INFO:tasks.workunit.client.0.smithi160.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-17T16:31:08.529 INFO:tasks.workunit.client.0.smithi160.stdout:PP (76483 ms total) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (179 ms) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (179 ms total) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (50 ms) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (5 ms) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (55 ms total) 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.530 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (35 ms) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (9 ms) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (44 ms total) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (87 ms) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (3 ms) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (90 ms total) 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.531 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1042 ms) 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1042 ms total) 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (127758 ms total) 2024-09-17T16:31:08.532 INFO:tasks.workunit.client.0.smithi160.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-17T16:31:19.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2336 ms) 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (16546 ms) 2024-09-17T16:31:19.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (5911 ms) 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3046 ms) 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3033 ms) 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2981 ms) 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2952 ms) 2024-09-17T16:31:19.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3061 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2949 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3037 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2946 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (6011 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3032 ms) 2024-09-17T16:31:19.539 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2998 ms) 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (2973 ms) 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2999 ms) 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-17T16:31:19.540 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (23078 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2930 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3018 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (22081 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3012 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (2905 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2999 ms) 2024-09-17T16:31:19.541 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3024 ms) 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2970 ms) 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3031 ms) 2024-09-17T16:31:19.542 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-17T16:32:08.772 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAor()=8:b29083e3:::5:head expected=8:b29083e3:::5:head 2024-09-17T16:32:08.772 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:b29083e3:::5:head -> 5 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=5 expected=5 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:89d3ae78:::11:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:89d3ae78:::11:head expected=8:89d3ae78:::11:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:89d3ae78:::11:head -> 11 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=11 expected=11 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:863748b0:::15:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:863748b0:::15:head expected=8:863748b0:::15:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:863748b0:::15:head -> 15 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=15 expected=15 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:6cac518f:::0:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:6cac518f:::0:head expected=8:6cac518f:::0:head 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:6cac518f:::0:head -> 0 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=0 expected=0 2024-09-17T16:32:08.773 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:62a1935d:::14:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:62a1935d:::14:head expected=8:62a1935d:::14:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:62a1935d:::14:head -> 14 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=14 expected=14 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:5c6b0b28:::7:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:5c6b0b28:::7:head expected=8:5c6b0b28:::7:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:5c6b0b28:::7:head -> 7 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=7 expected=7 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:52ea6a34:::10:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:52ea6a34:::10:head expected=8:52ea6a34:::10:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:52ea6a34:::10:head -> 10 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=10 expected=10 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : seek to 8:566253c9:::13:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : cursor()=8:566253c9:::13:head expected=8:566253c9:::13:head 2024-09-17T16:32:08.774 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: > 8:566253c9:::13:head -> 13 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: : entry=13 expected=13 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (660 ms) 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (51097 ms) 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 0/5 -> MIN 8:33333333::::head 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 1/5 -> 8:33333333::::head 8:66666666::::head 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 2/5 -> 8:66666666::::head 8:99999999::::head 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 3/5 -> 8:99999999::::head 8:cccccccc::::head 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: split 4/5 -> 8:cccccccc::::head MAX 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (129058 ms) 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 7 tests from LibRadosList (181637 ms total) 2024-09-17T16:32:08.775 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1061 ms) 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo2,foo3 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo2 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo3 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo1,foo4,foo5 2024-09-17T16:32:08.776 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo4 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo5 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo1 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset foo6,foo7 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo7 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: foo6 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo4 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo5 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo7 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns2:foo6 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: ns1:foo1 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo1 2024-09-17T16:32:08.777 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo2 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: :foo3 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (76 ms) 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 1 0 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 10 0 2024-09-17T16:32:08.778 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 13 0 2024-09-17T16:32:14.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 7 0 2024-09-17T16:32:14.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 14 0 2024-09-17T16:32:14.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 0 0 2024-09-17T16:32:14.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 15 0 2024-09-17T16:32:14.536 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 11 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 5 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 8 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 6 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 3 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 4 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 12 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 9 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2 0 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (131 ms) 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 3 tests from LibRadosListEC (1268 ms total) 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-17T16:32:14.537 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2924 ms) 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] 1 test from LibRadosListNP (2924 ms total) 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [----------] Global test environment tear-down 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [==========] 11 tests from 3 test suites ran. (193827 ms total) 2024-09-17T16:32:14.538 INFO:tasks.workunit.client.0.smithi160.stdout: api_list: [ PASSED ] 11 tests. 2024-09-17T16:32:29.266 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-17T16:32:29.266 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T16:32:29.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T16:32:29.270 INFO:tasks.ceph.mon.b:Started 2024-09-17T16:32:29.270 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-17T16:32:29.270 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-17T16:32:29.270 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:32:29.367 INFO:tasks.ceph.mon.b.smithi105.stderr:2024-09-17T16:32:29.359+0000 7fe2d8b41e00 -1 mon.b@-1(???).log v0 log_external_backlog rewinding external_log_to from 124 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:32:44.271 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-17T16:32:44.272 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-17T16:32:44.272 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:32:44.272 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:32:44.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: get_monmap_and_config 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: build_initial_monmap 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: monmap: 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:44.401161+0000 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:44.401161+0000 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:44.404 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:44.405 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 Processor -- start 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 -- start start 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: init 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient: _add_conns ranks=[3,0,7,2,8,6,1,4,5,9,10] 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:44.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): picked mon.noname-g con 0x7f10840a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f10840a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): picked mon.noname-i con 0x7f10840a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108406ebc0 con 0x7f10840a4910 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f10840a3fc0 con 0x7f10840a68e0 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f10840a63c0 con 0x7f10840a7200 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1089298640 10 monclient(hunting): authenticate will time out at 1247.666945s 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10827fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f10840a4910 0x7f10841396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38950/0 (socket says 172.21.15.105:38950) 2024-09-17T16:32:44.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 learned_addr learned my addr 172.21.15.105:0/2154836475 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f10840a7200 auth_method 0 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f10840a68e0 auth_method 0 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:44.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:32:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:44.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.399+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f10840a7200 auth_method 0 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f10840a68e0 auth_method 0 2024-09-17T16:32:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:44.603+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:32:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f10840a68e0 auth_method 0 2024-09-17T16:32:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:45.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f10840a7200 auth_method 0 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:45.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:45.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:32:45.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.007+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:32:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f10840a68e0 auth_method 0 2024-09-17T16:32:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:45.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f10840a7200 auth_method 0 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:32:45.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:32:45.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:45.811+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:32:47.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): tick 2024-09-17T16:32:47.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:32:47.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:32:47.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:32:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 msgr2=0x7f1084139db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:32:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f10840a7200 0x7f1084139db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 msgr2=0x7f10840a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:32:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10840a68e0 0x7f10840a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 >> v1:172.21.15.105:6789/0 conn(0x7f10840a4910 legacy=0x7f10841396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient: _add_conns ranks=[1,9,10,4,6,5,8,2] 2024-09-17T16:32:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1070003850 addr v1:172.21.15.160:6789/0 2024-09-17T16:32:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f1070007490 addr v2:172.21.15.105:3300/0 2024-09-17T16:32:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f107000a020 addr v2:172.21.15.160:3300/0 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10840a3690 con 0x7f1070003850 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f10840a3fc0 con 0x7f1070007490 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 1 -- 172.21.15.105:0/2154836475 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f107000dbe0 con 0x7f107000a020 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f1080ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f1070007490 auth_method 0 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:47.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f107000a020 auth_method 0 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.399+0000 7f10837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.403+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.403+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.403+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:47.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.403+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f1070007490 auth_method 0 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.603+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:32:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f107000a020 auth_method 0 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:47.607+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f1070007490 auth_method 0 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:48.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:48.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.007+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f107000a020 auth_method 0 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:48.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:48.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.011+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:32:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f1070007490 auth_method 0 2024-09-17T16:32:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.811+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f107000a020 auth_method 0 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:48.815+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:32:49.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2108487769 0 0) 0x7f10840a3690 con 0x7f1070003850 2024-09-17T16:32:49.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 10 monclient(hunting): _negotiate 2024-09-17T16:32:49.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:49.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 10 monclient(hunting): my global_id is 25021 2024-09-17T16:32:49.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.303+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f106c003530 con 0x7f1070003850 2024-09-17T16:32:49.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 479533620 0 0) 0x7f106c003530 con 0x7f1070003850 2024-09-17T16:32:49.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 msgr2=0x7f107000c4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:32:49.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.160:3300/0 conn(0x7f107000a020 0x7f107000c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 msgr2=0x7f1070009930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 --2- 172.21.15.105:0/2154836475 >> v2:172.21.15.105:3300/0 conn(0x7f1070007490 0x7f1070009930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _finish_hunting 0 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 monclient: found mon.noname-f-legacy 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 1 -- 172.21.15.105:0/2154836475 reap_dead start 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.160:6789/0 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107000ddf0 con 0x7f1070003850 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _finish_auth 0 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f107800f040 con 0x7f1070003850 2024-09-17T16:32:49.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: dump: 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:49.322 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] but i'm connected to v1:172.21.15.160:6789/0, reconnecting 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 >> v1:172.21.15.160:6789/0 conn(0x7f1070003850 legacy=0x7f1070005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient: _add_conns ranks=[0,3,4,2,1,5,8,6,7] 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 0x7f106c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): picked mon.a con 0x7f106c004210 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f106c004e20 0x7f106c0052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): picked mon.g con 0x7f106c004e20 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c009c00 0x7f106c00c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): picked mon.c con 0x7f106c009c00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10840a3fc0 con 0x7f106c004210 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f107000dbe0 con 0x7f106c004e20 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f106c00c9a0 con 0x7f106c009c00 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1081ffb640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 0x7f106c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f106c004e20 0x7f106c0052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f106c004210 auth_method 0 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c009c00 0x7f106c00c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f106c004e20 auth_method 0 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:49.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f106c009c00 auth_method 0 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient(hunting): handle_auth_done global_id 25021 payload 293 2024-09-17T16:32:49.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c009c00 msgr2=0x7f106c00c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c009c00 0x7f106c00c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f106c004e20 msgr2=0x7f106c0052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f106c004e20 0x7f106c0052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 monclient: found mon.a 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 1 -- 172.21.15.105:0/2154836475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c003530 con 0x7f106c004210 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient: _finish_auth 0 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:32:49.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1082ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1089298640 5 monclient: authenticate success, global_id 25021 2024-09-17T16:32:49.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.315+0000 7f1089298640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.319+0000 7f1082ffd640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 0x7f106c004810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f107800eda0 tx=0x7f107801d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b200cdfbe05cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10780b3070 con 0x7f106c004210 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: dump: 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:49.330 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:49.331 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:49.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10780ad810 con 0x7f106c004210 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 1 -- 172.21.15.105:0/2154836475 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f107800e9f0 con 0x7f106c004210 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 4 set_mon_vals no callback set 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1081ffb640 10 monclient: dump: 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:49.333 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 10 monclient: shutdown 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 msgr2=0x7f106c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 0x7f106c004810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f107800eda0 tx=0x7f107801d5a0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- 172.21.15.105:0/2154836475 shutdown_connections 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c009c00 0x7f106c00c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f106c004e20 0x7f106c0052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 --2- 172.21.15.105:0/2154836475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c004210 0x7f106c004810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- 172.21.15.105:0/2154836475 >> 172.21.15.105:0/2154836475 conn(0x7f1084068ba0 msgr2=0x7f1084068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- 172.21.15.105:0/2154836475 shutdown_connections 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- 172.21.15.105:0/2154836475 wait complete. 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 10 monclient: build_initial_monmap 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 10 monclient: monmap: 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 librados: starting msgr at 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 librados: starting objecter 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 Processor -- start 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 -- start start 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 librados: setting wanted keys 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 1 librados: calling monclient init 2024-09-17T16:32:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.323+0000 7f1089298640 10 monclient: init 2024-09-17T16:32:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:49.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _add_conns ranks=[2,4,6,3,5,8,7,1,0] 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): picked mon.noname-b con 0x7f106c004210 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 0x7f10841386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): picked mon.noname-c con 0x7f106c004e20 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f106c009c00 0x7f1084122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): picked mon.noname-d con 0x7f106c009c00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f108406e540 con 0x7f106c004210 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f108406b5b0 con 0x7f106c004e20 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f108406e6c0 con 0x7f106c009c00 2024-09-17T16:32:49.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient(hunting): authenticate will time out at 1252.593006s 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 0x7f10841386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f106c009c00 0x7f1084122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:48092/0 (socket says 172.21.15.105:48092) 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 -- 172.21.15.105:0/1413744574 learned_addr learned my addr 172.21.15.105:0/1413744574 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 10 monclient(hunting): get_auth_request con 0x7f106c004210 auth_method 0 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): get_auth_request con 0x7f106c004e20 auth_method 0 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): get_auth_request con 0x7f106c009c00 auth_method 0 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 msgr2=0x7f10841381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 msgr2=0x7f10841381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:49.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1082ffd640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient(hunting): handle_auth_done global_id 22954 payload 824 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f106c009c00 msgr2=0x7f1084122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f106c009c00 0x7f1084122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 msgr2=0x7f10841381a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient: _finish_hunting 0 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 monclient: found mon.noname-c 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 -- 172.21.15.105:0/1413744574 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1084125d00 con 0x7f106c004e20 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f10827fc640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 0x7f10841386e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1074017680 tx=0x7f1074017b40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 client.?.objecter ms_handle_connect 0x7f106c004e20 2024-09-17T16:32:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1074017e80 con 0x7f106c004e20 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: dump: 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10740158b0 con 0x7f106c004e20 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 5 monclient: authenticate success, global_id 22954 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _renew_subs 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1084125fe0 con 0x7f106c004e20 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10740070d0 con 0x7f106c004e20 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1080ff9640 4 set_mon_vals no callback set 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 monclient: dump: 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 client.22954.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _renew_subs 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f10840aa510 con 0x7f106c004e20 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 librados: init done 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084139eb0 con 0x7f106c004e20 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1074005ab0 con 0x7f106c004e20 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 0x7f1070041a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(202..202 src has 1..202) ==== 6600+0+0 (secure 0 0 0) 0x7f1074010040 con 0x7f106c004e20 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 10 client.22954.objecter ms_dispatch 0x7f1084000ba0 osd_map(202..202 src has 1..202) 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 3 client.22954.objecter handle_osd_map got epochs [202,202] > 0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.327+0000 7f106b7fe640 3 client.22954.objecter handle_osd_map decoding full epoch 202 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.331+0000 7f106b7fe640 20 client.22954.objecter dump_active .. 0 homeless 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.331+0000 7f1082ffd640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 0x7f1070041a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.331+0000 7f1082ffd640 10 monclient: get_auth_request con 0x7f107003f5b0 auth_method 0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.331+0000 7f106b7fe640 10 client.22954.objecter ms_handle_connect 0x7f107003f5b0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.331+0000 7f1082ffd640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 0x7f1070041a70 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f1078000c10 tx=0x7f1078004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:49.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.335+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f107401f330 con 0x7f106c004e20 2024-09-17T16:32:49.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.335+0000 7f106b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:49.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.335+0000 7f106b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:49.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f1089298640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f1089298640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f1089298640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1084138f20 con 0x7f106c004e20 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f106b7fe640 1 -- 172.21.15.105:0/1413744574 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3206 (secure 0 0 0) 0x7f1074005da0 con 0x7f106c004e20 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f106b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.579+0000 7f106b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:49.585 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":44,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:32:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 10 librados: watch_flush enter 2024-09-17T16:32:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 10 librados: watch_flush exit 2024-09-17T16:32:49.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 20 client.22954.objecter shutdown clearing up homeless session... 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 10 client.22954.objecter successfully canceled tick 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 msgr2=0x7f1070041a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 0x7f1070041a70 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f1078000c10 tx=0x7f1078004740 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 10 monclient: shutdown 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 msgr2=0x7f10841386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 0x7f10841386e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1074017680 tx=0x7f1074017b40 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 shutdown_connections 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f107003f5b0 0x7f1070041a70 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f106c004e20 0x7f10841386e0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f106c009c00 0x7f1084122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 --2- 172.21.15.105:0/1413744574 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f106c004210 0x7f10841381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:49.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 >> 172.21.15.105:0/1413744574 conn(0x7f1084068ba0 msgr2=0x7f10840aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 shutdown_connections 2024-09-17T16:32:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 -- 172.21.15.105:0/1413744574 wait complete. 2024-09-17T16:32:49.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:49.583+0000 7f1089298640 1 librados: shutdown 2024-09-17T16:32:52.603 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:32:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: get_monmap_and_config 2024-09-17T16:32:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: build_initial_monmap 2024-09-17T16:32:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: monmap: 2024-09-17T16:32:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:52.721177+0000 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:52.721177+0000 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:52.725 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 Processor -- start 2024-09-17T16:32:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 -- start start 2024-09-17T16:32:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: init 2024-09-17T16:32:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:52.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient: _add_conns ranks=[9,6,10,5,1,8,3,7,4,2,0] 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 0x7fcb540a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-a con 0x7fcb540a4750 addr v2:172.21.15.105:3300/0 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-d con 0x7fcb540a5070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcb54137a50 0x7fcb54137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-f con 0x7fcb54137a50 addr v2:172.21.15.160:3300/0 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fcb540a1300 con 0x7fcb540a4750 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5406cf10 con 0x7fcb54137a50 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fcb540a6300 con 0x7fcb540a5070 2024-09-17T16:32:52.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb5ae29640 10 monclient(hunting): authenticate will time out at 1255.986828s 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 0x7fcb540a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:41512/0 (socket says 172.21.15.105:41512) 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 -- 172.21.15.105:0/4166818142 learned_addr learned my addr 172.21.15.105:0/4166818142 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): get_auth_request con 0x7fcb540a5070 auth_method 0 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): get_auth_request con 0x7fcb540a4750 auth_method 0 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:52.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb593c6640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.160:3300/0 conn(0x7fcb54137a50 0x7fcb54137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb593c6640 10 monclient(hunting): get_auth_request con 0x7fcb54137a50 auth_method 0 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb593c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb593c6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb593c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient(hunting): handle_auth_done global_id 23838 payload 274 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 -- 172.21.15.105:0/4166818142 >> v2:172.21.15.160:3300/0 conn(0x7fcb54137a50 msgr2=0x7fcb54137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:52.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.160:3300/0 conn(0x7fcb54137a50 0x7fcb54137ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 -- 172.21.15.105:0/4166818142 >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 msgr2=0x7fcb540a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 0x7fcb540a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient: _finish_hunting 0 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 monclient: found mon.noname-d 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb53fff640 1 -- 172.21.15.105:0/4166818142 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5406c590 con 0x7fcb540a5070 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.719+0000 7fcb58bc5640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 0x7fcb540a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb53fff640 1 --2- 172.21.15.105:0/4166818142 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fcb5409fd90 tx=0x7fcb4801c3e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c9e24d2ba8eaf0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 1 -- 172.21.15.105:0/4166818142 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb480b0070 con 0x7fcb540a5070 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: dump: 2024-09-17T16:32:52.737 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:52.738 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: _finish_auth 0 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: _check_auth_tickets 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 5 monclient: authenticate success, global_id 23838 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 1 -- 172.21.15.105:0/4166818142 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb480aa810 con 0x7fcb540a5070 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:52.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 4 set_mon_vals no callback set 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 1 -- 172.21.15.105:0/4166818142 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb480a9300 con 0x7fcb540a5070 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb537fe640 10 monclient: dump: 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:52.740 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: shutdown 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- 172.21.15.105:0/4166818142 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 msgr2=0x7fcb54137510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/4166818142 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fcb5409fd90 tx=0x7fcb4801c3e0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- 172.21.15.105:0/4166818142 shutdown_connections 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/4166818142 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb540a5070 0x7fcb54137510 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.160:3300/0 conn(0x7fcb54137a50 0x7fcb54137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/4166818142 >> v2:172.21.15.105:3300/0 conn(0x7fcb540a4750 0x7fcb540a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- 172.21.15.105:0/4166818142 >> 172.21.15.105:0/4166818142 conn(0x7fcb540a94d0 msgr2=0x7fcb540a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- 172.21.15.105:0/4166818142 shutdown_connections 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- 172.21.15.105:0/4166818142 wait complete. 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: build_initial_monmap 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: monmap: 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:52.742 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:52.743 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 librados: starting msgr at 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 librados: starting objecter 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 Processor -- start 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 -- start start 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 librados: setting wanted keys 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 librados: calling monclient init 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: init 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 10 monclient: _add_conns ranks=[4,2,7,5,0,3,1,6,8] 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.723+0000 7fcb5ae29640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-c con 0x7fcb540a4750 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:52.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-b con 0x7fcb540a5070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fcb54137a50 0x7fcb5413cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): picked mon.noname-i con 0x7fcb54137a50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5406c990 con 0x7fcb540a5070 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fcb5406c810 con 0x7fcb540a4750 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fcb540a03d0 con 0x7fcb54137a50 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient(hunting): authenticate will time out at 1255.992071s 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:44960/0 (socket says 172.21.15.105:44960) 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 -- 172.21.15.105:0/3011547684 learned_addr learned my addr 172.21.15.105:0/3011547684 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): get_auth_request con 0x7fcb540a4750 auth_method 0 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 10 monclient(hunting): get_auth_request con 0x7fcb540a5070 auth_method 0 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:52.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 msgr2=0x7fcb5413c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 msgr2=0x7fcb5413c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient(hunting): handle_auth_done global_id 22963 payload 824 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb593c6640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fcb54137a50 0x7fcb5413cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fcb54137a50 msgr2=0x7fcb5413cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fcb54137a50 0x7fcb5413cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 msgr2=0x7fcb5413c690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient: _finish_hunting 0 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 monclient: found mon.noname-c 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:52.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 -- 172.21.15.105:0/3011547684 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb541218d0 con 0x7fcb540a4750 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb58bc5640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e9b0 tx=0x7fcb4400ee70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 client.?.objecter ms_handle_connect 0x7fcb540a4750 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb540a4750 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: dump: 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:52.748 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: _finish_auth 0 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: _check_auth_tickets 2024-09-17T16:32:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4400ce70 con 0x7fcb540a4750 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 5 monclient: authenticate success, global_id 22963 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _renew_subs 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb54137e90 con 0x7fcb540a4750 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb440138e0 con 0x7fcb540a4750 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb52ffd640 4 set_mon_vals no callback set 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 client.22963.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 monclient: dump: 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:52.751 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _renew_subs 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb54120a60 con 0x7fcb540a4750 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 librados: init done 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fcb4401a070 con 0x7fcb540a4750 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb54137510 con 0x7fcb540a4750 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 0x7fcb40040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(207..207 src has 1..207) ==== 6261+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb540a4750 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 client.22963.objecter ms_dispatch 0x7fcb54000ba0 osd_map(207..207 src has 1..207) 2024-09-17T16:32:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 3 client.22963.objecter handle_osd_map got epochs [207,207] > 0 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 3 client.22963.objecter handle_osd_map decoding full epoch 207 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 20 client.22963.objecter dump_active .. 0 homeless 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 0x7fcb40040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 10 monclient: get_auth_request con 0x7fcb4003e550 auth_method 0 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb517fa640 10 client.22963.objecter ms_handle_connect 0x7fcb4003e550 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.727+0000 7fcb53fff640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 0x7fcb40040a10 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ad000 tx=0x7fcb480a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.731+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb44021070 con 0x7fcb540a4750 2024-09-17T16:32:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.731+0000 7fcb517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.731+0000 7fcb517fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:32:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:32:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcb54137270 con 0x7fcb540a4750 2024-09-17T16:32:52.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb517fa640 1 -- 172.21.15.105:0/3011547684 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3206 (secure 0 0 0) 0x7fcb4401add0 con 0x7fcb540a4750 2024-09-17T16:32:52.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:52.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb517fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:52.972 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":44,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:32:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 librados: watch_flush enter 2024-09-17T16:32:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 librados: watch_flush exit 2024-09-17T16:32:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 20 client.22963.objecter shutdown clearing up homeless session... 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.967+0000 7fcb5ae29640 10 client.22963.objecter successfully canceled tick 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 msgr2=0x7fcb40040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 0x7fcb40040a10 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7fcb480ad000 tx=0x7fcb480a6000 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 10 monclient: shutdown 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 msgr2=0x7fcb5413c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e9b0 tx=0x7fcb4400ee70 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 shutdown_connections 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb4003e550 0x7fcb40040a10 unknown :-1 s=CLOSED pgs=3424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fcb54137a50 0x7fcb5413cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb540a5070 0x7fcb5413c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 --2- 172.21.15.105:0/3011547684 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fcb540a4750 0x7fcb5413c150 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 >> 172.21.15.105:0/3011547684 conn(0x7fcb540a94d0 msgr2=0x7fcb540a1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:52.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 shutdown_connections 2024-09-17T16:32:52.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 -- 172.21.15.105:0/3011547684 wait complete. 2024-09-17T16:32:52.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:52.971+0000 7fcb5ae29640 1 librados: shutdown 2024-09-17T16:32:55.991 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:32:56.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: get_monmap_and_config 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: monmap: 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:56.117408+0000 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:56.117408+0000 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.120 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:56.121 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 Processor -- start 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 -- start start 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: init 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient: _add_conns ranks=[4,2,3,7,10,5,1,8,6,0,9] 2024-09-17T16:32:56.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-c con 0x7f14ac0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-b con 0x7f14ac0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14ac1396a0 0x7f14ac139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-g con 0x7f14ac1396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac0a3fc0 con 0x7f14ac0a4910 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac0a63c0 con 0x7f14ac1396a0 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f14ac0a7740 con 0x7f14ac0a68e0 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b3b6a640 10 monclient(hunting): authenticate will time out at 1259.383196s 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:44974/0 (socket says 172.21.15.105:44974) 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:48112/0 (socket says 172.21.15.105:48112) 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 -- 172.21.15.105:0/3390382861 learned_addr learned my addr 172.21.15.105:0/3390382861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): get_auth_request con 0x7f14ac0a68e0 auth_method 0 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 10 monclient(hunting): get_auth_request con 0x7f14ac0a4910 auth_method 0 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 -- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 -- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1105640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b2107640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14ac1396a0 0x7f14ac139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b2107640 10 monclient(hunting): get_auth_request con 0x7f14ac1396a0 auth_method 0 2024-09-17T16:32:56.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b2107640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b2107640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b2107640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient(hunting): handle_auth_done global_id 22981 payload 274 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 -- 172.21.15.105:0/3390382861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14ac1396a0 msgr2=0x7f14ac139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14ac1396a0 0x7f14ac139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 -- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac0a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 monclient: found mon.noname-c 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.115+0000 7f14b1906640 1 -- 172.21.15.105:0/3390382861 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac06ec10 con 0x7f14ac0a68e0 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b1906640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098d0 tx=0x7f14a401cea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fc936950f184b9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 1 -- 172.21.15.105:0/3390382861 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14a40b0070 con 0x7f14ac0a68e0 2024-09-17T16:32:56.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: dump: 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.139 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 1 -- 172.21.15.105:0/3390382861 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a40aa810 con 0x7f14ac0a68e0 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 1 -- 172.21.15.105:0/3390382861 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14a40a9300 con 0x7f14ac0a68e0 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 5 monclient: authenticate success, global_id 22981 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:56.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 4 set_mon_vals no callback set 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b0904640 10 monclient: dump: 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.141 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 10 monclient: shutdown 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 msgr2=0x7f14ac0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098d0 tx=0x7f14a401cea0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3390382861 shutdown_connections 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14ac1396a0 0x7f14ac139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3390382861 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f14ac0a68e0 0x7f14ac0a6cc0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3390382861 >> 172.21.15.105:0/3390382861 conn(0x7f14ac068ba0 msgr2=0x7f14ac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3390382861 shutdown_connections 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3390382861 wait complete. 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.119+0000 7f14b3b6a640 10 monclient: monmap: 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:56.143 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.144 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 librados: starting msgr at 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 librados: starting objecter 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 Processor -- start 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- start start 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 librados: setting wanted keys 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 librados: calling monclient init 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: init 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _add_conns ranks=[2,8,7,3,6,5,4,1,0] 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-b con 0x7f14ac0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-e con 0x7f14ac0a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f14ac1396a0 0x7f14ac13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): picked mon.noname-i con 0x7f14ac1396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f14ac143320 con 0x7f14ac0a4910 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f14ac06e630 con 0x7f14ac1396a0 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f14ac1434a0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient(hunting): authenticate will time out at 1259.388809s 2024-09-17T16:32:56.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:48120/0 (socket says 172.21.15.105:48120) 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34532/0 (socket says 172.21.15.105:34532) 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 -- 172.21.15.105:0/3830268139 learned_addr learned my addr 172.21.15.105:0/3830268139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): get_auth_request con 0x7f14ac0a68e0 auth_method 0 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 10 monclient(hunting): get_auth_request con 0x7f14ac0a4910 auth_method 0 2024-09-17T16:32:56.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b2107640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f14ac1396a0 0x7f14ac13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b2107640 10 monclient(hunting): get_auth_request con 0x7f14ac1396a0 auth_method 0 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b2107640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b2107640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b2107640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient(hunting): handle_auth_done global_id 23840 payload 824 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f14ac1396a0 msgr2=0x7f14ac13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f14ac1396a0 0x7f14ac13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 msgr2=0x7f14ac137ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 monclient: found mon.noname-e 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:32:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 -- 172.21.15.105:0/3830268139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac13fcd0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1105640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fd0 tx=0x7f14a000e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 client.?.objecter ms_handle_connect 0x7f14ac0a68e0 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14a000edd0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: dump: 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.150 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a0019070 con 0x7f14ac0a68e0 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14a0014d00 con 0x7f14ac0a68e0 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 monclient: dump: 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14abfff640 4 set_mon_vals no callback set 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 5 monclient: authenticate success, global_id 23840 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _renew_subs 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:32:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac0aa090 con 0x7f14ac0a68e0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 client.23840.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _renew_subs 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac123b70 con 0x7f14ac0a68e0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 librados: init done 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f14a0014420 con 0x7f14ac0a68e0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac139ae0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 0x7f1498040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(210..210 src has 1..210) ==== 7388+0+0 (secure 0 0 0) 0x7f14a0050bd0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 10 client.23840.objecter ms_dispatch 0x7f14ac000ba0 osd_map(210..210 src has 1..210) 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 3 client.23840.objecter handle_osd_map got epochs [210,210] > 0 2024-09-17T16:32:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 3 client.23840.objecter handle_osd_map decoding full epoch 210 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14aa7fc640 20 client.23840.objecter dump_active .. 0 homeless 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 0x7f1498040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.123+0000 7f14b1906640 10 monclient: get_auth_request con 0x7f149803e550 auth_method 0 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.127+0000 7f14b1906640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 0x7f1498040a10 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f14a401c270 tx=0x7f14a4004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.127+0000 7f14aa7fc640 10 client.23840.objecter ms_handle_connect 0x7f149803e550 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.127+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14ac0aa090 con 0x7f14ac0a68e0 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.127+0000 7f14aa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.127+0000 7f14aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:56.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14b3b6a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14b3b6a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14b3b6a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f14ac0746d0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14aa7fc640 1 -- 172.21.15.105:0/3830268139 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f14ac13fcd0 con 0x7f14ac0a68e0 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14aa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:56.372 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":46,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:32:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.367+0000 7f14b3b6a640 10 librados: watch_flush enter 2024-09-17T16:32:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 10 librados: watch_flush exit 2024-09-17T16:32:56.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 20 client.23840.objecter shutdown clearing up homeless session... 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 10 client.23840.objecter successfully canceled tick 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 msgr2=0x7f1498040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 0x7f1498040a10 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f14a401c270 tx=0x7f14a4004740 comp rx=0 tx=0).stop 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 10 monclient: shutdown 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:56.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 msgr2=0x7f14ac138430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fd0 tx=0x7f14a000e4b0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 shutdown_connections 2024-09-17T16:32:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f149803e550 0x7f1498040a10 unknown :-1 s=CLOSED pgs=3426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f14ac1396a0 0x7f14ac13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f14ac0a68e0 0x7f14ac138430 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 --2- 172.21.15.105:0/3830268139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14ac0a4910 0x7f14ac137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 >> 172.21.15.105:0/3830268139 conn(0x7f14ac068ba0 msgr2=0x7f14ac0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 shutdown_connections 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 -- 172.21.15.105:0/3830268139 wait complete. 2024-09-17T16:32:56.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.371+0000 7f14b3b6a640 1 librados: shutdown 2024-09-17T16:32:56.390 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:32:56.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:32:56.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: get_monmap_and_config 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: monmap: 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:56.508706+0000 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:56.508706+0000 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:56.512 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 Processor -- start 2024-09-17T16:32:56.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 -- start start 2024-09-17T16:32:56.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: init 2024-09-17T16:32:56.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient: _add_conns ranks=[7,10,4,9,8,6,0,3,1,2,5] 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb9240a68e0 0x7fb9240a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-i con 0x7fb9240a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb9240a4910 0x7fb9240a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-f con 0x7fb9240a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:32:56.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-c con 0x7fb9241396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb9240a3fc0 con 0x7fb9240a4910 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb9240a63c0 con 0x7fb9241396a0 2024-09-17T16:32:56.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb9240a7740 con 0x7fb9240a68e0 2024-09-17T16:32:56.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb92a031640 10 monclient(hunting): authenticate will time out at 1259.774462s 2024-09-17T16:32:56.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:44984/0 (socket says 172.21.15.105:44984) 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 -- 172.21.15.105:0/2666670772 learned_addr learned my addr 172.21.15.105:0/2666670772 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): get_auth_request con 0x7fb9241396a0 auth_method 0 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb922ffd640 1 --2- 172.21.15.105:0/2666670772 >> v2:172.21.15.160:3300/0 conn(0x7fb9240a4910 0x7fb9240a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb9237fe640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb9240a68e0 0x7fb9240a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb9237fe640 10 monclient(hunting): get_auth_request con 0x7fb9240a68e0 auth_method 0 2024-09-17T16:32:56.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb922ffd640 10 monclient(hunting): get_auth_request con 0x7fb9240a4910 auth_method 0 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb922ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb922ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient(hunting): handle_auth_done global_id 22990 payload 274 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 -- 172.21.15.105:0/2666670772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb9240a68e0 msgr2=0x7fb9240a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb9240a68e0 0x7fb9240a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 -- 172.21.15.105:0/2666670772 >> v2:172.21.15.160:3300/0 conn(0x7fb9240a4910 msgr2=0x7fb9240a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 --2- 172.21.15.105:0/2666670772 >> v2:172.21.15.160:3300/0 conn(0x7fb9240a4910 0x7fb9240a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 monclient: found mon.noname-c 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.507+0000 7fb923fff640 1 -- 172.21.15.105:0/2666670772 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92406ebc0 con 0x7fb9241396a0 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb923fff640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb914009870 tx=0x7fb91401c3f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d573754fc86b4490 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 1 -- 172.21.15.105:0/2666670772 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb9140b2070 con 0x7fb9241396a0 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: dump: 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.524 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.525 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.526 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.526 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.526 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.526 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.526 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 1 -- 172.21.15.105:0/2666670772 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9140ab7b0 con 0x7fb9241396a0 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 5 monclient: authenticate success, global_id 22990 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:56.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 1 -- 172.21.15.105:0/2666670772 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb9140abe50 con 0x7fb9241396a0 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 4 set_mon_vals no callback set 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9227fc640 10 monclient: dump: 2024-09-17T16:32:56.528 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.529 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.530 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.531 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.531 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: shutdown 2024-09-17T16:32:56.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- 172.21.15.105:0/2666670772 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 msgr2=0x7fb924139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb914009870 tx=0x7fb91401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- 172.21.15.105:0/2666670772 shutdown_connections 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9241396a0 0x7fb924139b50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- 172.21.15.105:0/2666670772 >> v2:172.21.15.160:3300/0 conn(0x7fb9240a4910 0x7fb9240a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- 172.21.15.105:0/2666670772 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb9240a68e0 0x7fb9240a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- 172.21.15.105:0/2666670772 >> 172.21.15.105:0/2666670772 conn(0x7fb924068ba0 msgr2=0x7fb924068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- 172.21.15.105:0/2666670772 shutdown_connections 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- 172.21.15.105:0/2666670772 wait complete. 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: monmap: 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.544 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 librados: starting msgr at 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 librados: starting objecter 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 Processor -- start 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- start start 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 librados: setting wanted keys 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 librados: calling monclient init 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: init 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient: _add_conns ranks=[2,0,8,7,4,3,6,1,5] 2024-09-17T16:32:56.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-b con 0x7fb9240a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9240a68e0 0x7fb9241d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-a con 0x7fb9240a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb9241396a0 0x7fb92413d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): picked mon.noname-e con 0x7fb9241396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406e630 con 0x7fb9240a68e0 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240a7af0 con 0x7fb9240a4910 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb924143320 con 0x7fb9241396a0 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb92a031640 10 monclient(hunting): authenticate will time out at 1259.779846s 2024-09-17T16:32:56.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:48134/0 (socket says 172.21.15.105:48134) 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb923fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb9241396a0 0x7fb92413d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 1 -- 172.21.15.105:0/1188588065 learned_addr learned my addr 172.21.15.105:0/1188588065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 10 monclient(hunting): get_auth_request con 0x7fb9240a4910 auth_method 0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.511+0000 7fb9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): get_auth_request con 0x7fb9241396a0 auth_method 0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9240a68e0 0x7fb9241d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): get_auth_request con 0x7fb9240a68e0 auth_method 0 2024-09-17T16:32:56.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 msgr2=0x7fb9241d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 msgr2=0x7fb9241d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:56.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient(hunting): handle_auth_done global_id 34767 payload 824 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb9241396a0 msgr2=0x7fb92413d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb9241396a0 0x7fb92413d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 msgr2=0x7fb9241d1500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9240a4910 0x7fb9241d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 monclient: found mon.noname-a 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb924123650 con 0x7fb9240a68e0 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb922ffd640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9240a68e0 0x7fb9241d1a40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00d5f0 tx=0x7fb90c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.?.objecter ms_handle_connect 0x7fb9240a68e0 2024-09-17T16:32:56.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c004240 con 0x7fb9240a68e0 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: dump: 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.550 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.551 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c004400 con 0x7fb9240a68e0 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c012420 con 0x7fb9240a68e0 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: dump: 2024-09-17T16:32:56.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb921ffb640 4 set_mon_vals no callback set 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 5 monclient: authenticate success, global_id 34767 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 monclient: _renew_subs 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92413f2e0 con 0x7fb9240a68e0 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 client.34767.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 monclient: _renew_subs 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb924123370 con 0x7fb9240a68e0 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 1 librados: init done 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 0x7fb92413ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241225a0 con 0x7fb92413d9d0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 0x7fb92413ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient: get_auth_request con 0x7fb92413d9d0 auth_method 0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb90c01b020 con 0x7fb9240a68e0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 10 monclient(hunting): handle_auth_done global_id 34776 payload 824 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb924139b50 con 0x7fb92413d9d0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb90803e550 0x7fb908040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 7388+0+0 (secure 0 0 0) 0x7fb90c019020 con 0x7fb9240a68e0 2024-09-17T16:32:56.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.34767.objecter ms_dispatch 0x7fb924000ba0 osd_map(210..210 src has 1..210) 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 3 client.34767.objecter handle_osd_map got epochs [210,210] > 0 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 3 client.34767.objecter handle_osd_map decoding full epoch 210 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 0x7fb92413ddb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 20 client.34767.objecter dump_active .. 0 homeless 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.34767.objecter ms_handle_connect 0x7fb92413d9d0 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.34767.objecter resend_mon_ops 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb91c014880 con 0x7fb92413d9d0 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: dump: 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.556 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb90803e550 0x7fb908040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 10 monclient: get_auth_request con 0x7fb90803e550 auth_method 0 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb91c019070 con 0x7fb92413d9d0 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 msgr2=0x7fb92413ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 0x7fb92413ddb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c007700 comp rx=0 tx=0).stop 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 msgr2=0x7fb92413ddb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413d9d0 0x7fb92413ddb0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c007700 comp rx=0 tx=0).stop 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb91bfff640 10 client.34767.objecter ms_handle_connect 0x7fb90803e550 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.515+0000 7fb923fff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb90803e550 0x7fb908040a10 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7fb9140af000 tx=0x7fb9140a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 0x7fb9240a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb924051e80 con 0x7fb92413e2f0 2024-09-17T16:32:56.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 0x7fb9240a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient: get_auth_request con 0x7fb92413e2f0 auth_method 0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 10 monclient(hunting): handle_auth_done global_id 34785 payload 824 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 1 -- 172.21.15.105:0/1188588065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb924123b70 con 0x7fb92413e2f0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 client.34767.objecter ms_handle_connect 0x7fb92413e2f0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb9237fe640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 0x7fb9240a0a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 client.34767.objecter resend_mon_ops 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb91c014cf0 con 0x7fb92413e2f0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 monclient: dump: 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.560 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fb91c014cf0 con 0x7fb92413e2f0 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 msgr2=0x7fb9240a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 0x7fb9240a0a20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c004a50 comp rx=0 tx=0).stop 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 msgr2=0x7fb9240a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:56.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb91bfff640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92413e2f0 0x7fb9240a0a20 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009790 tx=0x7fb91c004a50 comp rx=0 tx=0).stop 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 167326, 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:32:56.562 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:56.563 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:32:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:56.567 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.569 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:32:56.570 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:32:56.572 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.573 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:32:56.575 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:32:56.576 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.578 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.580 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.581 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:32:56.582 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:32:56.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:32:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:56.586 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:56.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:56.586 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:32:56.586 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:32:56.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 librados: watch_flush enter 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 librados: watch_flush exit 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 20 client.34767.objecter shutdown clearing up homeless session... 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 client.34767.objecter successfully canceled tick 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb90803e550 msgr2=0x7fb908040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb90803e550 0x7fb908040a10 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7fb9140af000 tx=0x7fb9140a7070 comp rx=0 tx=0).stop 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 10 monclient: shutdown 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9240a68e0 msgr2=0x7fb9241d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 --2- 172.21.15.105:0/1188588065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9240a68e0 0x7fb9241d1a40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00d5f0 tx=0x7fb90c00dab0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb923fff640 1 -- 172.21.15.105:0/1188588065 reap_dead start 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb923fff640 1 -- 172.21.15.105:0/1188588065 reap_dead start 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 shutdown_connections 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 >> 172.21.15.105:0/1188588065 conn(0x7fb924068ba0 msgr2=0x7fb9240a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:56.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 shutdown_connections 2024-09-17T16:32:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 -- 172.21.15.105:0/1188588065 wait complete. 2024-09-17T16:32:56.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.523+0000 7fb92a031640 1 librados: shutdown 2024-09-17T16:32:56.589 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:32:56.591 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:32:56.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: get_monmap_and_config 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: monmap: 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:56.712886+0000 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:56.712886+0000 2024-09-17T16:32:56.715 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.716 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.717 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.717 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.717 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:56.717 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:56.717 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 Processor -- start 2024-09-17T16:32:56.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 -- start start 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: init 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient: _add_conns ranks=[6,7,10,3,4,5,1,2,9,0,8] 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-d con 0x7f21f40a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 0x7f21f40a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-i con 0x7f21f40a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f21f41396a0 0x7f21f4139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-f con 0x7f21f41396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f21f40a3fc0 con 0x7f21f41396a0 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f21f40a63c0 con 0x7f21f40a68e0 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f21f40a7740 con 0x7f21f40a4910 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f9a5b640 10 monclient(hunting): authenticate will time out at 1259.978575s 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:41514/0 (socket says 172.21.15.105:41514) 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 -- 172.21.15.105:0/2758944391 learned_addr learned my addr 172.21.15.105:0/2758944391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): get_auth_request con 0x7f21f40a68e0 auth_method 0 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 0x7f21f40a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 10 monclient(hunting): get_auth_request con 0x7f21f40a4910 auth_method 0 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f37fe640 1 --2- 172.21.15.105:0/2758944391 >> v2:172.21.15.160:3300/0 conn(0x7f21f41396a0 0x7f21f4139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f37fe640 10 monclient(hunting): get_auth_request con 0x7f21f41396a0 auth_method 0 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_done global_id 23856 payload 274 2024-09-17T16:32:56.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 -- 172.21.15.105:0/2758944391 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 msgr2=0x7f21f40a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 0x7f21f40a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 -- 172.21.15.105:0/2758944391 >> v2:172.21.15.160:3300/0 conn(0x7f21f41396a0 msgr2=0x7f21f4139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/2758944391 >> v2:172.21.15.160:3300/0 conn(0x7f21f41396a0 0x7f21f4139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 monclient: found mon.noname-d 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f2ffd640 1 -- 172.21.15.105:0/2758944391 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f406ebc0 con 0x7f21f40a68e0 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.711+0000 7f21f27fc640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 0x7f21f40a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009870 tx=0x7f21e801c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c33c842fe725dcfb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 1 -- 172.21.15.105:0/2758944391 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e80b0070 con 0x7f21f40a68e0 2024-09-17T16:32:56.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: dump: 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.731 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 1 -- 172.21.15.105:0/2758944391 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e80aa810 con 0x7f21f40a68e0 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 5 monclient: authenticate success, global_id 23856 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 1 -- 172.21.15.105:0/2758944391 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e80a9300 con 0x7f21f40a68e0 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 4 set_mon_vals no callback set 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f1ffb640 10 monclient: dump: 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.733 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.734 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 10 monclient: shutdown 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- 172.21.15.105:0/2758944391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 msgr2=0x7f21f40a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009870 tx=0x7f21e801c3f0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- 172.21.15.105:0/2758944391 shutdown_connections 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/2758944391 >> v2:172.21.15.160:3300/0 conn(0x7f21f41396a0 0x7f21f4139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21f40a4910 0x7f21f40a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/2758944391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a68e0 0x7f21f40a6cc0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- 172.21.15.105:0/2758944391 >> 172.21.15.105:0/2758944391 conn(0x7f21f4068ba0 msgr2=0x7f21f4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- 172.21.15.105:0/2758944391 shutdown_connections 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- 172.21.15.105:0/2758944391 wait complete. 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 10 monclient: build_initial_monmap 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 10 monclient: monmap: 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 librados: starting msgr at 2024-09-17T16:32:56.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 librados: starting objecter 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 Processor -- start 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 -- start start 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 librados: setting wanted keys 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 1 librados: calling monclient init 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.715+0000 7f21f9a5b640 10 monclient: init 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _add_conns ranks=[6,4,5,8,2,1,0,3,7] 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 0x7f21f4137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-d con 0x7f21f40a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f21f40a68e0 0x7f21f4137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-c con 0x7f21f40a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f21f41396a0 0x7f21f4138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): picked mon.noname-h con 0x7f21f41396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:32:56.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f21f406e630 con 0x7f21f40a68e0 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f21f40a7af0 con 0x7f21f41396a0 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f21f4143320 con 0x7f21f40a4910 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): authenticate will time out at 1259.984517s 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 0x7f21f4137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 0x7f21f4137a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:41516/0 (socket says 172.21.15.105:41516) 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 -- 172.21.15.105:0/1950190694 learned_addr learned my addr 172.21.15.105:0/1950190694 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): get_auth_request con 0x7f21f40a4910 auth_method 0 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f21f40a68e0 0x7f21f4137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): get_auth_request con 0x7f21f40a68e0 auth_method 0 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient(hunting): handle_auth_done global_id 23865 payload 824 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f37fe640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f21f41396a0 0x7f21f4138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f21f41396a0 msgr2=0x7f21f4138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f21f41396a0 0x7f21f4138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f21f40a68e0 msgr2=0x7f21f4137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f21f40a68e0 0x7f21f4137f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 monclient: found mon.noname-d 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40a9f70 con 0x7f21f40a4910 2024-09-17T16:32:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f2ffd640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 0x7f21f4137a10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f21e80ae040 tx=0x7f21e8019190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 client.?.objecter ms_handle_connect 0x7f21f40a4910 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e80b9070 con 0x7f21f40a4910 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:32:56.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: dump: 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.744 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: _finish_auth 0 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: _check_auth_tickets 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e80044b0 con 0x7f21f40a4910 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e80b0040 con 0x7f21f40a4910 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 monclient: dump: 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:56.747 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f17fa640 4 set_mon_vals no callback set 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 5 monclient: authenticate success, global_id 23865 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _renew_subs 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f4123d70 con 0x7f21f40a4910 2024-09-17T16:32:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 client.23865.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _renew_subs 2024-09-17T16:32:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:32:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f413f0b0 con 0x7f21f40a4910 2024-09-17T16:32:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 librados: init done 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f21f4051e80 con 0x7f21f40a0a20 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a0a20 auth_method 0 2024-09-17T16:32:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:32:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f21e8004040 con 0x7f21f40a4910 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21e003e550 0x7f21e0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(211..211 src has 1..211) ==== 7495+0+0 (secure 0 0 0) 0x7f21e80e9950 con 0x7f21f40a4910 2024-09-17T16:32:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 10 client.23865.objecter ms_dispatch 0x7f21f4000ba0 osd_map(211..211 src has 1..211) 2024-09-17T16:32:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 3 client.23865.objecter handle_osd_map got epochs [211,211] > 0 2024-09-17T16:32:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 3 client.23865.objecter handle_osd_map decoding full epoch 211 2024-09-17T16:32:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21dbfff640 20 client.23865.objecter dump_active .. 0 homeless 2024-09-17T16:32:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f37fe640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21e003e550 0x7f21e0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.719+0000 7f21f37fe640 10 monclient: get_auth_request con 0x7f21e003e550 auth_method 0 2024-09-17T16:32:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.723+0000 7f21dbfff640 10 client.23865.objecter ms_handle_connect 0x7f21e003e550 2024-09-17T16:32:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.723+0000 7f21f37fe640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21e003e550 0x7f21e0040a10 secure :-1 s=READY pgs=3428 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009e10 tx=0x7f21ec00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a0a20 auth_method 0 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:56.923+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:32:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a0a20 auth_method 0 2024-09-17T16:32:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:57.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:32:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:57.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:57.327+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:32:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a0a20 auth_method 0 2024-09-17T16:32:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:32:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:58.131+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:32:59.559 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (21138 ms) 2024-09-17T16:32:59.559 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (11808 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7019 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7020 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (6970 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7041 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (6976 ms) 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-17T16:32:59.560 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7022 ms) 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6982 ms) 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7011 ms) 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7047 ms) 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6945 ms) 2024-09-17T16:32:59.561 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7034 ms) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6983 ms) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (24987 ms) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7018 ms) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (149001 ms total) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [----------] Global test environment tear-down 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (238888 ms total) 2024-09-17T16:32:59.562 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio: [ PASSED ] 42 tests. 2024-09-17T16:32:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.719+0000 7f21daffd640 10 monclient: tick 2024-09-17T16:32:59.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.719+0000 7f21daffd640 10 monclient: _check_auth_tickets 2024-09-17T16:32:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.719+0000 7f21daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:59.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.719+0000 7f21daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a0a20 auth_method 0 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 10 monclient(hunting): handle_auth_done global_id 44102 payload 824 2024-09-17T16:32:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21f40abb20 con 0x7f21f40a0a20 2024-09-17T16:32:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e4019c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 client.23865.objecter ms_handle_connect 0x7f21f40a0a20 2024-09-17T16:32:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 client.23865.objecter resend_mon_ops 2024-09-17T16:32:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e4026070 con 0x7f21f40a0a20 2024-09-17T16:32:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 monclient: dump: 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f21e4019e50 con 0x7f21f40a0a20 2024-09-17T16:32:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e4019c60 comp rx=0 tx=0).stop 2024-09-17T16:32:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 msgr2=0x7f21f40a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.735+0000 7f21dbfff640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a0a20 0x7f21f40a0e00 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e4019c60 comp rx=0 tx=0).stop 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f9a5b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 0x7f21f413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f9a5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f21f4123660 con 0x7f21f40a1340 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 0x7f21f413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient: get_auth_request con 0x7f21f40a1340 auth_method 0 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 10 monclient(hunting): handle_auth_done global_id 44111 payload 824 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 1 -- 172.21.15.105:0/1950190694 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f21f4139ae0 con 0x7f21f40a1340 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 10 client.23865.objecter ms_handle_connect 0x7f21f40a1340 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 10 client.23865.objecter resend_mon_ops 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21f27fc640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 0x7f21f413d700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e40046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21e4026040 con 0x7f21f40a1340 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.743+0000 7f21dbfff640 10 monclient: dump: 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.753 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f21e4028620 con 0x7f21f40a1340 2024-09-17T16:32:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:32:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 msgr2=0x7f21f413d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 0x7f21f413d700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e40046d0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 msgr2=0x7f21f413d700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21dbfff640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21f40a1340 0x7f21f413d700 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21f4074500 tx=0x7f21e40046d0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:32:59.755 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 30399, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.756 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.757 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.758 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:32:59.759 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.760 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.761 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.762 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.764 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:32:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.767 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:32:59.768 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.769 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.771 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.772 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.773 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.774 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:32:59.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.777 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:32:59.778 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 10 librados: watch_flush enter 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 10 librados: watch_flush exit 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 20 client.23865.objecter shutdown clearing up homeless session... 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 10 client.23865.objecter successfully canceled tick 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21e003e550 msgr2=0x7f21e0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21e003e550 0x7f21e0040a10 secure :-1 s=READY pgs=3428 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009e10 tx=0x7f21ec00b040 comp rx=0 tx=0).stop 2024-09-17T16:32:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 10 monclient: shutdown 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 msgr2=0x7f21f4137a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 --2- 172.21.15.105:0/1950190694 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f21f40a4910 0x7f21f4137a10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f21e80ae040 tx=0x7f21e8019190 comp rx=0 tx=0).stop 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f37fe640 1 -- 172.21.15.105:0/1950190694 reap_dead start 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f37fe640 1 -- 172.21.15.105:0/1950190694 reap_dead start 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 shutdown_connections 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 >> 172.21.15.105:0/1950190694 conn(0x7f21f4068ba0 msgr2=0x7f21f40a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 shutdown_connections 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 -- 172.21.15.105:0/1950190694 wait complete. 2024-09-17T16:32:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.747+0000 7f21f9a5b640 1 librados: shutdown 2024-09-17T16:32:59.781 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:32:59.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:32:59.782 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:32:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.899+0000 7f654f335640 10 monclient: get_monmap_and_config 2024-09-17T16:32:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.899+0000 7f654f335640 10 monclient: build_initial_monmap 2024-09-17T16:32:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient: monmap: 2024-09-17T16:32:59.930 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:32:59.903784+0000 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:32:59.903784+0000 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.931 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:32:59.932 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:32:59.932 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:59.932 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:59.932 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:59.932 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 Processor -- start 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 -- start start 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient: init 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient: _add_conns ranks=[6,4,5,8,2,7,9,10,0,1,3] 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): picked mon.noname-d con 0x7f65480a2620 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 0x7f6548137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): picked mon.noname-c con 0x7f65480a2f40 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 0x7f6548137d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): picked mon.noname-h con 0x7f6548137940 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f654806e580 con 0x7f65480a2f40 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f654806f2e0 con 0x7f6548137940 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f65480a15e0 con 0x7f65480a2620 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654f335640 10 monclient(hunting): authenticate will time out at 1263.169450s 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36436/0 (socket says 172.21.15.105:36436) 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 -- 172.21.15.105:0/3795063626 learned_addr learned my addr 172.21.15.105:0/3795063626 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): get_auth_request con 0x7f65480a2620 auth_method 0 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 0x7f6548137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 10 monclient(hunting): get_auth_request con 0x7f65480a2f40 auth_method 0 2024-09-17T16:32:59.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d8d2640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 0x7f6548137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient(hunting): handle_auth_done global_id 23874 payload 274 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 -- 172.21.15.105:0/3795063626 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 msgr2=0x7f6548137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 0x7f6548137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 -- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 msgr2=0x7f6548137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 0x7f6548137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient: _finish_hunting 0 2024-09-17T16:32:59.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 monclient: found mon.noname-d 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d0d1640 1 -- 172.21.15.105:0/3795063626 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65480a3c30 con 0x7f65480a2620 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d8d2640 10 monclient: get_auth_request con 0x7f6548137940 auth_method 0 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654c8d0640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 0x7f6548137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.903+0000 7f654d8d2640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 0x7f6548137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65440098a0 tx=0x7f654401c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=83db9232794a6ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 1 -- 172.21.15.105:0/3795063626 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65440b0070 con 0x7f65480a2620 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:32:59.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: dump: 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.940 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: _finish_auth 0 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 5 monclient: authenticate success, global_id 23874 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 1 -- 172.21.15.105:0/3795063626 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65440aa810 con 0x7f65480a2620 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 1 -- 172.21.15.105:0/3795063626 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65440a9300 con 0x7f65480a2620 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: get_monmap_and_config success 2024-09-17T16:32:59.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 4 set_mon_vals no callback set 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653ffff640 10 monclient: dump: 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: shutdown 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 msgr2=0x7f65480a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65440098a0 tx=0x7f654401c440 comp rx=0 tx=0).stop 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- 172.21.15.105:0/3795063626 shutdown_connections 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6548137940 0x7f6548137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f65480a2f40 0x7f6548137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- 172.21.15.105:0/3795063626 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65480a2620 0x7f65480a2a00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- 172.21.15.105:0/3795063626 >> 172.21.15.105:0/3795063626 conn(0x7f6548068ba0 msgr2=0x7f6548068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- 172.21.15.105:0/3795063626 shutdown_connections 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- 172.21.15.105:0/3795063626 wait complete. 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: build_initial_monmap 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: monmap: 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:32:59.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 librados: starting msgr at 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 librados: starting objecter 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 Processor -- start 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- start start 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 librados: setting wanted keys 2024-09-17T16:32:59.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 librados: calling monclient init 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: init 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: _reopen_session rank -1 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient: _add_conns ranks=[7,3,1,0,5,6,2,4,8] 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 0x7f654813d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): picked mon.noname-i con 0x7f65480a2620 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65480a2f40 0x7f654809f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): picked mon.noname-g con 0x7f65480a2f40 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6548137940 0x7f654809f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): picked mon.noname-f con 0x7f6548137940 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f65480a1f50 con 0x7f6548137940 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f65480a1dd0 con 0x7f65480a2f40 2024-09-17T16:32:59.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6548141500 con 0x7f65480a2620 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): _renew_subs 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654f335640 10 monclient(hunting): authenticate will time out at 1263.174615s 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 0x7f654813d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6548137940 0x7f654809f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65480a2f40 0x7f654809f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 0x7f654813d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:40124/0 (socket says 172.21.15.105:40124) 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 -- 172.21.15.105:0/1733271027 learned_addr learned my addr 172.21.15.105:0/1733271027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): get_auth_request con 0x7f65480a2620 auth_method 0 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): get_auth_request con 0x7f65480a2f40 auth_method 0 2024-09-17T16:32:59.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): get_auth_request con 0x7f6548137940 auth_method 0 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d8d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient(hunting): handle_auth_done global_id 34468 payload 824 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65480a2f40 msgr2=0x7f654809f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65480a2f40 0x7f654809f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6548137940 msgr2=0x7f654809f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6548137940 0x7f654809f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient: _finish_hunting 0 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 monclient: found mon.noname-i 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65480a03f0 con 0x7f65480a2620 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f654d0d1640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 0x7f654813d0b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65440ae040 tx=0x7f65440191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653dffb640 10 client.?.objecter ms_handle_connect 0x7f65480a2620 2024-09-17T16:32:59.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.907+0000 7f653dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65440b9070 con 0x7f65480a2620 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: dump: 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.950 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: _finish_auth 0 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 5 monclient: authenticate success, global_id 34468 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 monclient: _renew_subs 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:32:59.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6548122bd0 con 0x7f65480a2620 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65440aad60 con 0x7f65480a2620 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 client.34468.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 monclient: _renew_subs 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f65480a0f50 con 0x7f65480a2620 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 1 librados: init done 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653f7fe640 4 set_mon_vals no callback set 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65440b0040 con 0x7f65480a2620 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:32:59.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: dump: 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.953 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 0x7f6548135ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f654809e660 con 0x7f654809fc10 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 0x7f6548135ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient: get_auth_request con 0x7f654809fc10 auth_method 0 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 10 monclient(hunting): handle_auth_done global_id 23017 payload 824 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65480a0890 con 0x7f654809fc10 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 client.34468.objecter ms_handle_connect 0x7f654809fc10 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 client.34468.objecter resend_mon_ops 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654c8d0640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 0x7f6548135ee0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6538006e90 tx=0x7f6538011a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6544004040 con 0x7f65480a2620 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652c03e550 0x7f652c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(214..214 src has 1..214) ==== 6620+0+0 (secure 0 0 0) 0x7f65440e9780 con 0x7f65480a2620 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 client.34468.objecter ms_dispatch 0x7f6548000ba0 osd_map(214..214 src has 1..214) 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 3 client.34468.objecter handle_osd_map got epochs [214,214] > 0 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 3 client.34468.objecter handle_osd_map decoding full epoch 214 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 20 client.34468.objecter dump_active .. 0 homeless 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6538019070 con 0x7f654809fc10 2024-09-17T16:32:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: dump: 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.956 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f653800fc40 con 0x7f654809fc10 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 msgr2=0x7f6548135ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 0x7f6548135ee0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6538006e90 tx=0x7f6538011a40 comp rx=0 tx=0).stop 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 msgr2=0x7f6548135ee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654809fc10 0x7f6548135ee0 secure :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6538006e90 tx=0x7f6538011a40 comp rx=0 tx=0).stop 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654d8d2640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652c03e550 0x7f652c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654d8d2640 10 monclient: get_auth_request con 0x7f652c03e550 auth_method 0 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f654d8d2640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652c03e550 0x7f652c040a10 secure :-1 s=READY pgs=3429 cs=0 l=1 rev1=1 crypto rx=0x7f6548072ff0 tx=0x7f6540007700 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.911+0000 7f653dffb640 10 client.34468.objecter ms_handle_connect 0x7f652c03e550 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 0x7f654802d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 monclient(hunting): start opening mon connection 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6548051e80 con 0x7f654802d450 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 0x7f654802d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient: get_auth_request con 0x7f654802d450 auth_method 0 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:32:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 10 monclient(hunting): handle_auth_done global_id 23026 payload 824 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 1 -- 172.21.15.105:0/1733271027 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f654809fff0 con 0x7f654802d450 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 client.34468.objecter ms_handle_connect 0x7f654802d450 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654c8d0640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 0x7f654802d860 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f653800f200 tx=0x7f6538004740 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 client.34468.objecter resend_mon_ops 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f653800f040 con 0x7f654802d450 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 monclient: dump: 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:32:59.959 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:32:59.960 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f6538015cc0 con 0x7f654802d450 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 msgr2=0x7f654802d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 0x7f654802d860 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f653800f200 tx=0x7f6538004740 comp rx=0 tx=0).stop 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 msgr2=0x7f654802d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f653dffb640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f654802d450 0x7f654802d860 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:32:59.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 327558, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:32:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:59.965 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:32:59.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:32:59.967 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.970 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:32:59.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.975 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.977 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.978 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:32:59.980 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:32:59.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:32:59.982 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:32:59.983 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:32:59.984 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 librados: watch_flush enter 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 librados: watch_flush exit 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 20 client.34468.objecter shutdown clearing up homeless session... 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 client.34468.objecter successfully canceled tick 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652c03e550 msgr2=0x7f652c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652c03e550 0x7f652c040a10 secure :-1 s=READY pgs=3429 cs=0 l=1 rev1=1 crypto rx=0x7f6548072ff0 tx=0x7f6540007700 comp rx=0 tx=0).stop 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 10 monclient: shutdown 2024-09-17T16:32:59.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 msgr2=0x7f654813d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 --2- 172.21.15.105:0/1733271027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65480a2620 0x7f654813d0b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f65440ae040 tx=0x7f65440191e0 comp rx=0 tx=0).stop 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654d8d2640 1 -- 172.21.15.105:0/1733271027 reap_dead start 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654d8d2640 1 -- 172.21.15.105:0/1733271027 reap_dead start 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 shutdown_connections 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 >> 172.21.15.105:0/1733271027 conn(0x7f6548068ba0 msgr2=0x7f6548071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 shutdown_connections 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 -- 172.21.15.105:0/1733271027 wait complete. 2024-09-17T16:32:59.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:32:59.919+0000 7f654f335640 1 librados: shutdown 2024-09-17T16:32:59.987 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:32:59.988 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:32:59.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:33:00.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: get_monmap_and_config 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: monmap: 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:00.104349+0000 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:00.104349+0000 2024-09-17T16:33:00.107 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.108 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.109 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.109 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.109 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:00.109 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:00.109 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 Processor -- start 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 -- start start 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: init 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient: _add_conns ranks=[8,1,5,6,7,2,0,4,3,10,9] 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-e con 0x7f97740a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:00.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f977409e740 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f977409e0b0 0x7f97740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-h con 0x7f977409e0b0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97740a0a80 con 0x7f977409e740 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f977409ef60 con 0x7f977409e0b0 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f97740a31c0 con 0x7f97740a4750 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f977b89b640 10 monclient(hunting): authenticate will time out at 1263.370155s 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36486/0 (socket says 172.21.15.105:36486) 2024-09-17T16:33:00.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 -- 172.21.15.105:0/1713044827 learned_addr learned my addr 172.21.15.105:0/1713044827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): get_auth_request con 0x7f97740a4750 auth_method 0 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779e38640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f977409e0b0 0x7f97740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779e38640 10 monclient(hunting): get_auth_request con 0x7f977409e0b0 auth_method 0 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779e38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779e38640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779e38640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient(hunting): handle_auth_done global_id 23858 payload 274 2024-09-17T16:33:00.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 -- 172.21.15.105:0/1713044827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f977409e0b0 msgr2=0x7f97740a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f977409e0b0 0x7f97740a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 -- 172.21.15.105:0/1713044827 >> v1:172.21.15.160:6789/0 conn(0x7f977409e740 legacy=0x7f9774136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 monclient: found mon.noname-e 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.103+0000 7f9779637640 1 -- 172.21.15.105:0/1713044827 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97740a6710 con 0x7f97740a4750 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f97700098a0 tx=0x7f977001c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=32adec89265131e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 1 -- 172.21.15.105:0/1713044827 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97700b0070 con 0x7f97740a4750 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: dump: 2024-09-17T16:33:00.127 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.128 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 1 -- 172.21.15.105:0/1713044827 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97700aa810 con 0x7f97740a4750 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 5 monclient: authenticate success, global_id 23858 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 1 -- 172.21.15.105:0/1713044827 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97700a9300 con 0x7f97740a4750 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 4 set_mon_vals no callback set 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f976bfff640 10 monclient: dump: 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.130 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: shutdown 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- 172.21.15.105:0/1713044827 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 msgr2=0x7f97740a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f97700098a0 tx=0x7f977001c440 comp rx=0 tx=0).stop 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- 172.21.15.105:0/1713044827 shutdown_connections 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f977409e0b0 0x7f97740a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- 172.21.15.105:0/1713044827 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f97740a4b30 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- 172.21.15.105:0/1713044827 >> 172.21.15.105:0/1713044827 conn(0x7f97740a5840 msgr2=0x7f97740a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- 172.21.15.105:0/1713044827 shutdown_connections 2024-09-17T16:33:00.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- 172.21.15.105:0/1713044827 wait complete. 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: monmap: 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.132 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 librados: starting msgr at 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 librados: starting objecter 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 Processor -- start 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- start start 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 librados: setting wanted keys 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 librados: calling monclient init 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: init 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient: _add_conns ranks=[3,1,8,2,0,4,7,6,5] 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f977409e0b0 0x7f9774073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-g con 0x7f977409e0b0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f977409e740 0x7f977413b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-f con 0x7f977409e740 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f977413be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): picked mon.noname-e con 0x7f97740a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f977406c4a0 con 0x7f977409e740 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f977406c320 con 0x7f977409e0b0 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f977406c620 con 0x7f97740a4750 2024-09-17T16:33:00.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f977b89b640 10 monclient(hunting): authenticate will time out at 1263.375218s 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f977413be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f977413be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36496/0 (socket says 172.21.15.105:36496) 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 learned_addr learned my addr 172.21.15.105:0/2374666207 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): get_auth_request con 0x7f97740a4750 auth_method 0 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9778e36640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f977409e740 0x7f977413b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f977409e0b0 0x7f9774073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 10 monclient(hunting): get_auth_request con 0x7f977409e0b0 auth_method 0 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779637640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9778e36640 10 monclient(hunting): get_auth_request con 0x7f977409e740 auth_method 0 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9778e36640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9778e36640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9778e36640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient(hunting): handle_auth_done global_id 23867 payload 824 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f977409e0b0 msgr2=0x7f9774073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f977409e0b0 0x7f9774073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f977409e740 msgr2=0x7f977413b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f977409e740 0x7f977413b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 monclient: found mon.noname-e 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9774142180 con 0x7f97740a4750 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9779e38640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f977413be60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f976c007e70 tx=0x7f976c00e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9769ffb640 10 client.?.objecter ms_handle_connect 0x7f97740a4750 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.107+0000 7f9769ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f976c00ee90 con 0x7f97740a4750 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: dump: 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.137 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.138 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c019070 con 0x7f97740a4750 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 5 monclient: authenticate success, global_id 23867 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 monclient: _renew_subs 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9774142460 con 0x7f97740a4750 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f976c014d00 con 0x7f97740a4750 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: dump: 2024-09-17T16:33:00.139 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f976b7fe640 4 set_mon_vals no callback set 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 client.23867.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 monclient: _renew_subs 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f97740a5e40 con 0x7f97740a4750 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 1 librados: init done 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 0x7f97741329f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9774051e80 con 0x7f977413c3a0 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 0x7f97741329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient: get_auth_request con 0x7f977413c3a0 auth_method 0 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f976c014420 con 0x7f97740a4750 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 10 monclient(hunting): handle_auth_done global_id 23883 payload 824 2024-09-17T16:33:00.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9774136500 con 0x7f977413c3a0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f975803e720 0x7f9758040be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(214..214 src has 1..214) ==== 6620+0+0 (secure 0 0 0) 0x7f976c050930 con 0x7f97740a4750 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 client.23867.objecter ms_dispatch 0x7f9774000ba0 osd_map(214..214 src has 1..214) 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 3 client.23867.objecter handle_osd_map got epochs [214,214] > 0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 3 client.23867.objecter handle_osd_map decoding full epoch 214 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9779637640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 0x7f97741329f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f97740a6680 tx=0x7f97700191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 20 client.23867.objecter dump_active .. 0 homeless 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 client.23867.objecter ms_handle_connect 0x7f977413c3a0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 client.23867.objecter resend_mon_ops 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9778e36640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f975803e720 0x7f9758040be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9778e36640 10 monclient: get_auth_request con 0x7f975803e720 auth_method 0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97700ba030 con 0x7f977413c3a0 2024-09-17T16:33:00.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: dump: 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.144 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.145 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9770003850 con 0x7f977413c3a0 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 msgr2=0x7f97741329f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 0x7f97741329f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f97740a6680 tx=0x7f97700191e0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 msgr2=0x7f97741329f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f977413c3a0 0x7f97741329f0 secure :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f97740a6680 tx=0x7f97700191e0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9769ffb640 10 client.23867.objecter ms_handle_connect 0x7f975803e720 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.111+0000 7f9778e36640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f975803e720 0x7f9758040be0 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 crypto rx=0x7f9774136150 tx=0x7f9764009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 0x7f9774133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f977413d470 con 0x7f9774132f30 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 0x7f9774133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient: get_auth_request con 0x7f9774132f30 auth_method 0 2024-09-17T16:33:00.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 10 monclient(hunting): handle_auth_done global_id 23892 payload 824 2024-09-17T16:33:00.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 1 -- 172.21.15.105:0/2374666207 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9774132580 con 0x7f9774132f30 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779637640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 0x7f9774133340 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f97700097c0 tx=0x7f97700a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 client.23867.objecter ms_handle_connect 0x7f9774132f30 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 client.23867.objecter resend_mon_ops 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97700ba040 con 0x7f9774132f30 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 monclient: dump: 2024-09-17T16:33:00.149 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.150 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f97700c1c60 con 0x7f9774132f30 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:00.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:00.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 msgr2=0x7f9774133340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 0x7f9774133340 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f97700097c0 tx=0x7f97700a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:00.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 msgr2=0x7f9774133340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9769ffb640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9774132f30 0x7f9774133340 secure :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f97700097c0 tx=0x7f97700a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:00.152 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 327758, 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:00.153 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:00.154 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.155 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.156 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.157 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.159 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.160 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:00.161 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.162 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.164 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.165 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.169 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.170 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:00.171 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:00.173 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.175 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.176 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.177 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.178 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:00.178 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:00.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 librados: watch_flush enter 2024-09-17T16:33:00.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 librados: watch_flush exit 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 20 client.23867.objecter shutdown clearing up homeless session... 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 client.23867.objecter successfully canceled tick 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f975803e720 msgr2=0x7f9758040be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f975803e720 0x7f9758040be0 secure :-1 s=READY pgs=3430 cs=0 l=1 rev1=1 crypto rx=0x7f9774136150 tx=0x7f9764009040 comp rx=0 tx=0).stop 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 10 monclient: shutdown 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 msgr2=0x7f977413be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 --2- 172.21.15.105:0/2374666207 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f97740a4750 0x7f977413be60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f976c007e70 tx=0x7f976c00e4b0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 reap_dead start 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f9779e38640 1 -- 172.21.15.105:0/2374666207 reap_dead start 2024-09-17T16:33:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 shutdown_connections 2024-09-17T16:33:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 >> 172.21.15.105:0/2374666207 conn(0x7f97740a5840 msgr2=0x7f97741316b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 shutdown_connections 2024-09-17T16:33:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 -- 172.21.15.105:0/2374666207 wait complete. 2024-09-17T16:33:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.119+0000 7f977b89b640 1 librados: shutdown 2024-09-17T16:33:00.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:00.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:33:00.181 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: get_monmap_and_config 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: monmap: 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:00.300555+0000 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:00.300555+0000 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:00.303 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:00.304 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 Processor -- start 2024-09-17T16:33:00.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 -- start start 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: init 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient: _add_conns ranks=[1,10,9,8,5,0,7,3,2,6,4] 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f406c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f406c0a4910 0x7f406c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): picked mon.noname-f con 0x7f406c0a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f406c0a73d0 0x7f406c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): picked mon.noname-a con 0x7f406c0a73d0 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c06ebc0 con 0x7f406c0a68e0 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f406c0a3fc0 con 0x7f406c0a73d0 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f406c0a63c0 con 0x7f406c0a4910 2024-09-17T16:33:00.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f407160a640 10 monclient(hunting): authenticate will time out at 1263.566335s 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f406c0a73d0 0x7f406c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f406c0a73d0 0x7f406c141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49536/0 (socket says 172.21.15.105:49536) 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 -- 172.21.15.105:0/2369174326 learned_addr learned my addr 172.21.15.105:0/2369174326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): get_auth_request con 0x7f406c0a73d0 auth_method 0 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f40627fc640 1 --2- 172.21.15.105:0/2369174326 >> v2:172.21.15.160:3300/0 conn(0x7f406c0a4910 0x7f406c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f40627fc640 10 monclient(hunting): get_auth_request con 0x7f406c0a4910 auth_method 0 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f40627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f40627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f40627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient(hunting): handle_auth_done global_id 34794 payload 274 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 -- 172.21.15.105:0/2369174326 >> v2:172.21.15.160:3300/0 conn(0x7f406c0a4910 msgr2=0x7f406c13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 --2- 172.21.15.105:0/2369174326 >> v2:172.21.15.160:3300/0 conn(0x7f406c0a4910 0x7f406c13eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 -- 172.21.15.105:0/2369174326 >> v1:172.21.15.160:6789/0 conn(0x7f406c0a68e0 legacy=0x7f406c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 monclient: found mon.noname-a 2024-09-17T16:33:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.299+0000 7f406b7fe640 1 -- 172.21.15.105:0/2369174326 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0a3690 con 0x7f406c0a73d0 2024-09-17T16:33:00.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 1 --2- 172.21.15.105:0/2369174326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c141040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f406c074500 tx=0x7f406401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6284944452709ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 1 -- 172.21.15.105:0/2369174326 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40640090d0 con 0x7f406c0a73d0 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: dump: 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 1 -- 172.21.15.105:0/2369174326 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40640a5b00 con 0x7f406c0a73d0 2024-09-17T16:33:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 5 monclient: authenticate success, global_id 34794 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 1 -- 172.21.15.105:0/2369174326 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40640a5cf0 con 0x7f406c0a73d0 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 4 set_mon_vals no callback set 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406a7fc640 10 monclient: dump: 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.323 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.324 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: shutdown 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- 172.21.15.105:0/2369174326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 msgr2=0x7f406c141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- 172.21.15.105:0/2369174326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c141040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f406c074500 tx=0x7f406401c880 comp rx=0 tx=0).stop 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- 172.21.15.105:0/2369174326 shutdown_connections 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- 172.21.15.105:0/2369174326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c141040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- 172.21.15.105:0/2369174326 >> v2:172.21.15.160:3300/0 conn(0x7f406c0a4910 0x7f406c13eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- 172.21.15.105:0/2369174326 >> 172.21.15.105:0/2369174326 conn(0x7f406c068ba0 msgr2=0x7f406c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- 172.21.15.105:0/2369174326 shutdown_connections 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- 172.21.15.105:0/2369174326 wait complete. 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: monmap: 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.325 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.326 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 librados: starting msgr at 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 librados: starting objecter 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 Processor -- start 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- start start 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 librados: setting wanted keys 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 librados: calling monclient init 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: init 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient: _add_conns ranks=[2,3,0,4,7,1,5,8,6] 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 0x7f406c1d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): picked mon.noname-b con 0x7f406c0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:00.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f406c0a68e0 0x7f406c1d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): picked mon.noname-g con 0x7f406c0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c1d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): picked mon.noname-a con 0x7f406c0a73d0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f406c143f20 con 0x7f406c0a73d0 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f406c143da0 con 0x7f406c0a4910 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f406c1440a0 con 0x7f406c0a68e0 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f407160a640 10 monclient(hunting): authenticate will time out at 1263.571673s 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 0x7f406c1d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c1d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 0x7f406c1d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53028/0 (socket says 172.21.15.105:53028) 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 1 -- 172.21.15.105:0/2755620156 learned_addr learned my addr 172.21.15.105:0/2755620156 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): get_auth_request con 0x7f406c0a73d0 auth_method 0 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): get_auth_request con 0x7f406c0a4910 auth_method 0 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.303+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f406c0a68e0 0x7f406c1d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 10 monclient(hunting): handle_auth_done global_id 34803 payload 824 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f406c0a68e0 msgr2=0x7f406c1d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f406c0a68e0 0x7f406c1d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 msgr2=0x7f406c1d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 0x7f406c1d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 monclient: found mon.noname-a 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c1292e0 con 0x7f406c0a73d0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 10 monclient: get_auth_request con 0x7f406c0a68e0 auth_method 0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f406c0a68e0 0x7f406c1d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f406c0a4910 0x7f406c1d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406b7fe640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c1d1b10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40640ab000 tx=0x7f4064019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.?.objecter ms_handle_connect 0x7f406c0a73d0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40640bb020 con 0x7f406c0a73d0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: dump: 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.331 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.332 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064009250 con 0x7f406c0a73d0 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4064003bb0 con 0x7f406c0a73d0 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: dump: 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.333 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.334 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40697fa640 4 set_mon_vals no callback set 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 5 monclient: authenticate success, global_id 34803 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 monclient: _renew_subs 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c12a150 con 0x7f406c0a73d0 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 client.34803.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 monclient: _renew_subs 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c06ddd0 con 0x7f406c0a73d0 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 1 librados: init done 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 0x7f406c1263a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f406c0a26b0 con 0x7f406c12a600 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 0x7f406c1263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient: get_auth_request con 0x7f406c12a600 auth_method 0 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f40640be020 con 0x7f406c0a73d0 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f404c03e550 0x7f404c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 10 monclient(hunting): handle_auth_done global_id 23876 payload 824 2024-09-17T16:33:00.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f406c127df0 con 0x7f406c12a600 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(214..214 src has 1..214) ==== 6620+0+0 (secure 0 0 0) 0x7f40640ead80 con 0x7f406c0a73d0 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.34803.objecter ms_dispatch 0x7f406c000ba0 osd_map(214..214 src has 1..214) 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 3 client.34803.objecter handle_osd_map got epochs [214,214] > 0 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 3 client.34803.objecter handle_osd_map decoding full epoch 214 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 20 client.34803.objecter dump_active .. 0 homeless 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f406affd640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 0x7f406c1263a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f405c00fb10 tx=0x7f405c0045a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.34803.objecter ms_handle_connect 0x7f406c12a600 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.34803.objecter resend_mon_ops 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f404c03e550 0x7f404c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f405c018030 con 0x7f406c12a600 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: dump: 2024-09-17T16:33:00.337 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.338 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 10 monclient: get_auth_request con 0x7f404c03e550 auth_method 0 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f405c00db60 con 0x7f406c12a600 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 msgr2=0x7f406c1263a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 0x7f406c1263a0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f405c00fb10 tx=0x7f405c0045a0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 msgr2=0x7f406c1263a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c12a600 0x7f406c1263a0 secure :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f405c00fb10 tx=0x7f405c0045a0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f4063fff640 10 client.34803.objecter ms_handle_connect 0x7f404c03e550 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.307+0000 7f40627fc640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f404c03e550 0x7f404c040a10 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 crypto rx=0x7f406c127b00 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:00.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 0x7f406c126cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f406c051e80 con 0x7f406c1268e0 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 0x7f406c126cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient: get_auth_request con 0x7f406c1268e0 auth_method 0 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 10 monclient(hunting): handle_auth_done global_id 23885 payload 824 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 1 -- 172.21.15.105:0/2755620156 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f406c127230 con 0x7f406c1268e0 2024-09-17T16:33:00.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406affd640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 0x7f406c126cf0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f405c018000 tx=0x7f405c007f90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 client.34803.objecter ms_handle_connect 0x7f406c1268e0 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 client.34803.objecter resend_mon_ops 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f405c004a10 con 0x7f406c1268e0 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 monclient: dump: 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.341 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f405c0045a0 con 0x7f406c1268e0 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 msgr2=0x7f406c126cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 0x7f406c126cf0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f405c018000 tx=0x7f405c007f90 comp rx=0 tx=0).stop 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 msgr2=0x7f406c126cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f4063fff640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f406c1268e0 0x7f406c126cf0 secure :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f405c018000 tx=0x7f405c007f90 comp rx=0 tx=0).stop 2024-09-17T16:33:00.343 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 327958, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:00.344 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.345 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:00.346 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.347 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:00.348 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.349 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:00.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.356 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.357 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:00.369 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:00.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 librados: watch_flush enter 2024-09-17T16:33:00.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 librados: watch_flush exit 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 20 client.34803.objecter shutdown clearing up homeless session... 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 client.34803.objecter successfully canceled tick 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f404c03e550 msgr2=0x7f404c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f404c03e550 0x7f404c040a10 secure :-1 s=READY pgs=3431 cs=0 l=1 rev1=1 crypto rx=0x7f406c127b00 tx=0x7f4054008040 comp rx=0 tx=0).stop 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 10 monclient: shutdown 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 msgr2=0x7f406c1d1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c1d1b10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40640ab000 tx=0x7f4064019620 comp rx=0 tx=0).stop 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f406b7fe640 1 -- 172.21.15.105:0/2755620156 reap_dead start 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 shutdown_connections 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 --2- 172.21.15.105:0/2755620156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f406c0a73d0 0x7f406c1d1b10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 >> 172.21.15.105:0/2755620156 conn(0x7f406c068ba0 msgr2=0x7f406c09fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 shutdown_connections 2024-09-17T16:33:00.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 -- 172.21.15.105:0/2755620156 wait complete. 2024-09-17T16:33:00.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.315+0000 7f407160a640 1 librados: shutdown 2024-09-17T16:33:00.372 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:00.372 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:33:00.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: get_monmap_and_config 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: monmap: 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:00.484556+0000 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:00.484556+0000 2024-09-17T16:33:00.487 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:00.488 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 Processor -- start 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 -- start start 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: init 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient: _add_conns ranks=[7,8,9,0,6,2,4,10,3,5,1] 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc8580a68e0 0x7fc8580a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-i con 0x7fc8580a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-e con 0x7fc8580a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8581396a0 0x7fc858139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-a con 0x7fc8581396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc8580a3fc0 con 0x7fc8581396a0 2024-09-17T16:33:00.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc8580a63c0 con 0x7fc8580a68e0 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc8580a7740 con 0x7fc8580a4910 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc8605b6640 10 monclient(hunting): authenticate will time out at 1263.750299s 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8581396a0 0x7fc858139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36528/0 (socket says 172.21.15.105:36528) 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 1 -- 172.21.15.105:0/2168648531 learned_addr learned my addr 172.21.15.105:0/2168648531 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): get_auth_request con 0x7fc8580a4910 auth_method 0 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): get_auth_request con 0x7fc8581396a0 auth_method 0 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85e352640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc8580a68e0 0x7fc8580a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85e352640 10 monclient(hunting): get_auth_request con 0x7fc8580a68e0 auth_method 0 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85e352640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85e352640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85e352640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient(hunting): handle_auth_done global_id 34812 payload 274 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 -- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 msgr2=0x7fc8580a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 -- 172.21.15.105:0/2168648531 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc8580a68e0 msgr2=0x7fc8580a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc8580a68e0 0x7fc8580a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 monclient: found mon.noname-a 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85eb53640 1 -- 172.21.15.105:0/2168648531 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85806ebc0 con 0x7fc8581396a0 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.483+0000 7fc85db51640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85eb53640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8581396a0 0x7fc858139ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc850009d90 tx=0x7fc85001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875b49cb57288c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 1 -- 172.21.15.105:0/2168648531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8500b2070 con 0x7fc8581396a0 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: dump: 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 1 -- 172.21.15.105:0/2168648531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8500a5ab0 con 0x7fc8581396a0 2024-09-17T16:33:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 1 -- 172.21.15.105:0/2168648531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8500a5ce0 con 0x7fc8581396a0 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 5 monclient: authenticate success, global_id 34812 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 4 set_mon_vals no callback set 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85d350640 10 monclient: dump: 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.507 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: shutdown 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8581396a0 msgr2=0x7fc858139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8581396a0 0x7fc858139ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc850009d90 tx=0x7fc85001c800 comp rx=0 tx=0).stop 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- 172.21.15.105:0/2168648531 shutdown_connections 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8581396a0 0x7fc858139ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc8580a4910 0x7fc8580a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- 172.21.15.105:0/2168648531 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc8580a68e0 0x7fc8580a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- 172.21.15.105:0/2168648531 >> 172.21.15.105:0/2168648531 conn(0x7fc858068ba0 msgr2=0x7fc858068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- 172.21.15.105:0/2168648531 shutdown_connections 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- 172.21.15.105:0/2168648531 wait complete. 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: monmap: 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:00.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 librados: starting msgr at 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 librados: starting objecter 2024-09-17T16:33:00.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 Processor -- start 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- start start 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 librados: setting wanted keys 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 librados: calling monclient init 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: init 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient: _add_conns ranks=[4,6,1,5,3,8,0,7,2] 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 0x7fc8581d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-c con 0x7fc8580a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc8580a68e0 0x7fc8581d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-d con 0x7fc8580a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8581396a0 0x7fc85813d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): picked mon.noname-f con 0x7fc8581396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc85806e630 con 0x7fc8581396a0 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc8580a7af0 con 0x7fc8580a4910 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc858143320 con 0x7fc8580a68e0 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc8605b6640 10 monclient(hunting): authenticate will time out at 1263.755268s 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 0x7fc8581d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc8580a68e0 0x7fc8581d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 0x7fc8581d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:45680/0 (socket says 172.21.15.105:45680) 2024-09-17T16:33:00.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 -- 172.21.15.105:0/892421176 learned_addr learned my addr 172.21.15.105:0/892421176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): get_auth_request con 0x7fc8580a4910 auth_method 0 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): get_auth_request con 0x7fc8580a68e0 auth_method 0 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient(hunting): handle_auth_done global_id 23035 payload 824 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85eb53640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8581396a0 0x7fc85813d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc8580a68e0 msgr2=0x7fc8581d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc8580a68e0 0x7fc8581d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8581396a0 msgr2=0x7fc85813d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8581396a0 0x7fc85813d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 monclient: found mon.noname-c 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc858123d10 con 0x7fc8580a4910 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc85e352640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 0x7fc8581d16f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc84800d830 tx=0x7fc84800dcf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc84effd640 10 client.?.objecter ms_handle_connect 0x7fc8580a4910 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.487+0000 7fc84effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc84800bb30 con 0x7fc8580a4910 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: dump: 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 5 monclient: authenticate success, global_id 23035 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848019070 con 0x7fc8580a4910 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc84800bcf0 con 0x7fc8580a4910 2024-09-17T16:33:00.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: dump: 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.517 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85cb4f640 4 set_mon_vals no callback set 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 monclient: _renew_subs 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:00.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85813f4b0 con 0x7fc8580a4910 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 client.23035.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 monclient: _renew_subs 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc858123a30 con 0x7fc8580a4910 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 1 librados: init done 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 0x7fc85813dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc858051e80 con 0x7fc85813db00 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc84800b6c0 con 0x7fc8580a4910 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 0x7fc85813dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc83c03e550 0x7fc83c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient: get_auth_request con 0x7fc85813db00 auth_method 0 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(214..214 src has 1..214) ==== 6620+0+0 (secure 0 0 0) 0x7fc84800f040 con 0x7fc8580a4910 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 client.23035.objecter ms_dispatch 0x7fc858000ba0 osd_map(214..214 src has 1..214) 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 3 client.23035.objecter handle_osd_map got epochs [214,214] > 0 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 3 client.23035.objecter handle_osd_map decoding full epoch 214 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 20 client.23035.objecter dump_active .. 0 homeless 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85eb53640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc83c03e550 0x7fc83c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85eb53640 10 monclient: get_auth_request con 0x7fc83c03e550 auth_method 0 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 10 monclient(hunting): handle_auth_done global_id 25093 payload 824 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8580a9e90 con 0x7fc85813db00 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 client.23035.objecter ms_handle_connect 0x7fc85813db00 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 client.23035.objecter resend_mon_ops 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85db51640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 0x7fc85813dee0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fc858074500 tx=0x7fc854011d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc854014440 con 0x7fc85813db00 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: dump: 2024-09-17T16:33:00.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.522 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 client.23035.objecter ms_handle_connect 0x7fc83c03e550 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc85eb53640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc83c03e550 0x7fc83c040a10 secure :-1 s=READY pgs=3432 cs=0 l=1 rev1=1 crypto rx=0x7fc8500af000 tx=0x7fc8500a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc85400fb00 con 0x7fc85813db00 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 msgr2=0x7fc85813dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 0x7fc85813dee0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fc858074500 tx=0x7fc854011d60 comp rx=0 tx=0).stop 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 msgr2=0x7fc85813dee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.491+0000 7fc84effd640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813db00 0x7fc85813dee0 secure :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fc858074500 tx=0x7fc854011d60 comp rx=0 tx=0).stop 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc8605b6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc8605b6640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 0x7fc8580a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc8605b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc858139ae0 con 0x7fc85813e420 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 0x7fc8580a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient: get_auth_request con 0x7fc85813e420 auth_method 0 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 10 monclient(hunting): handle_auth_done global_id 25102 payload 824 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 1 -- 172.21.15.105:0/892421176 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc858139480 con 0x7fc85813e420 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc85db51640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 0x7fc8580a0a20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fc858074500 tx=0x7fc8540046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 10 client.23035.objecter ms_handle_connect 0x7fc85813e420 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 10 client.23035.objecter resend_mon_ops 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8540148b0 con 0x7fc85813e420 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.499+0000 7fc84effd640 10 monclient: dump: 2024-09-17T16:33:00.526 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.527 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fc85402b470 con 0x7fc85813e420 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 msgr2=0x7fc8580a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 0x7fc8580a0a20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fc858074500 tx=0x7fc8540046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 msgr2=0x7fc8580a0a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc84effd640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85813e420 0x7fc8580a0a20 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 328099, 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:00.529 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:33:00.530 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.532 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:00.533 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.535 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:00.536 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.537 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.538 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.539 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.540 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.541 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.545 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.547 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:00.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.550 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.552 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.553 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:00.554 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 10 librados: watch_flush enter 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 10 librados: watch_flush exit 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 20 client.23035.objecter shutdown clearing up homeless session... 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 10 client.23035.objecter successfully canceled tick 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc83c03e550 msgr2=0x7fc83c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc83c03e550 0x7fc83c040a10 secure :-1 s=READY pgs=3432 cs=0 l=1 rev1=1 crypto rx=0x7fc8500af000 tx=0x7fc8500a7070 comp rx=0 tx=0).stop 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 10 monclient: shutdown 2024-09-17T16:33:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 msgr2=0x7fc8581d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 --2- 172.21.15.105:0/892421176 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc8580a4910 0x7fc8581d16f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc84800d830 tx=0x7fc84800dcf0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc85eb53640 1 -- 172.21.15.105:0/892421176 reap_dead start 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc85eb53640 1 -- 172.21.15.105:0/892421176 reap_dead start 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 shutdown_connections 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 >> 172.21.15.105:0/892421176 conn(0x7fc858068ba0 msgr2=0x7fc8580a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 shutdown_connections 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 -- 172.21.15.105:0/892421176 wait complete. 2024-09-17T16:33:00.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.503+0000 7fc8605b6640 1 librados: shutdown 2024-09-17T16:33:00.558 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:00.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:33:00.558 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:33:00.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 10 monclient: get_monmap_and_config 2024-09-17T16:33:00.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 10 monclient: monmap: 2024-09-17T16:33:00.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:00.674772+0000 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:00.674772+0000 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.678 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:00.679 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 1 Processor -- start 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 1 -- start start 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.671+0000 7fe87c1e3640 10 monclient: init 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient: _add_conns ranks=[10,8,4,0,7,9,3,1,2,5,6] 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 0x7fe8740a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-f con 0x7fe8740a4750 addr v2:172.21.15.160:3300/0 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe8740a5070 0x7fe874137580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-e con 0x7fe8740a5070 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-c con 0x7fe874137ac0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe8740a0b90 con 0x7fe8740a4750 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe8740a08b0 con 0x7fe874137ac0 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe8740a0e70 con 0x7fe8740a5070 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient(hunting): authenticate will time out at 1263.940496s 2024-09-17T16:33:00.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:45688/0 (socket says 172.21.15.105:45688) 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 -- 172.21.15.105:0/1835711938 learned_addr learned my addr 172.21.15.105:0/1835711938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): get_auth_request con 0x7fe874137ac0 auth_method 0 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe8740a5070 0x7fe874137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): get_auth_request con 0x7fe8740a5070 auth_method 0 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 1 --2- 172.21.15.105:0/1835711938 >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 0x7fe8740a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 10 monclient(hunting): get_auth_request con 0x7fe8740a4750 auth_method 0 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87977e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient(hunting): handle_auth_done global_id 23044 payload 274 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 -- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe8740a5070 msgr2=0x7fe874137580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 --2- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe8740a5070 0x7fe874137580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 -- 172.21.15.105:0/1835711938 >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 msgr2=0x7fe8740a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 --2- 172.21.15.105:0/1835711938 >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 0x7fe8740a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 monclient: found mon.noname-c 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 -- 172.21.15.105:0/1835711938 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87409fca0 con 0x7fe874137ac0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe879f7f640 1 --2- 172.21.15.105:0/1835711938 >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 0x7fe8740a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87a780640 1 --2- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009870 tx=0x7fe86c020830 comp rx=0 tx=0).ready entity=mon.4 client_cookie=643e1612bf3f87ea server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 1 -- 172.21.15.105:0/1835711938 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe86c0b0070 con 0x7fe874137ac0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: dump: 2024-09-17T16:33:00.692 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.693 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 5 monclient: authenticate success, global_id 23044 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 1 -- 172.21.15.105:0/1835711938 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c0aa900 con 0x7fe874137ac0 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:00.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe87c1e3640 4 set_mon_vals no callback set 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 1 -- 172.21.15.105:0/1835711938 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe86c0ab470 con 0x7fe874137ac0 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.675+0000 7fe878f7d640 10 monclient: dump: 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.695 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: shutdown 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 msgr2=0x7fe874137f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009870 tx=0x7fe86c020830 comp rx=0 tx=0).stop 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/1835711938 shutdown_connections 2024-09-17T16:33:00.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe874137ac0 0x7fe874137f00 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/1835711938 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe8740a5070 0x7fe874137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/1835711938 >> v2:172.21.15.160:3300/0 conn(0x7fe8740a4750 0x7fe8740a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/1835711938 >> 172.21.15.105:0/1835711938 conn(0x7fe8740a5840 msgr2=0x7fe8740a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/1835711938 shutdown_connections 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/1835711938 wait complete. 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: monmap: 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:00.697 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 librados: starting msgr at 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 librados: starting objecter 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 Processor -- start 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- start start 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 librados: setting wanted keys 2024-09-17T16:33:00.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 librados: calling monclient init 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: init 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _add_conns ranks=[5,7,2,4,0,6,8,3,1] 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8740a4750 0x7fe87413c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-h con 0x7fe8740a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe8740a5070 0x7fe87413c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-i con 0x7fe8740a5070 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): picked mon.noname-b con 0x7fe874137ac0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe8740a0430 con 0x7fe874137ac0 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe8740a02b0 con 0x7fe8740a4750 2024-09-17T16:33:00.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe87413f460 con 0x7fe8740a5070 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): authenticate will time out at 1263.945615s 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53044/0 (socket says 172.21.15.105:53044) 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 -- 172.21.15.105:0/2368551837 learned_addr learned my addr 172.21.15.105:0/2368551837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): get_auth_request con 0x7fe874137ac0 auth_method 0 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8740a4750 0x7fe87413c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87977e640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe8740a5070 0x7fe87413c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87977e640 10 monclient(hunting): get_auth_request con 0x7fe8740a5070 auth_method 0 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87977e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87977e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87977e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient(hunting): handle_auth_done global_id 44129 payload 824 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe8740a5070 msgr2=0x7fe87413c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe8740a5070 0x7fe87413c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8740a4750 msgr2=0x7fe87413c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8740a4750 0x7fe87413c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 monclient: found mon.noname-b 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe874121320 con 0x7fe874137ac0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 10 monclient: get_auth_request con 0x7fe8740a4750 auth_method 0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8740a4750 0x7fe87413c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87a780640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe86c020660 tx=0x7fe86c01d5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 client.?.objecter ms_handle_connect 0x7fe874137ac0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe86c0b0050 con 0x7fe874137ac0 2024-09-17T16:33:00.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: dump: 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.703 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c0b9070 con 0x7fe874137ac0 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 5 monclient: authenticate success, global_id 44129 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _renew_subs 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe874121600 con 0x7fe874137ac0 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe86c0aae60 con 0x7fe874137ac0 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 monclient: dump: 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.705 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.706 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86bfff640 4 set_mon_vals no callback set 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 client.44129.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _renew_subs 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe874120a70 con 0x7fe874137ac0 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 librados: init done 2024-09-17T16:33:00.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 0x7fe8741389e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe874051e80 con 0x7fe874138600 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe86c0b3070 con 0x7fe874137ac0 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe85803e550 0x7fe858040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 0x7fe8741389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(215..215 src has 1..215) ==== 5908+0+0 (secure 0 0 0) 0x7fe87409fa10 con 0x7fe874137ac0 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 10 client.44129.objecter ms_dispatch 0x7fe874000ba0 osd_map(215..215 src has 1..215) 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 3 client.44129.objecter handle_osd_map got epochs [215,215] > 0 2024-09-17T16:33:00.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 3 client.44129.objecter handle_osd_map decoding full epoch 215 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe86a7fc640 20 client.44129.objecter dump_active .. 0 homeless 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 10 monclient: get_auth_request con 0x7fe874138600 auth_method 0 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.679+0000 7fe879f7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe87977e640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe85803e550 0x7fe858040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe87977e640 10 monclient: get_auth_request con 0x7fe85803e550 auth_method 0 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 10 monclient(hunting): handle_auth_done global_id 19416 payload 824 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8740a5eb0 con 0x7fe874138600 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 client.44129.objecter ms_handle_connect 0x7fe85803e550 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe87977e640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe85803e550 0x7fe858040a10 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 crypto rx=0x7fe87413d710 tx=0x7fe864008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 client.44129.objecter ms_handle_connect 0x7fe874138600 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 client.44129.objecter resend_mon_ops 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 0x7fe8741389e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe870009f10 tx=0x7fe87000e510 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe870015880 con 0x7fe874138600 2024-09-17T16:33:00.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 monclient: dump: 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe87001a070 con 0x7fe874138600 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:00.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 msgr2=0x7fe8741389e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 0x7fe8741389e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe870009f10 tx=0x7fe87000e510 comp rx=0 tx=0).stop 2024-09-17T16:33:00.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 msgr2=0x7fe8741389e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.683+0000 7fe86a7fc640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138600 0x7fe8741389e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 0x7fe8741d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe874137580 con 0x7fe874138f20 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 0x7fe8741d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient: get_auth_request con 0x7fe874138f20 auth_method 0 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 10 monclient(hunting): handle_auth_done global_id 19425 payload 824 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 1 -- 172.21.15.105:0/2368551837 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe87411f580 con 0x7fe874138f20 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe879f7f640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 0x7fe8741d73d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe870019000 tx=0x7fe870004a50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 client.44129.objecter ms_handle_connect 0x7fe874138f20 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 client.44129.objecter resend_mon_ops 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe870015c00 con 0x7fe874138f20 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 monclient: dump: 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.716 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9289 (secure 0 0 0) 0x7fe87001f030 con 0x7fe874138f20 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 msgr2=0x7fe8741d73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 0x7fe8741d73d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe870019000 tx=0x7fe870004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 msgr2=0x7fe8741d73d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe86a7fc640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe874138f20 0x7fe8741d73d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:33:00.718 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 328315, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:33:00.719 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.720 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.721 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:00.722 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.723 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:00.724 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.725 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:00.726 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:00.727 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:00.729 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:00.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.732 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.739 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.740 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:00.741 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 librados: watch_flush enter 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 librados: watch_flush exit 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 20 client.44129.objecter shutdown clearing up homeless session... 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 client.44129.objecter successfully canceled tick 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe85803e550 msgr2=0x7fe858040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe85803e550 0x7fe858040a10 secure :-1 s=READY pgs=3433 cs=0 l=1 rev1=1 crypto rx=0x7fe87413d710 tx=0x7fe864008040 comp rx=0 tx=0).stop 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 10 monclient: shutdown 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 msgr2=0x7fe87413cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87a780640 1 -- 172.21.15.105:0/2368551837 reap_dead start 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe86c020660 tx=0x7fe86c01d5d0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 shutdown_connections 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 --2- 172.21.15.105:0/2368551837 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe874137ac0 0x7fe87413cab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 >> 172.21.15.105:0/2368551837 conn(0x7fe8740a5840 msgr2=0x7fe8740a6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 shutdown_connections 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 -- 172.21.15.105:0/2368551837 wait complete. 2024-09-17T16:33:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.691+0000 7fe87c1e3640 1 librados: shutdown 2024-09-17T16:33:00.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:00.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:33:00.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: get_monmap_and_config 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: monmap: 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:00.864735+0000 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:00.864735+0000 2024-09-17T16:33:00.867 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:00.868 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 Processor -- start 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 -- start start 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: init 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient: _add_conns ranks=[10,9,4,5,0,8,3,1,7,6,2] 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a68e0 0x7fd1e00a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-f con 0x7fd1e00a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1e00a4910 0x7fd1e00a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-a con 0x7fd1e00a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd1e01396a0 0x7fd1e0139b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-c con 0x7fd1e01396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00a3fc0 con 0x7fd1e00a4910 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00a63c0 con 0x7fd1e00a68e0 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fd1e00a7740 con 0x7fd1e01396a0 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1e4e62640 10 monclient(hunting): authenticate will time out at 1264.130520s 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1e00a4910 0x7fd1e00a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd1e01396a0 0x7fd1e0139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1e00a4910 0x7fd1e00a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49578/0 (socket says 172.21.15.105:49578) 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/220894183 learned_addr learned my addr 172.21.15.105:0/220894183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): get_auth_request con 0x7fd1e01396a0 auth_method 0 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): get_auth_request con 0x7fd1e00a4910 auth_method 0 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1de59c640 1 --2- 172.21.15.105:0/220894183 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a68e0 0x7fd1e00a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1de59c640 10 monclient(hunting): get_auth_request con 0x7fd1e00a68e0 auth_method 0 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1de59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_done global_id 34821 payload 274 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd1e01396a0 msgr2=0x7fd1e0139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd1e01396a0 0x7fd1e0139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/220894183 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a68e0 msgr2=0x7fd1e00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/220894183 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a68e0 0x7fd1e00a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 monclient: found mon.noname-a 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.863+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/220894183 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e006ebc0 con 0x7fd1e00a4910 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4910 0x7fd1e00a7200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0074a10 tx=0x7fd1c801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e67bf8d4119a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 1 -- 172.21.15.105:0/220894183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1c80b0070 con 0x7fd1e00a4910 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: dump: 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.876 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 1 -- 172.21.15.105:0/220894183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c80aa810 con 0x7fd1e00a4910 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 5 monclient: authenticate success, global_id 34821 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 1 -- 172.21.15.105:0/220894183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1c80a9300 con 0x7fd1e00a4910 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 4 set_mon_vals no callback set 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1dd59a640 10 monclient: dump: 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.878 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: shutdown 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4910 msgr2=0x7fd1e00a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4910 0x7fd1e00a7200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0074a10 tx=0x7fd1c801c3b0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- 172.21.15.105:0/220894183 shutdown_connections 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd1e01396a0 0x7fd1e0139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/220894183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1e00a4910 0x7fd1e00a7200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/220894183 >> v2:172.21.15.160:3300/0 conn(0x7fd1e00a68e0 0x7fd1e00a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- 172.21.15.105:0/220894183 >> 172.21.15.105:0/220894183 conn(0x7fd1e0068ba0 msgr2=0x7fd1e0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- 172.21.15.105:0/220894183 shutdown_connections 2024-09-17T16:33:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- 172.21.15.105:0/220894183 wait complete. 2024-09-17T16:33:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: build_initial_monmap 2024-09-17T16:33:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: monmap: 2024-09-17T16:33:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 librados: starting msgr at 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 librados: starting objecter 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 Processor -- start 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- start start 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 librados: setting wanted keys 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 librados: calling monclient init 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: init 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient: _add_conns ranks=[3,8,5,4,7,6,2,1,0] 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4910 0x7fd1e0122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-g con 0x7fd1e00a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 0x7fd1e01d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-e con 0x7fd1e00a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01396a0 0x7fd1e013d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): picked mon.noname-h con 0x7fd1e01396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e006e630 con 0x7fd1e00a4910 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd1e00a7af0 con 0x7fd1e01396a0 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fd1e0143320 con 0x7fd1e00a68e0 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1e4e62640 10 monclient(hunting): authenticate will time out at 1264.135648s 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 0x7fd1e01d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 0x7fd1e01d1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36532/0 (socket says 172.21.15.105:36532) 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/3781010924 learned_addr learned my addr 172.21.15.105:0/3781010924 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): get_auth_request con 0x7fd1e00a68e0 auth_method 0 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.867+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01396a0 0x7fd1e013d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4910 0x7fd1e0122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 10 monclient(hunting): get_auth_request con 0x7fd1e01396a0 auth_method 0 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 10 monclient(hunting): handle_auth_done global_id 23912 payload 824 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01396a0 msgr2=0x7fd1e013d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01396a0 0x7fd1e013d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4910 msgr2=0x7fd1e0122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4910 0x7fd1e0122570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 10 monclient: _finish_hunting 0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 monclient: found mon.noname-e 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e0122ee0 con 0x7fd1e00a68e0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient: get_auth_request con 0x7fd1e00a4910 auth_method 0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1e00a4910 0x7fd1e0122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ddd9b640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 0x7fd1e01d1820 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097c0 tx=0x7fd1c8019150 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.?.objecter ms_handle_connect 0x7fd1e00a68e0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1c80b9070 con 0x7fd1e00a68e0 2024-09-17T16:33:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: dump: 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c80044b0 con 0x7fd1e00a68e0 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 5 monclient: authenticate success, global_id 23912 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1c80b0040 con 0x7fd1e00a68e0 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1dcd99640 4 set_mon_vals no callback set 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: dump: 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.896 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 monclient: _renew_subs 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e0123a70 con 0x7fd1e00a68e0 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 client.23912.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 monclient: _renew_subs 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e0123d50 con 0x7fd1e00a68e0 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 1 librados: init done 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 0x7fd1e013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd1e01c81b0 con 0x7fd1e01d1d60 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd1c8004040 con 0x7fd1e00a68e0 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 0x7fd1e013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient: get_auth_request con 0x7fd1e01d1d60 auth_method 0 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1c003e580 0x7fd1c0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(215..215 src has 1..215) ==== 5908+0+0 (secure 0 0 0) 0x7fd1c80e92b0 con 0x7fd1e00a68e0 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.23912.objecter ms_dispatch 0x7fd1e0000ba0 osd_map(215..215 src has 1..215) 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 3 client.23912.objecter handle_osd_map got epochs [215,215] > 0 2024-09-17T16:33:00.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 3 client.23912.objecter handle_osd_map decoding full epoch 215 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 20 client.23912.objecter dump_active .. 0 homeless 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1c003e580 0x7fd1c0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 10 monclient: get_auth_request con 0x7fd1c003e580 auth_method 0 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 10 monclient(hunting): handle_auth_done global_id 19598 payload 824 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1e0077b60 con 0x7fd1e01d1d60 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.23912.objecter ms_handle_connect 0x7fd1e01d1d60 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.23912.objecter resend_mon_ops 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 0x7fd1e013db90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007db0 tx=0x7fd1d800f520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 client.23912.objecter ms_handle_connect 0x7fd1c003e580 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1ded9d640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1c003e580 0x7fd1c0040a40 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009a30 tx=0x7fd1d0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d8003e10 con 0x7fd1e01d1d60 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.871+0000 7fd1d6ffd640 10 monclient: dump: 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.900 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1d8019070 con 0x7fd1e01d1d60 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 msgr2=0x7fd1e013db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 0x7fd1e013db90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007db0 tx=0x7fd1d800f520 comp rx=0 tx=0).stop 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 msgr2=0x7fd1e013db90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.875+0000 7fd1d6ffd640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e01d1d60 0x7fd1e013db90 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1e4e62640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 0x7fd1e00a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1e4e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:00.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd1e0051e80 con 0x7fd1e013e0d0 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 0x7fd1e00a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient: get_auth_request con 0x7fd1e013e0d0 auth_method 0 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 10 monclient(hunting): handle_auth_done global_id 19607 payload 824 2024-09-17T16:33:00.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 1 -- 172.21.15.105:0/3781010924 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd1e013e4e0 con 0x7fd1e013e0d0 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1de59c640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 0x7fd1e00a0a20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007db0 tx=0x7fd1d8004a50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 10 client.23912.objecter ms_handle_connect 0x7fd1e013e0d0 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 10 client.23912.objecter resend_mon_ops 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd1d80046b0 con 0x7fd1e013e0d0 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:00.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.879+0000 7fd1d6ffd640 10 monclient: dump: 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:00.904 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fd1d8004240 con 0x7fd1e013e0d0 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 msgr2=0x7fd1e00a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 0x7fd1e00a0a20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007db0 tx=0x7fd1d8004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 msgr2=0x7fd1e00a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1d6ffd640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd1e013e0d0 0x7fd1e00a0a20 secure :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007db0 tx=0x7fd1d8004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:00.905 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 328479, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:00.906 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.907 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.909 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:00.910 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.912 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.914 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.915 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.917 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.922 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:00.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:00.925 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.926 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:00.927 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 10 librados: watch_flush enter 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 10 librados: watch_flush exit 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 20 client.23912.objecter shutdown clearing up homeless session... 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 10 client.23912.objecter successfully canceled tick 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1c003e580 msgr2=0x7fd1c0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd1c003e580 0x7fd1c0040a40 secure :-1 s=READY pgs=3434 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009a30 tx=0x7fd1d0008040 comp rx=0 tx=0).stop 2024-09-17T16:33:00.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 10 monclient: shutdown 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 msgr2=0x7fd1e01d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 --2- 172.21.15.105:0/3781010924 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd1e00a68e0 0x7fd1e01d1820 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097c0 tx=0x7fd1c8019150 comp rx=0 tx=0).stop 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1ded9d640 1 -- 172.21.15.105:0/3781010924 reap_dead start 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1ded9d640 1 -- 172.21.15.105:0/3781010924 reap_dead start 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 shutdown_connections 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 >> 172.21.15.105:0/3781010924 conn(0x7fd1e0068ba0 msgr2=0x7fd1e00a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 shutdown_connections 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 -- 172.21.15.105:0/3781010924 wait complete. 2024-09-17T16:33:00.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:00.883+0000 7fd1e4e62640 1 librados: shutdown 2024-09-17T16:33:00.930 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:00.931 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:33:00.931 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:33:01.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.043+0000 7f236ebb6640 10 monclient: get_monmap_and_config 2024-09-17T16:33:01.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.043+0000 7f236ebb6640 10 monclient: build_initial_monmap 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.043+0000 7f236ebb6640 10 monclient: monmap: 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:01.047367+0000 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:01.047367+0000 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:01.051 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 Processor -- start 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 -- start start 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient: init 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient: _add_conns ranks=[6,5,9,3,8,2,10,0,7,1,4] 2024-09-17T16:33:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-d con 0x7f23680a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 0x7f23680a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-h con 0x7f23680a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 0x7f2368139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-a con 0x7f23681396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f23680a3fc0 con 0x7f23681396a0 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f23680a63c0 con 0x7f23680a4910 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f23680a7740 con 0x7f23680a68e0 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236ebb6640 10 monclient(hunting): authenticate will time out at 1264.313366s 2024-09-17T16:33:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 0x7f2368139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36480/0 (socket says 172.21.15.105:36480) 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 -- 172.21.15.105:0/3412099891 learned_addr learned my addr 172.21.15.105:0/3412099891 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): get_auth_request con 0x7f23680a68e0 auth_method 0 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): get_auth_request con 0x7f23681396a0 auth_method 0 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 0x7f23680a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 10 monclient(hunting): get_auth_request con 0x7f23680a4910 auth_method 0 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient(hunting): handle_auth_done global_id 23910 payload 274 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 -- 172.21.15.105:0/3412099891 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 msgr2=0x7f23680a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 0x7f23680a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 -- 172.21.15.105:0/3412099891 >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 msgr2=0x7f2368139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 --2- 172.21.15.105:0/3412099891 >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 0x7f2368139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient: _finish_hunting 0 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 monclient: found mon.noname-d 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236c952640 1 -- 172.21.15.105:0/3412099891 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236806ebc0 con 0x7f23680a68e0 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f235ffff640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 0x7f23680a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:33:01.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.047+0000 7f236d153640 1 --2- 172.21.15.105:0/3412099891 >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 0x7f2368139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2364009870 tx=0x7f236401c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=25f60fa7e327bff server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 1 -- 172.21.15.105:0/3412099891 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23640b0070 con 0x7f23680a68e0 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: dump: 2024-09-17T16:33:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 5 monclient: authenticate success, global_id 23910 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 1 -- 172.21.15.105:0/3412099891 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23640aa810 con 0x7f23680a68e0 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 4 set_mon_vals no callback set 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 1 -- 172.21.15.105:0/3412099891 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23640a9300 con 0x7f23680a68e0 2024-09-17T16:33:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235f7fe640 10 monclient: dump: 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: shutdown 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- 172.21.15.105:0/3412099891 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 msgr2=0x7f23680a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2364009870 tx=0x7f236401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- 172.21.15.105:0/3412099891 shutdown_connections 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- 172.21.15.105:0/3412099891 >> v2:172.21.15.105:3300/0 conn(0x7f23681396a0 0x7f2368139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23680a4910 0x7f23680a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- 172.21.15.105:0/3412099891 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23680a68e0 0x7f23680a6cc0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- 172.21.15.105:0/3412099891 >> 172.21.15.105:0/3412099891 conn(0x7f2368068ba0 msgr2=0x7f2368068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- 172.21.15.105:0/3412099891 shutdown_connections 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- 172.21.15.105:0/3412099891 wait complete. 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: build_initial_monmap 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: monmap: 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:01.070 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 librados: starting msgr at 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 librados: starting objecter 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 Processor -- start 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- start start 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 librados: setting wanted keys 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 librados: calling monclient init 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: init 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:01.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient: _add_conns ranks=[8,2,0,4,1,3,5,6,7] 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 0x7f2368137e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-e con 0x7f23680a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23680a68e0 0x7f2368138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-b con 0x7f23680a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23681396a0 0x7f236813d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): picked mon.noname-a con 0x7f23681396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f236806e630 con 0x7f23681396a0 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23680a7af0 con 0x7f23680a68e0 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f2368143320 con 0x7f23680a4910 2024-09-17T16:33:01.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236ebb6640 10 monclient(hunting): authenticate will time out at 1264.319249s 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 0x7f2368137e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 0x7f2368137e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36536/0 (socket says 172.21.15.105:36536) 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23680a68e0 0x7f2368138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 -- 172.21.15.105:0/454097242 learned_addr learned my addr 172.21.15.105:0/454097242 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): get_auth_request con 0x7f23680a4910 auth_method 0 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): get_auth_request con 0x7f23680a68e0 auth_method 0 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23681396a0 0x7f236813d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): get_auth_request con 0x7f23681396a0 auth_method 0 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236d153640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient(hunting): handle_auth_done global_id 23921 payload 824 2024-09-17T16:33:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23680a68e0 msgr2=0x7f2368138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23680a68e0 0x7f2368138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23681396a0 msgr2=0x7f236813d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23681396a0 0x7f236813d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient: _finish_hunting 0 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 monclient: found mon.noname-e 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680a9f30 con 0x7f23680a4910 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f236c952640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 0x7f2368137e20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f23640ae040 tx=0x7f2364019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235d7fa640 10 client.?.objecter ms_handle_connect 0x7f23680a4910 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.051+0000 7f235d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23640b9070 con 0x7f23680a4910 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: dump: 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23640044b0 con 0x7f23680a4910 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23640b0040 con 0x7f23680a4910 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 5 monclient: authenticate success, global_id 23921 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235effd640 4 set_mon_vals no callback set 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: dump: 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 monclient: _renew_subs 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23680abb20 con 0x7f23680a4910 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 client.23921.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 monclient: _renew_subs 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f2368123ae0 con 0x7f23680a4910 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 1 librados: init done 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 0x7f236813e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2368051e80 con 0x7f236813dc80 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f23640aac90 con 0x7f23680a4910 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 0x7f236813e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient: get_auth_request con 0x7f236813dc80 auth_method 0 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f235403e5b0 0x7f2354040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(215..215 src has 1..215) ==== 5908+0+0 (secure 0 0 0) 0x7f23640e8de0 con 0x7f23680a4910 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 client.23921.objecter ms_dispatch 0x7f2368000ba0 osd_map(215..215 src has 1..215) 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 3 client.23921.objecter handle_osd_map got epochs [215,215] > 0 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 3 client.23921.objecter handle_osd_map decoding full epoch 215 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 20 client.23921.objecter dump_active .. 0 homeless 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236d153640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f235403e5b0 0x7f2354040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236d153640 10 monclient: get_auth_request con 0x7f235403e5b0 auth_method 0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 10 monclient(hunting): handle_auth_done global_id 34495 payload 824 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f236813fbd0 con 0x7f236813dc80 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 client.23921.objecter ms_handle_connect 0x7f235403e5b0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f236d153640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f235403e5b0 0x7f2354040a70 secure :-1 s=READY pgs=3435 cs=0 l=1 rev1=1 crypto rx=0x7f2368124290 tx=0x7f236000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 client.23921.objecter ms_handle_connect 0x7f236813dc80 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 client.23921.objecter resend_mon_ops 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235ffff640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 0x7f236813e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2358011520 tx=0x7f23580119e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2358019070 con 0x7f236813dc80 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: dump: 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2358015670 con 0x7f236813dc80 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 msgr2=0x7f236813e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 0x7f236813e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2358011520 tx=0x7f23580119e0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 msgr2=0x7f236813e060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:01.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.055+0000 7f235d7fa640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813dc80 0x7f236813e060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f236ebb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f236ebb6640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 0x7f23680a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f236ebb6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2368139ae0 con 0x7f236813e5a0 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 0x7f23680a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient: get_auth_request con 0x7f236813e5a0 auth_method 0 2024-09-17T16:33:01.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 10 monclient(hunting): handle_auth_done global_id 34504 payload 824 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 1 -- 172.21.15.105:0/454097242 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f23680008e0 con 0x7f236813e5a0 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235ffff640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 0x7f23680a0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2358008010 tx=0x7f2358004740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 10 client.23921.objecter ms_handle_connect 0x7f236813e5a0 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 10 client.23921.objecter resend_mon_ops 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f235800f040 con 0x7f236813e5a0 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:01.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.063+0000 7f235d7fa640 10 monclient: dump: 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.087 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f235800f670 con 0x7f236813e5a0 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 msgr2=0x7f23680a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 0x7f23680a0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2358008010 tx=0x7f2358004740 comp rx=0 tx=0).stop 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 msgr2=0x7f23680a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f235d7fa640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f236813e5a0 0x7f23680a0a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:01.088 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 171862, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:01.089 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:01.090 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:01.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:01.092 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:01.093 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.095 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:01.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:01.098 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.100 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.101 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.103 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:01.105 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:01.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:01.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:01.109 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:01.110 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:01.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:01.111 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:01.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:01.111 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:01.111 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 10 librados: watch_flush enter 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 10 librados: watch_flush exit 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 20 client.23921.objecter shutdown clearing up homeless session... 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 10 client.23921.objecter successfully canceled tick 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f235403e5b0 msgr2=0x7f2354040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f235403e5b0 0x7f2354040a70 secure :-1 s=READY pgs=3435 cs=0 l=1 rev1=1 crypto rx=0x7f2368124290 tx=0x7f236000b040 comp rx=0 tx=0).stop 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 10 monclient: shutdown 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 msgr2=0x7f2368137e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 --2- 172.21.15.105:0/454097242 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23680a4910 0x7f2368137e20 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f23640ae040 tx=0x7f2364019190 comp rx=0 tx=0).stop 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236d153640 1 -- 172.21.15.105:0/454097242 reap_dead start 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236d153640 1 -- 172.21.15.105:0/454097242 reap_dead start 2024-09-17T16:33:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 shutdown_connections 2024-09-17T16:33:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 >> 172.21.15.105:0/454097242 conn(0x7f2368068ba0 msgr2=0x7f23680a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 shutdown_connections 2024-09-17T16:33:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 -- 172.21.15.105:0/454097242 wait complete. 2024-09-17T16:33:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.067+0000 7f236ebb6640 1 librados: shutdown 2024-09-17T16:33:01.114 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:01.114 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:33:01.114 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: get_monmap_and_config 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: build_initial_monmap 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: monmap: 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:01.229820+0000 2024-09-17T16:33:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:01.229820+0000 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:01.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 Processor -- start 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 -- start start 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: init 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient: _add_conns ranks=[9,4,10,2,7,6,5,8,3,0,1] 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-a con 0x7fb2a80a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:01.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-c con 0x7fb2a80a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 0x7fb2a8139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-f con 0x7fb2a81396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a80a3fc0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a80a63c0 con 0x7fb2a81396a0 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb2a80a7740 con 0x7fb2a80a4910 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2af760640 10 monclient(hunting): authenticate will time out at 1264.495787s 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.227+0000 7fb2ad4fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49606/0 (socket says 172.21.15.105:49606) 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:45710/0 (socket says 172.21.15.105:45710) 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 1 -- 172.21.15.105:0/3985458368 learned_addr learned my addr 172.21.15.105:0/3985458368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): get_auth_request con 0x7fb2a80a68e0 auth_method 0 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): get_auth_request con 0x7fb2a80a4910 auth_method 0 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 0x7fb2a8139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 10 monclient(hunting): get_auth_request con 0x7fb2a81396a0 auth_method 0 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient(hunting): handle_auth_done global_id 23062 payload 274 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 -- 172.21.15.105:0/3985458368 >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 msgr2=0x7fb2a8139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 0x7fb2a8139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 -- 172.21.15.105:0/3985458368 >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 msgr2=0x7fb2a80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient: _finish_hunting 0 2024-09-17T16:33:01.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 monclient: found mon.noname-c 2024-09-17T16:33:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 -- 172.21.15.105:0/3985458368 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a806ebc0 con 0x7fb2a80a4910 2024-09-17T16:33:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2ad4fc640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:01.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2adcfd640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 0x7fb2a8139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2accfb640 1 --2- 172.21.15.105:0/3985458368 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80acf60 tx=0x7fb2a401c310 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4684abdcc64cd22b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 1 -- 172.21.15.105:0/3985458368 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2a40b0070 con 0x7fb2a80a4910 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: dump: 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.245 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: _finish_auth 0 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:01.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 5 monclient: authenticate success, global_id 23062 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 1 -- 172.21.15.105:0/3985458368 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a40aa810 con 0x7fb2a80a4910 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 4 set_mon_vals no callback set 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 1 -- 172.21.15.105:0/3985458368 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2a40a9300 con 0x7fb2a80a4910 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb29ffff640 10 monclient: dump: 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.247 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 10 monclient: shutdown 2024-09-17T16:33:01.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 -- 172.21.15.105:0/3985458368 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 msgr2=0x7fb2a80a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 --2- 172.21.15.105:0/3985458368 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80acf60 tx=0x7fb2a401c310 comp rx=0 tx=0).stop 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 -- 172.21.15.105:0/3985458368 shutdown_connections 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 --2- 172.21.15.105:0/3985458368 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a4910 0x7fb2a80a7200 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.160:3300/0 conn(0x7fb2a81396a0 0x7fb2a8139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 --2- 172.21.15.105:0/3985458368 >> v2:172.21.15.105:3300/0 conn(0x7fb2a80a68e0 0x7fb2a80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 -- 172.21.15.105:0/3985458368 >> 172.21.15.105:0/3985458368 conn(0x7fb2a8068ba0 msgr2=0x7fb2a8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 -- 172.21.15.105:0/3985458368 shutdown_connections 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.231+0000 7fb2af760640 1 -- 172.21.15.105:0/3985458368 wait complete. 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: build_initial_monmap 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: monmap: 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:01.249 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:01.250 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 librados: starting msgr at 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 librados: starting objecter 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 Processor -- start 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- start start 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 librados: setting wanted keys 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 librados: calling monclient init 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: init 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _add_conns ranks=[6,4,2,5,7,3,0,8,1] 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 0x7fb2a81d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-d con 0x7fb2a80a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 0x7fb2a81d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-c con 0x7fb2a80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:01.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2a81396a0 0x7fb2a813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): picked mon.noname-b con 0x7fb2a81396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a806e630 con 0x7fb2a81396a0 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb2a80a7af0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fb2a8143320 con 0x7fb2a80a4910 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient(hunting): authenticate will time out at 1264.501482s 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 0x7fb2a81d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2a81396a0 0x7fb2a813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 0x7fb2a81d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 0x7fb2a81d15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36484/0 (socket says 172.21.15.105:36484) 2024-09-17T16:33:01.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 1 -- 172.21.15.105:0/435270731 learned_addr learned my addr 172.21.15.105:0/435270731 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): get_auth_request con 0x7fb2a80a4910 auth_method 0 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): get_auth_request con 0x7fb2a81396a0 auth_method 0 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): get_auth_request con 0x7fb2a80a68e0 auth_method 0 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2ad4fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2adcfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient(hunting): handle_auth_done global_id 23071 payload 824 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 -- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 msgr2=0x7fb2a81d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 0x7fb2a81d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 -- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2a81396a0 msgr2=0x7fb2a813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2a81396a0 0x7fb2a813d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient: _finish_hunting 0 2024-09-17T16:33:01.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 monclient: found mon.noname-c 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 -- 172.21.15.105:0/435270731 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a81236f0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2accfb640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 0x7fb2a81d1af0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ab000 tx=0x7fb2a40190b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 client.?.objecter ms_handle_connect 0x7fb2a80a68e0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2a40b0040 con 0x7fb2a80a68e0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: dump: 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.255 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: _finish_auth 0 2024-09-17T16:33:01.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a40044b0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 5 monclient: authenticate success, global_id 23071 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _renew_subs 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a8122e60 con 0x7fb2a80a68e0 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2a40aaca0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 monclient: dump: 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:01.257 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29f7fe640 4 set_mon_vals no callback set 2024-09-17T16:33:01.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 client.23071.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _renew_subs 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a80a9f00 con 0x7fb2a80a68e0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 librados: init done 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a8139b50 con 0x7fb2a80a68e0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb2a4004040 con 0x7fb2a80a68e0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 0x7fb28c040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(215..215 src has 1..215) ==== 5908+0+0 (secure 0 0 0) 0x7fb2a40e9890 con 0x7fb2a80a68e0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 10 client.23071.objecter ms_dispatch 0x7fb2a8000ba0 osd_map(215..215 src has 1..215) 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 3 client.23071.objecter handle_osd_map got epochs [215,215] > 0 2024-09-17T16:33:01.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 3 client.23071.objecter handle_osd_map decoding full epoch 215 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.235+0000 7fb29dffb640 20 client.23071.objecter dump_active .. 0 homeless 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.239+0000 7fb2ad4fc640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 0x7fb28c040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.239+0000 7fb2ad4fc640 10 monclient: get_auth_request con 0x7fb28c03e540 auth_method 0 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.239+0000 7fb29dffb640 10 client.23071.objecter ms_handle_connect 0x7fb28c03e540 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.239+0000 7fb2ad4fc640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 0x7fb28c040a00 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 crypto rx=0x7fb298009630 tx=0x7fb298009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.243+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb2a4016050 con 0x7fb2a80a68e0 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.243+0000 7fb29dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:01.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.243+0000 7fb29dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:01.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.479+0000 7fb2af760640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:33:01.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.479+0000 7fb2af760640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:33:01.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.479+0000 7fb2af760640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:01.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.479+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb2a8123080 con 0x7fb2a80a68e0 2024-09-17T16:33:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.483+0000 7fb29dffb640 1 -- 172.21.15.105:0/435270731 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb2a40b43f0 con 0x7fb2a80a68e0 2024-09-17T16:33:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.483+0000 7fb29dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:33:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.483+0000 7fb29dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:01.489 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=7,config=2,health=12,kv=10,logm=69} crc {auth=2265227187,config=692619927,health=92814587,kv=2863082231,logm=1062329576}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=10,logm=66} crc {auth=381911012,config=692619927,health=92814587,kv=2863082231,logm=2802275143}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=9,config=2,health=11,kv=8,logm=70} crc {auth=3875858248,config=692619927,health=3788543542,kv=2974220280,logm=1995090561}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=10,config=2,health=11,kv=9,logm=68} crc {auth=1007457286,config=692619927,health=2791193012,kv=1551675710,logm=1716403286}) 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.490 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=2738329388,config=692619927,health=2662292419,kv=2863082231,logm=1700978053}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=10,config=2,health=12,kv=9,logm=67} crc {auth=1196568339,config=692619927,health=3190447941,kv=732018829,logm=467804540}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=11,config=2,health=12,kv=9,logm=66} crc {auth=3590491517,config=692619927,health=92814587,kv=80448710,logm=3367946564}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=3590491517,config=692619927,health=4005139485,kv=3818608592,logm=24006503}) 2024-09-17T16:33:01.491 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.481+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=11,config=2,health=10,kv=9,logm=68} crc {auth=3590491517,config=692619927,health=123206766,kv=732018829,logm=1132285276}) 2024-09-17T16:33:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 10 librados: watch_flush enter 2024-09-17T16:33:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 10 librados: watch_flush exit 2024-09-17T16:33:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 20 client.23071.objecter shutdown clearing up homeless session... 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 10 client.23071.objecter successfully canceled tick 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 msgr2=0x7fb28c040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 0x7fb28c040a00 secure :-1 s=READY pgs=3436 cs=0 l=1 rev1=1 crypto rx=0x7fb298009630 tx=0x7fb298009240 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 10 monclient: shutdown 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 msgr2=0x7fb2a81d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 0x7fb2a81d1af0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ab000 tx=0x7fb2a40190b0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 shutdown_connections 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb28c03e540 0x7fb28c040a00 unknown :-1 s=CLOSED pgs=3436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb2a80a68e0 0x7fb2a81d1af0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2a81396a0 0x7fb2a813d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 --2- 172.21.15.105:0/435270731 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb2a80a4910 0x7fb2a81d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 >> 172.21.15.105:0/435270731 conn(0x7fb2a8068ba0 msgr2=0x7fb2a80a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 shutdown_connections 2024-09-17T16:33:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 -- 172.21.15.105:0/435270731 wait complete. 2024-09-17T16:33:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:01.487+0000 7fb2af760640 1 librados: shutdown 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.485+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1808532808}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4020402960}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527572357}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=210635014}) 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.495 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800326051}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=757425645}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=75429777}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1698190316}) 2024-09-17T16:33:01.496 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3866192015}) 2024-09-17T16:33:01.498 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.498 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.498 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687485008}) 2024-09-17T16:33:01.498 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.498 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=880088053}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.489+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779900191}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=824168920}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003181451}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2673481809}) 2024-09-17T16:33:01.499 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.500 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.500 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1394616351}) 2024-09-17T16:33:01.500 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.500 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072158134}) 2024-09-17T16:33:01.500 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.493+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2539321846}) 2024-09-17T16:33:01.505 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141777542}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=835291142}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318252437}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=971520751}) 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.506 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1850124622}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=951439091}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1841462904}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252096439}) 2024-09-17T16:33:01.507 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.497+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2108753753}) 2024-09-17T16:33:01.508 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.508 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1124712978}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=631505876}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2546569766}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3006807800}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2901926764}) 2024-09-17T16:33:01.509 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3567542809}) 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1949864036}) 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601254578}) 2024-09-17T16:33:01.510 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.501+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1468897691}) 2024-09-17T16:33:01.511 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=354710418}) 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.512 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=512213714}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1740350124}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3493800687}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2824998696}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.513 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2655129810}) 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2210147814}) 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847656199}) 2024-09-17T16:33:01.514 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.505+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2493095324}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2430589622}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4168963386}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3503715981}) 2024-09-17T16:33:01.516 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4261896393}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2341668922}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1168236351}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1974248603}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898893807}) 2024-09-17T16:33:01.517 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.509+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2485971625}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436156278}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2792495830}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.521 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1668881810}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=286841815}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2768033349}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1857923690}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3588084825}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2635399674}) 2024-09-17T16:33:01.522 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.513+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1237282295}) 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2630715541}) 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2847117652}) 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.524 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3961503569}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1785795472}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4220329694}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2417710222}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1502237092}) 2024-09-17T16:33:01.525 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.526 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930670716}) 2024-09-17T16:33:01.526 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.517+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=904878335}) 2024-09-17T16:33:01.528 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.528 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.528 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821058543}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3172026867}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1005071074}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1563317924}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358443699}) 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.529 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1567652077}) 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=105358280}) 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198754925}) 2024-09-17T16:33:01.530 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.521+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=651199898}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=449560386}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=929938459}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=44024534}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.534 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2347843225}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=969635663}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1127240421}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1459846031}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258784161}) 2024-09-17T16:33:01.535 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.525+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3566588647}) 2024-09-17T16:33:01.549 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.550 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.550 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3332841949}) 2024-09-17T16:33:01.552 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.552 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.552 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2283110152}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1713010774}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2997771130}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2424922121}) 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.553 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1846608834}) 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2540062698}) 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3619551123}) 2024-09-17T16:33:01.554 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.541+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=244988003}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=78371670}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1528583999}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.570 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1049467414}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3844345895}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1117537635}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1548615184}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2622959417}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3807453795}) 2024-09-17T16:33:01.571 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.561+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=655038017}) 2024-09-17T16:33:01.573 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.573 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.573 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1109396904}) 2024-09-17T16:33:01.573 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2971243431}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4261164217}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2551243515}) 2024-09-17T16:33:01.574 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1194706703}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=54844748}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=566532890}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3065555384}) 2024-09-17T16:33:01.575 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.565+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3430375228}) 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=550991851}) 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2803240093}) 2024-09-17T16:33:01.577 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1623789240}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4103426749}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614259586}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4143356311}) 2024-09-17T16:33:01.578 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.579 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.579 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=324220965}) 2024-09-17T16:33:01.579 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.579 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524171300}) 2024-09-17T16:33:01.579 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3107695775}) 2024-09-17T16:33:01.583 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.583 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.583 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1042758954}) 2024-09-17T16:33:01.583 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.583 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.569+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203350511}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4092585863}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2114555017}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=399546697}) 2024-09-17T16:33:01.584 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3825601063}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1491942722}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490856395}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4112765198}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1991725797}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.585 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2241653028}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.573+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1686353576}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=985222760}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776996309}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1725704022}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.586 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2278351679}) 2024-09-17T16:33:01.587 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.587 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3718878377}) 2024-09-17T16:33:01.587 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.577+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2010909153}) 2024-09-17T16:33:01.590 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3858728408}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977740442}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2065262361}) 2024-09-17T16:33:01.591 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=190233435}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1680066614}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=947463929}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3028314322}) 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.592 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=729668815}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=312368567}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1363072560}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270849822}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2793335034}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3568566744}) 2024-09-17T16:33:01.593 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3984366592}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3284194340}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=267461871}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367670242}) 2024-09-17T16:33:01.594 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.581+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1972063083}) 2024-09-17T16:33:01.599 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3550688691}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632726910}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.589+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=975612765}) 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.600 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4112114945}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=232690489}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3044363820}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2377196923}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3037286942}) 2024-09-17T16:33:01.601 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3786418969}) 2024-09-17T16:33:01.609 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.609 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1920958513}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3440701556}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1300255405}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3104285392}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99057498}) 2024-09-17T16:33:01.610 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3392547807}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=732187317}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6191143}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.593+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2354003841}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=61755720}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.611 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2712233080}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=469321460}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=48856644}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3513126031}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1390907962}) 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.612 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3040380392}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403908463}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1641400693}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1706373531}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1751435355}) 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.613 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=879824312}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3497502652}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3321169955}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.597+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3709731062}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1534013597}) 2024-09-17T16:33:01.614 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948517015}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2072940932}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2894273091}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3818995679}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1017197516}) 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.615 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3690190887}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=123952451}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=10807034}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=19977407}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1925208635}) 2024-09-17T16:33:01.616 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.601+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2855099363}) 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=558722703}) 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1839340528}) 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.625 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2886299932}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3492874508}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=656990072}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3794212041}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3505127276}) 2024-09-17T16:33:01.626 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3858305545}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3371128599}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2513279752}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120503983}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4028356706}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1903069394}) 2024-09-17T16:33:01.627 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688336977}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1997888996}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4741710}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261212048}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.605+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1276016117}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.628 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=870350272}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=147442924}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=486489788}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3689800231}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3343906208}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.629 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2649418475}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3085238718}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=639980479}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.609+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1976378855}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3309090400}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3087383597}) 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.630 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2881983408}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4120832651}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103184734}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2669648704}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2065540008}) 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.631 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11773158}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1567176851}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1646033960}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3647852644}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1027542562}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1341879489}) 2024-09-17T16:33:01.632 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2369779253}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.613+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3769906096}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2564349610}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099550292}) 2024-09-17T16:33:01.633 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.617+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2188438453}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.621+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.621+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=779303109}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1349456420}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.635 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=881235073}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=564860800}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596848663}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=291465806}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.636 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3538140014}) 2024-09-17T16:33:01.637 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.637 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4121869114}) 2024-09-17T16:33:01.637 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.625+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1672334783}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461866181}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1421251555}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2158960396}) 2024-09-17T16:33:01.639 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3665555847}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.629+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3527874585}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=146434740}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2462220017}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615221269}) 2024-09-17T16:33:01.640 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.633+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1614672294}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3247568353}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633669963}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.706 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1823280584}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2117299604}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3209419112}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3938753664}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645415794}) 2024-09-17T16:33:01.707 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431268654}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.673+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=209927282}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794861418}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3105881811}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2186141249}) 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.708 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3478491155}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4012836375}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3099159295}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1948861702}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2592078074}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2291437510}) 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.709 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3945138524}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3748209539}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2913542834}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1120843533}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1572260248}) 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.710 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285344567}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3770720583}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3806010672}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.677+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1731986449}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947578928}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.711 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3366209160}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4199945634}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=286625854}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2166306368}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1293253914}) 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.712 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2529160366}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3032924048}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1316818578}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122593494}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3616438931}) 2024-09-17T16:33:01.713 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3873640943}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=476465019}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=986998216}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2433118383}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.714 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=732391302}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1264432376}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2851002144}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4010665143}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=964111433}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=589258118}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.715 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1741897272}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2924932320}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008496770}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=895359870}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898330154}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.681+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=903219870}) 2024-09-17T16:33:01.716 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4003899505}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=464691588}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=651048708}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3850466776}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4140165119}) 2024-09-17T16:33:01.717 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=584695734}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=414026885}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=969908223}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2343556810}) 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.718 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2882261079}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3888622139}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3861551691}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2206971844}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1429221940}) 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.719 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2811271615}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1321723361}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3580858548}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.685+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994390281}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=24,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=2} crc {logm=328982513,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=67365634,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=598337686,osd_metadata=352991311,osd_pg_creating=1685398224,osd_snap=875478172,osdmap=2544935827}) 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.720 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=22,osdmap=6} crc {logm=916210817,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2387148642,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=4058618413,osd_pg_creating=1685398224,osd_snap=3020597845,osdmap=966359910}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=5} crc {logm=2067857660,mdsmap=1248375647,mgr=1287546803,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1365694516,mon_config_key=3263060063,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1685398224,osd_snap=3650874665,osdmap=475726448}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=23,osdmap=4} crc {logm=1339582128,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1685398224,osd_snap=3727728798,osdmap=3522916766}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=23,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28,osdmap=3} crc {logm=3697938793,mdsmap=1502663596,mgr=702605724,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1370163439,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1685398224,osd_snap=875478172,osdmap=1073827092}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=21,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28,osdmap=3} crc {logm=1090757732,mdsmap=952268285,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2236978998,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1685398224,osd_snap=875478172,osdmap=1073827092}) 2024-09-17T16:33:01.721 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=24,mdsmap=3,mgr=6,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=26,osdmap=6} crc {logm=2017192974,mdsmap=1248375647,mgr=3050418281,mgr_metadata=3792311847,mgrstat=1558221751,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=2078864477,osd_pg_creating=1685398224,osd_snap=1207234049,osdmap=966359910}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=25,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=27,osdmap=2} crc {logm=518408812,mdsmap=1248375647,mgr=3159212071,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1168059636,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=26828413,osd_pg_creating=1685398224,osd_snap=767466185,osdmap=2544935827}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.689+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=25,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=1} crc {logm=916210817,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3079432290,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1685398224,osd_snap=785670441,osdmap=1118318926}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1423331038}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1981386162}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.722 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2144630616}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1801100454}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2911697476}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2243675208}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=654941143}) 2024-09-17T16:33:01.723 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2038698251}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2737820884}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=638302200}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=970693538}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1482853343}) 2024-09-17T16:33:01.724 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3022285187}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1352622561}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=881440751}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.725 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2169644187}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2483174634}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.693+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3904633661}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3197822617}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1691538410}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=523338012}) 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.726 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1742214722}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2220911359}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=606359795}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1320969824}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1159761628}) 2024-09-17T16:33:01.727 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.697+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=99163823}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=30} crc {osdmap=3114451632}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=32} crc {osdmap=807184288}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.729 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=31} crc {osdmap=3018492094}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=31} crc {osdmap=3375464012}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=32} crc {osdmap=807184288}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=31} crc {osdmap=1463885337}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=31} crc {osdmap=4025736091}) 2024-09-17T16:33:01.730 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:33:01.717+0000 7f60c3498640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=32} crc {osdmap=807184288}) 2024-09-17T16:33:02.511 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:33:02.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 10 monclient: get_monmap_and_config 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 10 monclient: build_initial_monmap 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 10 monclient: monmap: 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:02.645873+0000 2024-09-17T16:33:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:02.645873+0000 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:02.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 1 Processor -- start 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 1 -- start start 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.643+0000 7f179ad0c640 10 monclient: init 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient: _add_conns ranks=[6,2,8,4,1,9,0,7,10,5,3] 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 0x7f17940a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-d con 0x7f179406c110 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:02.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-b con 0x7f179406c4f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 0x7f1794137950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-e con 0x7f1794137510 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f17940a22a0 con 0x7f179406c4f0 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f17940a10a0 con 0x7f179406c110 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f17940a1380 con 0x7f1794137510 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient(hunting): authenticate will time out at 1265.912628s 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53058/0 (socket says 172.21.15.105:53058) 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 1 -- 172.21.15.105:0/1041248183 learned_addr learned my addr 172.21.15.105:0/1041248183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): get_auth_request con 0x7f179406c4f0 auth_method 0 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 0x7f1794137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 0x7f17940a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): get_auth_request con 0x7f1794137510 auth_method 0 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): get_auth_request con 0x7f179406c110 auth_method 0 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient(hunting): handle_auth_done global_id 23937 payload 274 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 -- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 msgr2=0x7f1794137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 0x7f1794137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 -- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 msgr2=0x7f179406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient: _finish_hunting 0 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 monclient: found mon.noname-d 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 -- 172.21.15.105:0/1041248183 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17940a1660 con 0x7f179406c110 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17992a9640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 0x7f1794137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1793fff640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f1798aa8640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 0x7f17940a1d60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f17880097f0 tx=0x7f178801cda0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=919f95b8c1c15365 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 1 -- 172.21.15.105:0/1041248183 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17880b0070 con 0x7f179406c110 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: dump: 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 1 -- 172.21.15.105:0/1041248183 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17880aa810 con 0x7f179406c110 2024-09-17T16:33:02.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 5 monclient: authenticate success, global_id 23937 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 1 -- 172.21.15.105:0/1041248183 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17880a9300 con 0x7f179406c110 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f179ad0c640 4 set_mon_vals no callback set 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.647+0000 7f17937fe640 10 monclient: dump: 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:02.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: shutdown 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 msgr2=0x7f17940a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 0x7f17940a1d60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f17880097f0 tx=0x7f178801cda0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- 172.21.15.105:0/1041248183 shutdown_connections 2024-09-17T16:33:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1794137510 0x7f1794137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f179406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- 172.21.15.105:0/1041248183 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f179406c110 0x7f17940a1d60 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- 172.21.15.105:0/1041248183 >> 172.21.15.105:0/1041248183 conn(0x7f17940a5840 msgr2=0x7f17940a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- 172.21.15.105:0/1041248183 shutdown_connections 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- 172.21.15.105:0/1041248183 wait complete. 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: build_initial_monmap 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: monmap: 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 librados: starting msgr at 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 librados: starting objecter 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 Processor -- start 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- start start 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 librados: setting wanted keys 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 librados: calling monclient init 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: init 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient: _add_conns ranks=[7,2,4,5,8,0,3,1,6] 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f179406c110 0x7f1794136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-i con 0x7f179406c110 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f1794136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-b con 0x7f179406c4f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 0x7f17941d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): picked mon.noname-c con 0x7f1794137510 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1794141190 con 0x7f179406c4f0 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f17940a4a10 con 0x7f1794137510 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f1794141310 con 0x7f179406c110 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f179ad0c640 10 monclient(hunting): authenticate will time out at 1265.918824s 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f1794136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f1794136760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53064/0 (socket says 172.21.15.105:53064) 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 1 -- 172.21.15.105:0/4276448981 learned_addr learned my addr 172.21.15.105:0/4276448981 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): get_auth_request con 0x7f179406c4f0 auth_method 0 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 0x7f17941d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): get_auth_request con 0x7f1794137510 auth_method 0 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1798aa8640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f179406c110 0x7f1794136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1798aa8640 10 monclient(hunting): get_auth_request con 0x7f179406c110 auth_method 0 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1798aa8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1798aa8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f1798aa8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient(hunting): handle_auth_done global_id 23080 payload 824 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 -- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f179406c110 msgr2=0x7f1794136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f179406c110 0x7f1794136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 -- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 msgr2=0x7f1794136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f1794136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient: _finish_hunting 0 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 monclient: found mon.noname-c 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 -- 172.21.15.105:0/4276448981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179413dab0 con 0x7f1794137510 2024-09-17T16:33:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.651+0000 7f17992a9640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 0x7f17941d1330 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f178c00ef90 tx=0x7f178c00c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.655+0000 7f17917fa640 10 client.?.objecter ms_handle_connect 0x7f1794137510 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.655+0000 7f17917fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f178c018020 con 0x7f1794137510 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: dump: 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:02.674 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:02.675 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: _finish_auth 0 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: _check_auth_tickets 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c00cd30 con 0x7f1794137510 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 5 monclient: authenticate success, global_id 23080 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _renew_subs 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17940a1e40 con 0x7f1794137510 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f178c014d60 con 0x7f1794137510 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 monclient: dump: 2024-09-17T16:33:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f1792ffd640 4 set_mon_vals no callback set 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 client.23080.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _renew_subs 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1794123fb0 con 0x7f1794137510 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 1 librados: init done 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f178c01b020 con 0x7f1794137510 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179413d810 con 0x7f1794137510 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 0x7f1778040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(217..217 src has 1..217) ==== 6556+0+0 (secure 0 0 0) 0x7f178c050dc0 con 0x7f1794137510 2024-09-17T16:33:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 10 client.23080.objecter ms_dispatch 0x7f1794000ba0 osd_map(217..217 src has 1..217) 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 3 client.23080.objecter handle_osd_map got epochs [217,217] > 0 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.659+0000 7f17917fa640 3 client.23080.objecter handle_osd_map decoding full epoch 217 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.663+0000 7f17917fa640 20 client.23080.objecter dump_active .. 0 homeless 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.663+0000 7f1798aa8640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 0x7f1778040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.663+0000 7f1798aa8640 10 monclient: get_auth_request con 0x7f177803e550 auth_method 0 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.663+0000 7f17917fa640 10 client.23080.objecter ms_handle_connect 0x7f177803e550 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.663+0000 7f1798aa8640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 0x7f1778040a10 secure :-1 s=READY pgs=3438 cs=0 l=1 rev1=1 crypto rx=0x7f17880ad000 tx=0x7f1788004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.667+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f178c022030 con 0x7f1794137510 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.667+0000 7f17917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.667+0000 7f17917fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f179ad0c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:33:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f179ad0c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f179ad0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f179406c910 con 0x7f1794137510 2024-09-17T16:33:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f17917fa640 1 -- 172.21.15.105:0/4276448981 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f17940a1e40 con 0x7f1794137510 2024-09-17T16:33:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.903+0000 7f17917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f17917fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:02.910 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":46,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":8,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 10 librados: watch_flush enter 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 10 librados: watch_flush exit 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 20 client.23080.objecter shutdown clearing up homeless session... 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 10 client.23080.objecter successfully canceled tick 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 msgr2=0x7f1778040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 0x7f1778040a10 secure :-1 s=READY pgs=3438 cs=0 l=1 rev1=1 crypto rx=0x7f17880ad000 tx=0x7f1788004740 comp rx=0 tx=0).stop 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 10 monclient: shutdown 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 msgr2=0x7f17941d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 0x7f17941d1330 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f178c00ef90 tx=0x7f178c00c490 comp rx=0 tx=0).stop 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 shutdown_connections 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f177803e550 0x7f1778040a10 unknown :-1 s=CLOSED pgs=3438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1794137510 0x7f17941d1330 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f179406c4f0 0x7f1794136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 --2- 172.21.15.105:0/4276448981 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f179406c110 0x7f1794136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 >> 172.21.15.105:0/4276448981 conn(0x7f17940a5840 msgr2=0x7f17940a7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 shutdown_connections 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 -- 172.21.15.105:0/4276448981 wait complete. 2024-09-17T16:33:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:02.907+0000 7f179ad0c640 1 librados: shutdown 2024-09-17T16:33:02.929 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:33:02.929 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:33:02.929 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:33:02.929 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: get_monmap_and_config 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: monmap: 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:03.048369+0000 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:03.048369+0000 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:03.051 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:03.052 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 Processor -- start 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 -- start start 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: init 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient: _add_conns ranks=[6,3,4,0,7,9,5,10,2,1,8] 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-d con 0x7f70ec0a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 0x7f70ec0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-g con 0x7f70ec0a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 0x7f70ec139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-c con 0x7f70ec1397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f70ec0a7c40 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f70ec0a7890 con 0x7f70ec1397f0 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f70ec0a4080 con 0x7f70ec0a6a30 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f47d2640 10 monclient(hunting): authenticate will time out at 1266.314968s 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36500/0 (socket says 172.21.15.105:36500) 2024-09-17T16:33:03.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 -- 172.21.15.105:0/1540569497 learned_addr learned my addr 172.21.15.105:0/1540569497 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): get_auth_request con 0x7f70ec0a6a30 auth_method 0 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 0x7f70ec139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): get_auth_request con 0x7f70ec1397f0 auth_method 0 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 0x7f70ec0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient(hunting): handle_auth_done global_id 23946 payload 274 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 -- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 msgr2=0x7f70ec139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 0x7f70ec139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 -- 172.21.15.105:0/1540569497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 msgr2=0x7f70ec0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 0x7f70ec0a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 monclient: found mon.noname-d 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f256e640 1 -- 172.21.15.105:0/1540569497 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec0a37e0 con 0x7f70ec0a6a30 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f1d6d640 10 monclient: get_auth_request con 0x7f70ec0a4ab0 auth_method 0 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f2d6f640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 0x7f70ec139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.047+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 0x7f70ec0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f256e640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098a0 tx=0x7f70e801c410 comp rx=0 tx=0).ready entity=mon.6 client_cookie=66247b0d3184e9c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 1 -- 172.21.15.105:0/1540569497 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80b1070 con 0x7f70ec0a6a30 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: dump: 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 1 -- 172.21.15.105:0/1540569497 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e80aa7b0 con 0x7f70ec0a6a30 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 5 monclient: authenticate success, global_id 23946 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 1 -- 172.21.15.105:0/1540569497 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80aae10 con 0x7f70ec0a6a30 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 4 set_mon_vals no callback set 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f156c640 10 monclient: dump: 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 10 monclient: shutdown 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 msgr2=0x7f70ec0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098a0 tx=0x7f70e801c410 comp rx=0 tx=0).stop 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- 172.21.15.105:0/1540569497 shutdown_connections 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70ec1397f0 0x7f70ec139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec0a4ab0 0x7f70ec0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1540569497 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a6a30 0x7f70ec0a6e10 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- 172.21.15.105:0/1540569497 >> 172.21.15.105:0/1540569497 conn(0x7f70ec0a8d00 msgr2=0x7f70ec0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- 172.21.15.105:0/1540569497 shutdown_connections 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- 172.21.15.105:0/1540569497 wait complete. 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 10 monclient: monmap: 2024-09-17T16:33:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 librados: starting msgr at 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 librados: starting objecter 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 Processor -- start 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 -- start start 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 librados: setting wanted keys 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 1 librados: calling monclient init 2024-09-17T16:33:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.051+0000 7f70f47d2640 10 monclient: init 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _add_conns ranks=[6,1,3,7,2,4,0,8,5] 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 0x7f70ec1d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-d con 0x7f70ec0a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ec0a6a30 0x7f70ec1d1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-f con 0x7f70ec0a6a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec1397f0 0x7f70ec13d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): picked mon.noname-g con 0x7f70ec1397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec06e770 con 0x7f70ec0a6a30 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f70ec13a5a0 con 0x7f70ec1397f0 2024-09-17T16:33:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f70ec143260 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): authenticate will time out at 1266.320285s 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 0x7f70ec1d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 0x7f70ec1d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36506/0 (socket says 172.21.15.105:36506) 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 -- 172.21.15.105:0/1916866761 learned_addr learned my addr 172.21.15.105:0/1916866761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): get_auth_request con 0x7f70ec0a4ab0 auth_method 0 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec1397f0 0x7f70ec13d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ec0a6a30 0x7f70ec1d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 10 monclient(hunting): get_auth_request con 0x7f70ec1397f0 auth_method 0 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient(hunting): handle_auth_done global_id 23955 payload 824 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec1397f0 msgr2=0x7f70ec13d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70ec1397f0 0x7f70ec13d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ec0a6a30 msgr2=0x7f70ec1d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ec0a6a30 0x7f70ec1d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 monclient: found mon.noname-d 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec13ef40 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient: get_auth_request con 0x7f70ec0a6a30 auth_method 0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70ec0a6a30 0x7f70ec1d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f256e640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 0x7f70ec1d13e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c240 tx=0x7f70e80191b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.?.objecter ms_handle_connect 0x7f70ec0a4ab0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80ba070 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: dump: 2024-09-17T16:33:03.083 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.084 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e80aadd0 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 5 monclient: authenticate success, global_id 23955 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _renew_subs 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec0aa100 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80b1040 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f0d6b640 4 set_mon_vals no callback set 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 client.23955.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: dump: 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.086 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _renew_subs 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec13faa0 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 librados: init done 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 0x7f70ec13de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec051e80 con 0x7f70ec13da80 2024-09-17T16:33:03.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f70e80043a0 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 0x7f70ec13de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient: get_auth_request con 0x7f70ec13da80 auth_method 0 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70d003e580 0x7f70d0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(217..217 src has 1..217) ==== 6556+0+0 (secure 0 0 0) 0x7f70e80e91b0 con 0x7f70ec0a4ab0 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.23955.objecter ms_dispatch 0x7f70ec000ba0 osd_map(217..217 src has 1..217) 2024-09-17T16:33:03.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 3 client.23955.objecter handle_osd_map got epochs [217,217] > 0 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 3 client.23955.objecter handle_osd_map decoding full epoch 217 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_done global_id 34866 payload 824 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70ec0abc30 con 0x7f70ec13da80 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 20 client.23955.objecter dump_active .. 0 homeless 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.23955.objecter ms_handle_connect 0x7f70ec13da80 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.23955.objecter resend_mon_ops 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 0x7f70ec13de60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00d760 tx=0x7f70dc00dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70d003e580 0x7f70d0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 10 monclient: get_auth_request con 0x7f70d003e580 auth_method 0 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70dc016070 con 0x7f70ec13da80 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: dump: 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f70dc00de10 con 0x7f70ec13da80 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 msgr2=0x7f70ec13de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 0x7f70ec13de60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00d760 tx=0x7f70dc00dc20 comp rx=0 tx=0).stop 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 msgr2=0x7f70ec13de60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13da80 0x7f70ec13de60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70e2ffd640 10 client.23955.objecter ms_handle_connect 0x7f70d003e580 2024-09-17T16:33:03.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.055+0000 7f70f2d6f640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70d003e580 0x7f70d0040a40 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 crypto rx=0x7f70ec13f370 tx=0x7f70e400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 0x7f70ec0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec06a200 con 0x7f70ec13e3a0 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 0x7f70ec0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient: get_auth_request con 0x7f70ec13e3a0 auth_method 0 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 10 monclient(hunting): handle_auth_done global_id 34875 payload 824 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 1 -- 172.21.15.105:0/1916866761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70ec139c50 con 0x7f70ec13e3a0 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f1d6d640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 0x7f70ec0a0a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 client.23955.objecter ms_handle_connect 0x7f70ec13e3a0 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 client.23955.objecter resend_mon_ops 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70dc01cdb0 con 0x7f70ec13e3a0 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 monclient: dump: 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.095 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f70dc012460 con 0x7f70ec13e3a0 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 msgr2=0x7f70ec0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 0x7f70ec0a0a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc0046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 msgr2=0x7f70ec0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70e2ffd640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70ec13e3a0 0x7f70ec0a0a20 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70dc000c10 tx=0x7f70dc0046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 173866, 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:03.097 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:03.098 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.099 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.100 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.101 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:03.103 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.104 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.106 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:03.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:03.125 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 librados: watch_flush enter 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 librados: watch_flush exit 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 20 client.23955.objecter shutdown clearing up homeless session... 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 client.23955.objecter successfully canceled tick 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70d003e580 msgr2=0x7f70d0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70d003e580 0x7f70d0040a40 secure :-1 s=READY pgs=3439 cs=0 l=1 rev1=1 crypto rx=0x7f70ec13f370 tx=0x7f70e400b040 comp rx=0 tx=0).stop 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 10 monclient: shutdown 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 msgr2=0x7f70ec1d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 --2- 172.21.15.105:0/1916866761 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70ec0a4ab0 0x7f70ec1d13e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c240 tx=0x7f70e80191b0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f2d6f640 1 -- 172.21.15.105:0/1916866761 reap_dead start 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f2d6f640 1 -- 172.21.15.105:0/1916866761 reap_dead start 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 shutdown_connections 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 >> 172.21.15.105:0/1916866761 conn(0x7f70ec0a8d00 msgr2=0x7f70ec0a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 shutdown_connections 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 -- 172.21.15.105:0/1916866761 wait complete. 2024-09-17T16:33:03.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.063+0000 7f70f47d2640 1 librados: shutdown 2024-09-17T16:33:03.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:03.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:33:03.128 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: get_monmap_and_config 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: monmap: 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:03.247900+0000 2024-09-17T16:33:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:03.247900+0000 2024-09-17T16:33:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:03.253 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 Processor -- start 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 -- start start 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: init 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient: _add_conns ranks=[1,7,3,4,10,0,9,8,2,6,5] 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b380a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8b380a4ab0 0x7f8b3813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-i con 0x7f8b380a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 0x7f8b38140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-g con 0x7f8b380a7520 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b380a4080 con 0x7f8b380a6a30 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380a7c40 con 0x7f8b380a7520 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8b380a37e0 con 0x7f8b380a4ab0 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3e6f8640 10 monclient(hunting): authenticate will time out at 1266.513630s 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b37fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8b380a6a30 0x7f8b380a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:46124/0 (socket says 172.21.15.105:46124) 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b37fff640 1 -- 172.21.15.105:0/2751775236 learned_addr learned my addr 172.21.15.105:0/2751775236 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 0x7f8b38140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8b380a4ab0 0x7f8b3813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): get_auth_request con 0x7f8b380a7520 auth_method 0 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): get_auth_request con 0x7f8b380a4ab0 auth_method 0 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b36ffd640 1 -- 172.21.15.105:0/2751775236 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274570317 0 0) 0x7f8b380a4080 con 0x7f8b380a6a30 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient(hunting): handle_auth_done global_id 19443 payload 274 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 -- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8b380a4ab0 msgr2=0x7f8b3813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8b380a4ab0 0x7f8b3813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 -- 172.21.15.105:0/2751775236 >> v1:172.21.15.160:6789/0 conn(0x7f8b380a6a30 legacy=0x7f8b380a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 monclient: found mon.noname-g 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b3cc95640 1 -- 172.21.15.105:0/2751775236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380a3ac0 con 0x7f8b380a7520 2024-09-17T16:33:03.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.247+0000 7f8b36ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 0x7f8b38140f80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009dc0 tx=0x7f8b3001c810 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a6743b1c97db63ab server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 1 -- 172.21.15.105:0/2751775236 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b300b2070 con 0x7f8b380a7520 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: dump: 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.269 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.270 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.271 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.271 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.271 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.271 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 1 -- 172.21.15.105:0/2751775236 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b300a5b60 con 0x7f8b380a7520 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 5 monclient: authenticate success, global_id 19443 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 1 -- 172.21.15.105:0/2751775236 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b300a5d50 con 0x7f8b380a7520 2024-09-17T16:33:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b36ffd640 10 monclient: dump: 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.273 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 4 set_mon_vals no callback set 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: shutdown 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 msgr2=0x7f8b38140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 0x7f8b38140f80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009dc0 tx=0x7f8b3001c810 comp rx=0 tx=0).stop 2024-09-17T16:33:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/2751775236 shutdown_connections 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a7520 0x7f8b38140f80 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/2751775236 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8b380a4ab0 0x7f8b3813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/2751775236 >> 172.21.15.105:0/2751775236 conn(0x7f8b380a8d00 msgr2=0x7f8b380a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/2751775236 shutdown_connections 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/2751775236 wait complete. 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: monmap: 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.276 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 librados: starting msgr at 2024-09-17T16:33:03.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 librados: starting objecter 2024-09-17T16:33:03.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 Processor -- start 2024-09-17T16:33:03.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 -- start start 2024-09-17T16:33:03.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 librados: setting wanted keys 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 1 librados: calling monclient init 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: init 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.251+0000 7f8b3e6f8640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient: _add_conns ranks=[2,3,1,8,5,6,7,0,4] 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-b con 0x7f8b380a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a6a30 0x7f8b381265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-g con 0x7f8b380a6a30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b380a7520 0x7f8b38126b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): picked mon.noname-f con 0x7f8b380a7520 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38143e60 con 0x7f8b380a7520 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8b38143ce0 con 0x7f8b380a4ab0 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8b38143fe0 con 0x7f8b380a6a30 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): authenticate will time out at 1266.520077s 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53074/0 (socket says 172.21.15.105:53074) 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 -- 172.21.15.105:0/3161125489 learned_addr learned my addr 172.21.15.105:0/3161125489 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): get_auth_request con 0x7f8b380a4ab0 auth_method 0 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b380a7520 0x7f8b38126b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a6a30 0x7f8b381265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 10 monclient(hunting): get_auth_request con 0x7f8b380a7520 auth_method 0 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): get_auth_request con 0x7f8b380a6a30 auth_method 0 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient(hunting): handle_auth_done global_id 44183 payload 824 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a6a30 msgr2=0x7f8b381265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380a6a30 0x7f8b381265e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b380a7520 msgr2=0x7f8b38126b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b380a7520 0x7f8b38126b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 monclient: found mon.noname-b 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b38128f60 con 0x7f8b380a4ab0 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b37fff640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c480 tx=0x7f8b2c00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.?.objecter ms_handle_connect 0x7f8b380a4ab0 2024-09-17T16:33:03.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b2c018070 con 0x7f8b380a4ab0 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: dump: 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 5 monclient: authenticate success, global_id 44183 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0047f0 con 0x7f8b380a4ab0 2024-09-17T16:33:03.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient: _renew_subs 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b38127a70 con 0x7f8b380a4ab0 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b2c004e50 con 0x7f8b380a4ab0 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 client.44183.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b367fc640 4 set_mon_vals no callback set 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: dump: 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.284 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.285 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient: _renew_subs 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b38129530 con 0x7f8b380a4ab0 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 librados: init done 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 0x7f8b380a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8b38051e80 con 0x7f8b38127060 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 0x7f8b380a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient: get_auth_request con 0x7f8b38127060 auth_method 0 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8b2c0049e0 con 0x7f8b380a4ab0 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 10 monclient(hunting): handle_auth_done global_id 44192 payload 824 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b2403e620 0x7f8b24040ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b380a6e10 con 0x7f8b38127060 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(217..217 src has 1..217) ==== 6556+0+0 (secure 0 0 0) 0x7f8b2c04f1d0 con 0x7f8b380a4ab0 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.44183.objecter ms_dispatch 0x7f8b38000ba0 osd_map(217..217 src has 1..217) 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 3 client.44183.objecter handle_osd_map got epochs [217,217] > 0 2024-09-17T16:33:03.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 3 client.44183.objecter handle_osd_map decoding full epoch 217 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 20 client.44183.objecter dump_active .. 0 homeless 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.44183.objecter ms_handle_connect 0x7f8b38127060 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.44183.objecter resend_mon_ops 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b377fe640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 0x7f8b380a0a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a7e50 tx=0x7f8b28011a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b2403e620 0x7f8b24040ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 10 monclient: get_auth_request con 0x7f8b2403e620 auth_method 0 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b280044a0 con 0x7f8b38127060 2024-09-17T16:33:03.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: dump: 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8b28004030 con 0x7f8b38127060 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 msgr2=0x7f8b380a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 0x7f8b380a0a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a7e50 tx=0x7f8b28011a60 comp rx=0 tx=0).stop 2024-09-17T16:33:03.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 msgr2=0x7f8b380a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b38127060 0x7f8b380a0a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b34ff9640 10 client.44183.objecter ms_handle_connect 0x7f8b2403e620 2024-09-17T16:33:03.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.255+0000 7f8b3cc95640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b2403e620 0x7f8b24040ae0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f8b300af000 tx=0x7f8b300a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 0x7f8b380a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3806a200 con 0x7f8b380a0f60 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 0x7f8b380a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient: get_auth_request con 0x7f8b380a0f60 auth_method 0 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 10 monclient(hunting): handle_auth_done global_id 44201 payload 824 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 1 -- 172.21.15.105:0/3161125489 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8b38129a10 con 0x7f8b380a0f60 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b377fe640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 0x7f8b380a1370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a7e50 tx=0x7f8b280046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 client.44183.objecter ms_handle_connect 0x7f8b380a0f60 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 client.44183.objecter resend_mon_ops 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b28003a90 con 0x7f8b380a0f60 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 monclient: dump: 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.296 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f8b28003620 con 0x7f8b380a0f60 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 msgr2=0x7f8b380a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 0x7f8b380a1370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a7e50 tx=0x7f8b280046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 msgr2=0x7f8b380a1370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b34ff9640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a0f60 0x7f8b380a1370 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b380a7e50 tx=0x7f8b280046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.298 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:03.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.298 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 33915, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:03.299 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.300 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.301 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:03.302 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.303 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:03.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.307 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.307 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.309 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.312 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.316 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:03.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:03.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:03.323 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.324 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.324 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:03.324 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 librados: watch_flush enter 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 librados: watch_flush exit 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 20 client.44183.objecter shutdown clearing up homeless session... 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 client.44183.objecter successfully canceled tick 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b2403e620 msgr2=0x7f8b24040ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b2403e620 0x7f8b24040ae0 secure :-1 s=READY pgs=3440 cs=0 l=1 rev1=1 crypto rx=0x7f8b300af000 tx=0x7f8b300a7070 comp rx=0 tx=0).stop 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 10 monclient: shutdown 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 msgr2=0x7f8b381260a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c480 tx=0x7f8b2c00c940 comp rx=0 tx=0).stop 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3cc95640 1 -- 172.21.15.105:0/3161125489 reap_dead start 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 shutdown_connections 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 --2- 172.21.15.105:0/3161125489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b380a4ab0 0x7f8b381260a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 >> 172.21.15.105:0/3161125489 conn(0x7f8b380a8d00 msgr2=0x7f8b380aac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 shutdown_connections 2024-09-17T16:33:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 -- 172.21.15.105:0/3161125489 wait complete. 2024-09-17T16:33:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.263+0000 7f8b3e6f8640 1 librados: shutdown 2024-09-17T16:33:03.327 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:03.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:33:03.327 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:33:03.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.443+0000 7fd879855640 10 monclient: get_monmap_and_config 2024-09-17T16:33:03.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.443+0000 7fd879855640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.443+0000 7fd879855640 10 monclient: monmap: 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:03.446937+0000 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:03.446937+0000 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.450 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:03.452 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 Processor -- start 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 -- start start 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient: init 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient: _add_conns ranks=[5,8,0,6,3,1,9,7,4,2,10] 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8740a3720 0x7fd8740a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): picked mon.noname-h con 0x7fd8740a3720 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 0x7fd8740a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): picked mon.noname-e con 0x7fd8740a3b00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8740a3f00 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87406db50 con 0x7fd8740a3f00 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd8740a0e80 con 0x7fd8740a3720 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fd8740a1490 con 0x7fd8740a3b00 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient(hunting): authenticate will time out at 1266.712986s 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8737fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd8740a3f00 0x7fd8740a5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44136/0 (socket says 172.21.15.105:44136) 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8737fe640 1 -- 172.21.15.105:0/1231606138 learned_addr learned my addr 172.21.15.105:0/1231606138 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 0x7fd8740a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): get_auth_request con 0x7fd8740a3b00 auth_method 0 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8740a3720 0x7fd8740a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): get_auth_request con 0x7fd8740a3720 auth_method 0 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 1 -- 172.21.15.105:0/1231606138 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262989178 0 0) 0x7fd87406db50 con 0x7fd8740a3f00 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient(hunting): handle_auth_done global_id 23939 payload 274 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 -- 172.21.15.105:0/1231606138 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8740a3720 msgr2=0x7fd8740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8740a3720 0x7fd8740a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 -- 172.21.15.105:0/1231606138 >> v1:172.21.15.105:6789/0 conn(0x7fd8740a3f00 legacy=0x7fd8740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 monclient: found mon.noname-e 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 -- 172.21.15.105:0/1231606138 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8740a7240 con 0x7fd8740a3b00 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd86bfff640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 0x7fd8740a56c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097f0 tx=0x7fd85c01c430 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e3ad7993955f5ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 1 -- 172.21.15.105:0/1231606138 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c0b0070 con 0x7fd8740a3b00 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: dump: 2024-09-17T16:33:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 5 monclient: authenticate success, global_id 23939 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 1 -- 172.21.15.105:0/1231606138 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c0aa810 con 0x7fd8740a3b00 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd879855640 4 set_mon_vals no callback set 2024-09-17T16:33:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 1 -- 172.21.15.105:0/1231606138 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c0a9300 con 0x7fd8740a3b00 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.447+0000 7fd8727fc640 10 monclient: dump: 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: shutdown 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/1231606138 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 msgr2=0x7fd8740a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 0x7fd8740a56c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097f0 tx=0x7fd85c01c430 comp rx=0 tx=0).stop 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/1231606138 shutdown_connections 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8740a3b00 0x7fd8740a56c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- 172.21.15.105:0/1231606138 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8740a3720 0x7fd8740a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/1231606138 >> 172.21.15.105:0/1231606138 conn(0x7fd8740aa390 msgr2=0x7fd8740a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/1231606138 shutdown_connections 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/1231606138 wait complete. 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: monmap: 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.476 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.477 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 librados: starting msgr at 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 librados: starting objecter 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 Processor -- start 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- start start 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 librados: setting wanted keys 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 librados: calling monclient init 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: init 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _add_conns ranks=[0,6,3,5,1,7,2,4,8] 2024-09-17T16:33:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 0x7fd874126e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): picked mon.noname-a con 0x7fd8740a3720 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 0x7fd8741273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): picked mon.noname-d con 0x7fd8740a3b00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8740a3f00 0x7fd874127900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): picked mon.noname-g con 0x7fd8740a3f00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741442e0 con 0x7fd8740a3720 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740a6520 con 0x7fd8740a3f00 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd874144460 con 0x7fd8740a3b00 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): authenticate will time out at 1266.717996s 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 0x7fd874126e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 0x7fd8741273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 0x7fd8741273c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36512/0 (socket says 172.21.15.105:36512) 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 0x7fd874126e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49632/0 (socket says 172.21.15.105:49632) 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 -- 172.21.15.105:0/3997951490 learned_addr learned my addr 172.21.15.105:0/3997951490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): get_auth_request con 0x7fd8740a3b00 auth_method 0 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): get_auth_request con 0x7fd8740a3720 auth_method 0 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd8737fe640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8740a3f00 0x7fd874127900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd8737fe640 10 monclient(hunting): get_auth_request con 0x7fd8740a3f00 auth_method 0 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd8737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd8737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd8737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient(hunting): handle_auth_done global_id 23964 payload 824 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8740a3f00 msgr2=0x7fd874127900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8740a3f00 0x7fd874127900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 msgr2=0x7fd874126e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 0x7fd874126e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 monclient: found mon.noname-d 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87412aca0 con 0x7fd8740a3b00 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8740a3720 0x7fd874126e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd86bfff640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 0x7fd8741273c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae040 tx=0x7fd85c0191d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 client.?.objecter ms_handle_connect 0x7fd8740a3b00 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c0b0040 con 0x7fd8740a3b00 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: dump: 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.483 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c0b9070 con 0x7fd8740a3b00 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 5 monclient: authenticate success, global_id 23964 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _renew_subs 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd874141650 con 0x7fd8740a3b00 2024-09-17T16:33:03.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c0044b0 con 0x7fd8740a3b00 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 client.23964.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 10 monclient: dump: 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.486 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd871ffb640 4 set_mon_vals no callback set 2024-09-17T16:33:03.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _renew_subs 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8740a5c60 con 0x7fd8740a3b00 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 librados: init done 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 0x7fd87409e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fd874051e80 con 0x7fd87409e270 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 0x7fd87409e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient: get_auth_request con 0x7fd87409e270 auth_method 0 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd85c004040 con 0x7fd8740a3b00 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.451+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd872ffd640 10 monclient(hunting): handle_auth_done global_id 23098 payload 824 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd872ffd640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd87412b1a0 con 0x7fd87409e270 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd85403e550 0x7fd854040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(217..217 src has 1..217) ==== 6556+0+0 (secure 0 0 0) 0x7fd85c0e9b20 con 0x7fd8740a3b00 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 client.23964.objecter ms_dispatch 0x7fd874000ba0 osd_map(217..217 src has 1..217) 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 3 client.23964.objecter handle_osd_map got epochs [217,217] > 0 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 3 client.23964.objecter handle_osd_map decoding full epoch 217 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd872ffd640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 0x7fd87409e650 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fd864011cf0 tx=0x7fd86400f610 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 20 client.23964.objecter dump_active .. 0 homeless 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 client.23964.objecter ms_handle_connect 0x7fd87409e270 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 client.23964.objecter resend_mon_ops 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8640148b0 con 0x7fd87409e270 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 monclient: dump: 2024-09-17T16:33:03.490 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.491 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd8737fe640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd85403e550 0x7fd854040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd8737fe640 10 monclient: get_auth_request con 0x7fd85403e550 auth_method 0 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd864014440 con 0x7fd87409e270 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 msgr2=0x7fd87409e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 0x7fd87409e650 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fd864011cf0 tx=0x7fd86400f610 comp rx=0 tx=0).stop 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 msgr2=0x7fd87409e650 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409e270 0x7fd87409e650 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd8737fe640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd85403e550 0x7fd854040a10 secure :-1 s=READY pgs=3441 cs=0 l=1 rev1=1 crypto rx=0x7fd8740a9c50 tx=0x7fd86c005b80 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.455+0000 7fd870ff9640 10 client.23964.objecter ms_handle_connect 0x7fd85403e550 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd879855640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd879855640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 0x7fd8741db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd879855640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fd8740a1e70 con 0x7fd87409eb90 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 0x7fd8741db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient: get_auth_request con 0x7fd87409eb90 auth_method 0 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 10 monclient(hunting): handle_auth_done global_id 23107 payload 824 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 1 -- 172.21.15.105:0/3997951490 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd874129e30 con 0x7fd87409eb90 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 client.23964.objecter ms_handle_connect 0x7fd87409eb90 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 client.23964.objecter resend_mon_ops 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd872ffd640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 0x7fd8741db3c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd86401b000 tx=0x7fd8640046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd86401dca0 con 0x7fd87409eb90 2024-09-17T16:33:03.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 monclient: dump: 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.496 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fd8640273f0 con 0x7fd87409eb90 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 msgr2=0x7fd8741db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 0x7fd8741db3c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd86401b000 tx=0x7fd8640046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 msgr2=0x7fd8741db3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.459+0000 7fd870ff9640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fd87409eb90 0x7fd8741db3c0 secure :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd86401b000 tx=0x7fd8640046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.498 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 331098, 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:03.499 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.500 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.501 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:03.503 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.504 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.506 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.511 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:03.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.516 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.520 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.522 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:03.522 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 10 librados: watch_flush enter 2024-09-17T16:33:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 10 librados: watch_flush exit 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 20 client.23964.objecter shutdown clearing up homeless session... 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 10 client.23964.objecter successfully canceled tick 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd85403e550 msgr2=0x7fd854040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd85403e550 0x7fd854040a10 secure :-1 s=READY pgs=3441 cs=0 l=1 rev1=1 crypto rx=0x7fd8740a9c50 tx=0x7fd86c005b80 comp rx=0 tx=0).stop 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 10 monclient: shutdown 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 msgr2=0x7fd8741273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 --2- 172.21.15.105:0/3997951490 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8740a3b00 0x7fd8741273c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae040 tx=0x7fd85c0191d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd8737fe640 1 -- 172.21.15.105:0/3997951490 reap_dead start 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd8737fe640 1 -- 172.21.15.105:0/3997951490 reap_dead start 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 shutdown_connections 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 >> 172.21.15.105:0/3997951490 conn(0x7fd8740aa390 msgr2=0x7fd8740a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 shutdown_connections 2024-09-17T16:33:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 -- 172.21.15.105:0/3997951490 wait complete. 2024-09-17T16:33:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.463+0000 7fd879855640 1 librados: shutdown 2024-09-17T16:33:03.525 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:03.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:33:03.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:33:03.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: get_monmap_and_config 2024-09-17T16:33:03.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: monmap: 2024-09-17T16:33:03.650 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:03.648090+0000 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:03.648090+0000 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.651 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.652 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:03.652 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 Processor -- start 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 -- start start 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: init 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient: _add_conns ranks=[6,2,0,4,10,7,3,5,9,1,8] 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 0x7f0b280a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-d con 0x7f0b280a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 0x7f0b280a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-b con 0x7f0b280a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0b281396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2806ebc0 con 0x7f0b281396a0 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0b280a3fc0 con 0x7f0b280a4910 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0b280a63c0 con 0x7f0b280a68e0 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2e390640 10 monclient(hunting): authenticate will time out at 1266.913861s 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 0x7f0b280a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2c92d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0b281396a0 0x7f0b28139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44146/0 (socket says 172.21.15.105:44146) 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b2c92d640 1 -- 172.21.15.105:0/822610775 learned_addr learned my addr 172.21.15.105:0/822610775 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:03.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): get_auth_request con 0x7f0b280a68e0 auth_method 0 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 0x7f0b280a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): get_auth_request con 0x7f0b280a4910 auth_method 0 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient(hunting): handle_auth_done global_id 44210 payload 274 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 -- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 msgr2=0x7f0b280a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 0x7f0b280a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 -- 172.21.15.105:0/822610775 >> v1:172.21.15.105:6789/0 conn(0x7f0b281396a0 legacy=0x7f0b28139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b26ffd640 1 -- 172.21.15.105:0/822610775 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838221450 0 0) 0x7f0b2806ebc0 con 0x7f0b281396a0 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 monclient: found mon.noname-b 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b277fe640 1 -- 172.21.15.105:0/822610775 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b280a3690 con 0x7f0b280a4910 2024-09-17T16:33:03.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b27fff640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 0x7f0b280a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.647+0000 7f0b26ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:33:03.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 0x7f0b280a7200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1e78dc38a823bc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 1 -- 172.21.15.105:0/822610775 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b100b0070 con 0x7f0b280a4910 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: dump: 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 5 monclient: authenticate success, global_id 44210 2024-09-17T16:33:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 1 -- 172.21.15.105:0/822610775 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b100aa810 con 0x7f0b280a4910 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 4 set_mon_vals no callback set 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 1 -- 172.21.15.105:0/822610775 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b100a9300 con 0x7f0b280a4910 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b26ffd640 10 monclient: dump: 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: shutdown 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 msgr2=0x7f0b280a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 0x7f0b280a7200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001c3a0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- 172.21.15.105:0/822610775 shutdown_connections 2024-09-17T16:33:03.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a4910 0x7f0b280a7200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- 172.21.15.105:0/822610775 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a68e0 0x7f0b280a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- 172.21.15.105:0/822610775 >> 172.21.15.105:0/822610775 conn(0x7f0b28068ba0 msgr2=0x7f0b28068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- 172.21.15.105:0/822610775 shutdown_connections 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- 172.21.15.105:0/822610775 wait complete. 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: monmap: 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.673 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:03.674 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:03.674 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:03.674 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.674 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:03.674 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.675 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 librados: starting msgr at 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 librados: starting objecter 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 Processor -- start 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- start start 2024-09-17T16:33:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 librados: setting wanted keys 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 librados: calling monclient init 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: init 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient: _add_conns ranks=[5,2,8,6,4,0,1,3,7] 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0b280a4910 0x7f0b281d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-h con 0x7f0b280a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 0x7f0b281d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-b con 0x7f0b280a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0b281396a0 0x7f0b281d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): picked mon.noname-e con 0x7f0b281396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0b281444b0 con 0x7f0b280a68e0 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0b28144330 con 0x7f0b280a4910 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0b28144630 con 0x7f0b281396a0 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2e390640 10 monclient(hunting): authenticate will time out at 1266.919046s 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 0x7f0b281d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 0x7f0b281d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53106/0 (socket says 172.21.15.105:53106) 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0b281396a0 0x7f0b281d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 -- 172.21.15.105:0/3003035413 learned_addr learned my addr 172.21.15.105:0/3003035413 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): get_auth_request con 0x7f0b280a68e0 auth_method 0 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): get_auth_request con 0x7f0b281396a0 auth_method 0 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient(hunting): handle_auth_done global_id 44219 payload 824 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0b280a4910 0x7f0b281d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0b281396a0 msgr2=0x7f0b281d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0b281396a0 0x7f0b281d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0b280a4910 msgr2=0x7f0b281d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0b280a4910 0x7f0b281d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 monclient: found mon.noname-b 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b28122780 con 0x7f0b280a68e0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b27fff640 10 monclient: get_auth_request con 0x7f0b280a4910 auth_method 0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b2c92d640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0b281396a0 0x7f0b281d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0b280a4910 0x7f0b281d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b277fe640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 0x7f0b281d1a80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b1001aa50 tx=0x7f0b10019140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 client.?.objecter ms_handle_connect 0x7f0b280a68e0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b100b0040 con 0x7f0b280a68e0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 monclient: dump: 2024-09-17T16:33:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.651+0000 7f0b24ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b100044b0 con 0x7f0b280a68e0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 5 monclient: authenticate success, global_id 44219 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 monclient: _renew_subs 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b28140b30 con 0x7f0b280a68e0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b100aaca0 con 0x7f0b280a68e0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: dump: 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.684 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b267fc640 4 set_mon_vals no callback set 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 client.44219.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 monclient: _renew_subs 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b280a9de0 con 0x7f0b280a68e0 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 1 librados: init done 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 0x7f0b280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0b281401a0 con 0x7f0b280a0a20 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 0x7f0b280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient: get_auth_request con 0x7f0b280a0a20 auth_method 0 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0b10004040 con 0x7f0b280a68e0 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 10 monclient(hunting): handle_auth_done global_id 23982 payload 824 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b0803e540 0x7f0b08040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b28139ae0 con 0x7f0b280a0a20 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(218..218 src has 1..218) ==== 6577+0+0 (secure 0 0 0) 0x7f0b100b9070 con 0x7f0b280a68e0 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 client.44219.objecter ms_dispatch 0x7f0b28000ba0 osd_map(218..218 src has 1..218) 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 3 client.44219.objecter handle_osd_map got epochs [218,218] > 0 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 3 client.44219.objecter handle_osd_map decoding full epoch 218 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 20 client.44219.objecter dump_active .. 0 homeless 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 client.44219.objecter ms_handle_connect 0x7f0b280a0a20 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 client.44219.objecter resend_mon_ops 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 0x7f0b280a0e00 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b2000f5e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b2000fe60 con 0x7f0b280a0a20 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: dump: 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2c92d640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b0803e540 0x7f0b08040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2c92d640 10 monclient: get_auth_request con 0x7f0b0803e540 auth_method 0 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0b2001b070 con 0x7f0b280a0a20 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 msgr2=0x7f0b280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 0x7f0b280a0e00 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b2000f5e0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 msgr2=0x7f0b280a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a0a20 0x7f0b280a0e00 secure :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b2000f5e0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b24ff9640 10 client.44219.objecter ms_handle_connect 0x7f0b0803e540 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.655+0000 7f0b2c92d640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b0803e540 0x7f0b08040a00 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 crypto rx=0x7f0b18007bc0 tx=0x7f0b18007b10 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.659+0000 7f0b2e390640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.659+0000 7f0b2e390640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 0x7f0b2813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.659+0000 7f0b2e390640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.659+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0b28051e80 con 0x7f0b280a1340 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 0x7f0b2813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient: get_auth_request con 0x7f0b280a1340 auth_method 0 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 10 monclient(hunting): handle_auth_done global_id 23991 payload 824 2024-09-17T16:33:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 1 -- 172.21.15.105:0/3003035413 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0b2813f390 con 0x7f0b280a1340 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 client.44219.objecter ms_handle_connect 0x7f0b280a1340 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 client.44219.objecter resend_mon_ops 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b27fff640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 0x7f0b2813d2a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b20004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b20020e40 con 0x7f0b280a1340 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 monclient: dump: 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f0b200209d0 con 0x7f0b280a1340 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 msgr2=0x7f0b2813d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 0x7f0b2813d2a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b20004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 msgr2=0x7f0b2813d2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b24ff9640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0b280a1340 0x7f0b2813d2a0 secure :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007f30 tx=0x7f0b20004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:03.693 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 331302, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:03.694 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.695 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.696 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:03.697 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:03.698 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:03.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.702 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.704 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.707 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:03.726 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:03.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 10 librados: watch_flush enter 2024-09-17T16:33:03.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 10 librados: watch_flush exit 2024-09-17T16:33:03.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 20 client.44219.objecter shutdown clearing up homeless session... 2024-09-17T16:33:03.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 10 client.44219.objecter successfully canceled tick 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b0803e540 msgr2=0x7f0b08040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b0803e540 0x7f0b08040a00 secure :-1 s=READY pgs=3442 cs=0 l=1 rev1=1 crypto rx=0x7f0b18007bc0 tx=0x7f0b18007b10 comp rx=0 tx=0).stop 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 10 monclient: shutdown 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 msgr2=0x7f0b281d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 --2- 172.21.15.105:0/3003035413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0b280a68e0 0x7f0b281d1a80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b1001aa50 tx=0x7f0b10019140 comp rx=0 tx=0).stop 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2c92d640 1 -- 172.21.15.105:0/3003035413 reap_dead start 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2c92d640 1 -- 172.21.15.105:0/3003035413 reap_dead start 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 shutdown_connections 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 >> 172.21.15.105:0/3003035413 conn(0x7f0b28068ba0 msgr2=0x7f0b280a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 shutdown_connections 2024-09-17T16:33:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 -- 172.21.15.105:0/3003035413 wait complete. 2024-09-17T16:33:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.663+0000 7f0b2e390640 1 librados: shutdown 2024-09-17T16:33:03.733 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:03.733 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:33:03.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:33:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: get_monmap_and_config 2024-09-17T16:33:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: monmap: 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:03.852795+0000 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:03.852795+0000 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:03.857 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 Processor -- start 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 -- start start 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: init 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient: _add_conns ranks=[2,6,5,8,7,3,0,1,9,10,4] 2024-09-17T16:33:03.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c0a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-b con 0x7fb12c0a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-d con 0x7fb12c09e740 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb12c09e0b0 0x7fb12c136500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-h con 0x7fb12c09e0b0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c09ef60 con 0x7fb12c0a4750 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb12c0a31c0 con 0x7fb12c09e0b0 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fb12c0a55b0 con 0x7fb12c09e740 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb131fff640 10 monclient(hunting): authenticate will time out at 1267.118930s 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c0a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36558/0 (socket says 172.21.15.105:36558) 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): get_auth_request con 0x7fb12c0a4750 auth_method 0 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 1 -- 172.21.15.105:0/3708467847 learned_addr learned my addr 172.21.15.105:0/3708467847 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): get_auth_request con 0x7fb12c09e740 auth_method 0 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_done global_id 44228 payload 274 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12bfff640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb12c09e0b0 0x7fb12c136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 -- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 msgr2=0x7fb12c0a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 -- 172.21.15.105:0/3708467847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb12c09e0b0 msgr2=0x7fb12c136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb12c09e0b0 0x7fb12c136500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 monclient: found mon.noname-b 2024-09-17T16:33:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12b7fe640 1 -- 172.21.15.105:0/3708467847 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c0a0a80 con 0x7fb12c0a4750 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.851+0000 7fb12affd640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c0a4b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb124009dc0 tx=0x7fb12401c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ccba7656988a936 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 1 -- 172.21.15.105:0/3708467847 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1240aa880 con 0x7fb12c0a4750 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: dump: 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 1 -- 172.21.15.105:0/3708467847 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1240af070 con 0x7fb12c0a4750 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 1 -- 172.21.15.105:0/3708467847 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1240a9370 con 0x7fb12c0a4750 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 5 monclient: authenticate success, global_id 44228 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 4 set_mon_vals no callback set 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb12a7fc640 10 monclient: dump: 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.867 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 10 monclient: shutdown 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 -- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 msgr2=0x7fb12c0a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c0a4b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb124009dc0 tx=0x7fb12401c590 comp rx=0 tx=0).stop 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 -- 172.21.15.105:0/3708467847 shutdown_connections 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb12c09e0b0 0x7fb12c136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e740 0x7fb12c0a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 --2- 172.21.15.105:0/3708467847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c0a4b30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 -- 172.21.15.105:0/3708467847 >> 172.21.15.105:0/3708467847 conn(0x7fb12c0a5840 msgr2=0x7fb12c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 -- 172.21.15.105:0/3708467847 shutdown_connections 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 1 -- 172.21.15.105:0/3708467847 wait complete. 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 10 monclient: build_initial_monmap 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.855+0000 7fb131fff640 10 monclient: monmap: 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:03.877 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 librados: starting msgr at 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 librados: starting objecter 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 Processor -- start 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- start start 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 librados: setting wanted keys 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 librados: calling monclient init 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: init 2024-09-17T16:33:03.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _add_conns ranks=[6,8,2,0,4,5,3,1,7] 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-d con 0x7fb12c09e0b0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 0x7fb12c1d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-e con 0x7fb12c09e740 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c1d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): picked mon.noname-b con 0x7fb12c0a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c13faf0 con 0x7fb12c0a4750 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fb12c13f970 con 0x7fb12c09e0b0 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb12c13fc70 con 0x7fb12c09e740 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): authenticate will time out at 1267.125099s 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 0x7fb12c1d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c1d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c1d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53128/0 (socket says 172.21.15.105:53128) 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 0x7fb12c1d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36576/0 (socket says 172.21.15.105:36576) 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36566/0 (socket says 172.21.15.105:36566) 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 1 -- 172.21.15.105:0/2395604385 learned_addr learned my addr 172.21.15.105:0/2395604385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): get_auth_request con 0x7fb12c0a4750 auth_method 0 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): get_auth_request con 0x7fb12c09e0b0 auth_method 0 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): get_auth_request con 0x7fb12c09e740 auth_method 0 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient(hunting): handle_auth_done global_id 24009 payload 824 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 msgr2=0x7fb12c1d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 0x7fb12c1d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 msgr2=0x7fb12c1d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb12c0a4750 0x7fb12c1d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 monclient: found mon.noname-d 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c0a1c60 con 0x7fb12c09e0b0 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c09e740 0x7fb12c1d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12b7fe640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d90 tx=0x7fb124019330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 client.?.objecter ms_handle_connect 0x7fb12c09e0b0 2024-09-17T16:33:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1240b5410 con 0x7fb12c09e0b0 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: dump: 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: _finish_auth 0 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1240b55f0 con 0x7fb12c09e0b0 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1240b5c10 con 0x7fb12c09e0b0 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: dump: 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb129ffb640 4 set_mon_vals no callback set 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 5 monclient: authenticate success, global_id 24009 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _renew_subs 2024-09-17T16:33:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c123b20 con 0x7fb12c09e0b0 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 client.24009.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _renew_subs 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c13b880 con 0x7fb12c09e0b0 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 librados: init done 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 0x7fb12c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb12c051e80 con 0x7fb12c135760 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 0x7fb12c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb1240b57a0 con 0x7fb12c09e0b0 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient: get_auth_request con 0x7fb12c135760 auth_method 0 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 10 monclient(hunting): handle_auth_done global_id 23966 payload 824 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb11003e580 0x7fb110040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb12c136500 con 0x7fb12c135760 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(218..218 src has 1..218) ==== 6577+0+0 (secure 0 0 0) 0x7fb1240eb290 con 0x7fb12c09e0b0 2024-09-17T16:33:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 client.24009.objecter ms_dispatch 0x7fb12c000ba0 osd_map(218..218 src has 1..218) 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 3 client.24009.objecter handle_osd_map got epochs [218,218] > 0 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 3 client.24009.objecter handle_osd_map decoding full epoch 218 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 20 client.24009.objecter dump_active .. 0 homeless 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 client.24009.objecter ms_handle_connect 0x7fb12c135760 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12affd640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 0x7fb12c02d450 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb114006cb0 tx=0x7fb11400f610 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 client.24009.objecter resend_mon_ops 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1140148b0 con 0x7fb12c135760 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb123fff640 10 monclient: dump: 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb11003e580 0x7fb110040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.859+0000 7fb12bfff640 10 monclient: get_auth_request con 0x7fb11003e580 auth_method 0 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb114014440 con 0x7fb12c135760 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 msgr2=0x7fb12c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 0x7fb12c02d450 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb114006cb0 tx=0x7fb11400f610 comp rx=0 tx=0).stop 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 msgr2=0x7fb12c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c135760 0x7fb12c02d450 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb123fff640 10 client.24009.objecter ms_handle_connect 0x7fb11003e580 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.863+0000 7fb12bfff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb11003e580 0x7fb110040a40 secure :-1 s=READY pgs=3443 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00aa80 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 0x7fb12c02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb12c0a2000 con 0x7fb12c06c370 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 0x7fb12c02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient: get_auth_request con 0x7fb12c06c370 auth_method 0 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 10 monclient(hunting): handle_auth_done global_id 23975 payload 824 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 1 -- 172.21.15.105:0/2395604385 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb12c13bd40 con 0x7fb12c06c370 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12affd640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 0x7fb12c02d990 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c10 tx=0x7fb1140046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 client.24009.objecter ms_handle_connect 0x7fb12c06c370 2024-09-17T16:33:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 client.24009.objecter resend_mon_ops 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb114022cd0 con 0x7fb12c06c370 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 monclient: dump: 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fb114022860 con 0x7fb12c06c370 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 msgr2=0x7fb12c02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 0x7fb12c02d990 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c10 tx=0x7fb1140046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 msgr2=0x7fb12c02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb123fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb12c06c370 0x7fb12c02d990 secure :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c10 tx=0x7fb1140046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:03.898 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 331510, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:03.899 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 9, 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.900 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.901 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.929 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:03.930 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.931 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:03.932 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:03.933 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 librados: watch_flush enter 2024-09-17T16:33:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 librados: watch_flush exit 2024-09-17T16:33:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 20 client.24009.objecter shutdown clearing up homeless session... 2024-09-17T16:33:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 client.24009.objecter successfully canceled tick 2024-09-17T16:33:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb11003e580 msgr2=0x7fb110040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb11003e580 0x7fb110040a40 secure :-1 s=READY pgs=3443 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00aa80 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 10 monclient: shutdown 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 msgr2=0x7fb12c1d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d90 tx=0x7fb124019330 comp rx=0 tx=0).stop 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb12bfff640 1 -- 172.21.15.105:0/2395604385 reap_dead start 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 shutdown_connections 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 --2- 172.21.15.105:0/2395604385 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb12c09e0b0 0x7fb12c1d0ed0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 >> 172.21.15.105:0/2395604385 conn(0x7fb12c0a5840 msgr2=0x7fb12c0a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 shutdown_connections 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 -- 172.21.15.105:0/2395604385 wait complete. 2024-09-17T16:33:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:03.867+0000 7fb131fff640 1 librados: shutdown 2024-09-17T16:33:03.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:03.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:33:03.937 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: get_monmap_and_config 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: monmap: 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:04.056382+0000 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:04.056382+0000 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:04.061 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 Processor -- start 2024-09-17T16:33:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 -- start start 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: init 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient: _add_conns ranks=[4,2,0,5,9,8,7,1,10,6,3] 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-c con 0x7fe0e80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 0x7fe0e80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-b con 0x7fe0e80a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0e81396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e806ebc0 con 0x7fe0e81396a0 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e80a3fc0 con 0x7fe0e80a4910 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe0e80a63c0 con 0x7fe0e80a68e0 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0ed16b640 10 monclient(hunting): authenticate will time out at 1267.322264s 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe0e81396a0 0x7fe0e8139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44152/0 (socket says 172.21.15.105:44152) 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 0x7fe0e80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:45768/0 (socket says 172.21.15.105:45768) 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e759e640 1 -- 172.21.15.105:0/219199180 learned_addr learned my addr 172.21.15.105:0/219199180 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request con 0x7fe0e80a68e0 auth_method 0 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): get_auth_request con 0x7fe0e80a4910 auth_method 0 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 1 -- 172.21.15.105:0/219199180 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526684928 0 0) 0x7fe0e806ebc0 con 0x7fe0e81396a0 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 10 monclient(hunting): _negotiate 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 10 monclient(hunting): my global_id is 34911 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e5d9b640 1 -- 172.21.15.105:0/219199180 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0d0003530 con 0x7fe0e81396a0 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_done global_id 23116 payload 274 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 -- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 msgr2=0x7fe0e80a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 0x7fe0e80a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 -- 172.21.15.105:0/219199180 >> v1:172.21.15.105:6789/0 conn(0x7fe0e81396a0 legacy=0x7fe0e8139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 monclient: found mon.noname-c 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e6d9d640 1 -- 172.21.15.105:0/219199180 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e80a3690 con 0x7fe0e80a68e0 2024-09-17T16:33:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.055+0000 7fe0e659c640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 0x7fe0e80a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097f0 tx=0x7fe0d801c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2a553f81dc24904d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 1 -- 172.21.15.105:0/219199180 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0d80b0070 con 0x7fe0e80a68e0 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: dump: 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 1 -- 172.21.15.105:0/219199180 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d80aa810 con 0x7fe0e80a68e0 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 1 -- 172.21.15.105:0/219199180 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0d80a9300 con 0x7fe0e80a68e0 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 5 monclient: authenticate success, global_id 23116 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 4 set_mon_vals no callback set 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e5d9b640 10 monclient: dump: 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: shutdown 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 msgr2=0x7fe0e80a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097f0 tx=0x7fe0d801c3a0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- 172.21.15.105:0/219199180 shutdown_connections 2024-09-17T16:33:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0e80a4910 0x7fe0e80a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/219199180 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe0e80a68e0 0x7fe0e80a6cc0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- 172.21.15.105:0/219199180 >> 172.21.15.105:0/219199180 conn(0x7fe0e8068ba0 msgr2=0x7fe0e8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- 172.21.15.105:0/219199180 shutdown_connections 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- 172.21.15.105:0/219199180 wait complete. 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: monmap: 2024-09-17T16:33:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.096 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 librados: starting msgr at 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 librados: starting objecter 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 Processor -- start 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- start start 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 librados: setting wanted keys 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 librados: calling monclient init 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: init 2024-09-17T16:33:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient: _add_conns ranks=[5,1,0,4,3,2,7,6,8] 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0e80a4910 0x7fe0e813d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-h con 0x7fe0e80a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0e80a68e0 0x7fe0e813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-f con 0x7fe0e80a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 0x7fe0e813dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): picked mon.noname-a con 0x7fe0e81396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81444b0 con 0x7fe0e81396a0 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8144330 con 0x7fe0e80a68e0 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe0e8144630 con 0x7fe0e80a4910 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0ed16b640 10 monclient(hunting): authenticate will time out at 1267.327402s 2024-09-17T16:33:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 0x7fe0e813dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 0x7fe0e813dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49644/0 (socket says 172.21.15.105:49644) 2024-09-17T16:33:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 learned_addr learned my addr 172.21.15.105:0/967038896 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): get_auth_request con 0x7fe0e81396a0 auth_method 0 2024-09-17T16:33:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0e80a4910 0x7fe0e813d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e659c640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0e80a68e0 0x7fe0e813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request con 0x7fe0e80a4910 auth_method 0 2024-09-17T16:33:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e659c640 10 monclient(hunting): get_auth_request con 0x7fe0e80a68e0 auth_method 0 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 10 monclient(hunting): handle_auth_done global_id 34920 payload 824 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0e80a4910 msgr2=0x7fe0e813d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0e80a4910 0x7fe0e813d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0e80a68e0 msgr2=0x7fe0e813d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.059+0000 7fe0e759e640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0e80a68e0 0x7fe0e813d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 1 monclient: found mon.noname-a 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e8139480 con 0x7fe0e81396a0 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e759e640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 0x7fe0e813dd30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c6b0 tx=0x7fe0e000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.?.objecter ms_handle_connect 0x7fe0e81396a0 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0e0018070 con 0x7fe0e81396a0 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: dump: 2024-09-17T16:33:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.107 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e000a640 con 0x7fe0e81396a0 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 5 monclient: authenticate success, global_id 34920 2024-09-17T16:33:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 monclient: _renew_subs 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e8122ad0 con 0x7fe0e81396a0 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0e000ace0 con 0x7fe0e81396a0 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: dump: 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.110 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 client.34920.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 monclient: _renew_subs 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81227f0 con 0x7fe0e81396a0 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 1 librados: init done 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e559a640 4 set_mon_vals no callback set 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 0x7fe0c8004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8007cc0 con 0x7fe0c8002890 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe0e000a870 con 0x7fe0e81396a0 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 0x7fe0c8004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient: get_auth_request con 0x7fe0c8002890 auth_method 0 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe0d403e580 0x7fe0d4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(218..218 src has 1..218) ==== 6577+0+0 (secure 0 0 0) 0x7fe0e004f3d0 con 0x7fe0e81396a0 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.34920.objecter ms_dispatch 0x7fe0e8000ba0 osd_map(218..218 src has 1..218) 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 3 client.34920.objecter handle_osd_map got epochs [218,218] > 0 2024-09-17T16:33:04.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 3 client.34920.objecter handle_osd_map decoding full epoch 218 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 20 client.34920.objecter dump_active .. 0 homeless 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_done global_id 25183 payload 824 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e659c640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe0d403e580 0x7fe0d4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0c8007ec0 con 0x7fe0c8002890 2024-09-17T16:33:04.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e659c640 10 monclient: get_auth_request con 0x7fe0d403e580 auth_method 0 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.34920.objecter ms_handle_connect 0x7fe0c8002890 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.34920.objecter resend_mon_ops 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 0x7fe0c8004d30 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8074620 tx=0x7fe0d8019140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0d80b0040 con 0x7fe0c8002890 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: dump: 2024-09-17T16:33:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 client.34920.objecter ms_handle_connect 0x7fe0d403e580 2024-09-17T16:33:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0e659c640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe0d403e580 0x7fe0d4040a40 secure :-1 s=READY pgs=3444 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009580 tx=0x7fe0dc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe0d8003620 con 0x7fe0c8002890 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 msgr2=0x7fe0c8004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 0x7fe0c8004d30 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8074620 tx=0x7fe0d8019140 comp rx=0 tx=0).stop 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 msgr2=0x7fe0c8004d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.063+0000 7fe0cf7fe640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8002890 0x7fe0c8004d30 secure :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8074620 tx=0x7fe0d8019140 comp rx=0 tx=0).stop 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0ed16b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:04.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 0x7fe0c80071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0ed16b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c800a1e0 con 0x7fe0c8006d30 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 0x7fe0c80071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient: get_auth_request con 0x7fe0c8006d30 auth_method 0 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 10 monclient(hunting): handle_auth_done global_id 25192 payload 824 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 1 -- 172.21.15.105:0/967038896 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe0c8007700 con 0x7fe0c8006d30 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 client.34920.objecter ms_handle_connect 0x7fe0c8006d30 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0e6d9d640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 0x7fe0c80071c0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe0d801c3e0 tx=0x7fe0d80a8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 client.34920.objecter resend_mon_ops 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0d80b0040 con 0x7fe0c8006d30 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 monclient: dump: 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.123 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fe0d80c2070 con 0x7fe0c8006d30 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 msgr2=0x7fe0c80071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 0x7fe0c80071c0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe0d801c3e0 tx=0x7fe0d80a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 msgr2=0x7fe0c80071c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.071+0000 7fe0cf7fe640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c8006d30 0x7fe0c80071c0 secure :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe0d801c3e0 tx=0x7fe0d80a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 331671, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 9, 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.129 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.130 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:04.131 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.132 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.133 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.134 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.135 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.136 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.138 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.141 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.143 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:04.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.148 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.151 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.152 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:04.152 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 10 librados: watch_flush enter 2024-09-17T16:33:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 10 librados: watch_flush exit 2024-09-17T16:33:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 20 client.34920.objecter shutdown clearing up homeless session... 2024-09-17T16:33:04.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 10 client.34920.objecter successfully canceled tick 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe0d403e580 msgr2=0x7fe0d4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe0d403e580 0x7fe0d4040a40 secure :-1 s=READY pgs=3444 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009580 tx=0x7fe0dc008040 comp rx=0 tx=0).stop 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 10 monclient: shutdown 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 msgr2=0x7fe0e813dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 --2- 172.21.15.105:0/967038896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0e81396a0 0x7fe0e813dd30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c6b0 tx=0x7fe0e000cb70 comp rx=0 tx=0).stop 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 reap_dead start 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0e759e640 1 -- 172.21.15.105:0/967038896 reap_dead start 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 shutdown_connections 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 >> 172.21.15.105:0/967038896 conn(0x7fe0e8068ba0 msgr2=0x7fe0e8141400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 shutdown_connections 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 -- 172.21.15.105:0/967038896 wait complete. 2024-09-17T16:33:04.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.075+0000 7fe0ed16b640 1 librados: shutdown 2024-09-17T16:33:04.155 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:04.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:33:04.155 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.267+0000 7fe0b41a1640 10 monclient: get_monmap_and_config 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.267+0000 7fe0b41a1640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient: monmap: 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:04.271790+0000 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:04.271790+0000 2024-09-17T16:33:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:04.275 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 Processor -- start 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 -- start start 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient: init 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient: _add_conns ranks=[7,5,1,10,9,0,4,6,2,8,3] 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 0x7fe0ac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-i con 0x7fe0ac0a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0ac0a4910 0x7fe0ac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-h con 0x7fe0ac0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe0ac1396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ac06ebc0 con 0x7fe0ac1396a0 2024-09-17T16:33:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe0ac0a3fc0 con 0x7fe0ac0a4910 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe0ac0a63c0 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b41a1640 10 monclient(hunting): authenticate will time out at 1267.537592s 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 0x7fe0ac0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b273e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe0ac1396a0 0x7fe0ac139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:46134/0 (socket says 172.21.15.105:46134) 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b273e640 1 -- 172.21.15.105:0/3382574991 learned_addr learned my addr 172.21.15.105:0/3382574991 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): get_auth_request con 0x7fe0ac0a68e0 auth_method 0 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0ac0a4910 0x7fe0ac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): get_auth_request con 0x7fe0ac0a4910 auth_method 0 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_done global_id 34540 payload 274 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0ac0a4910 msgr2=0x7fe0ac0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0ac0a4910 0x7fe0ac0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3382574991 >> v1:172.21.15.160:6789/0 conn(0x7fe0ac1396a0 legacy=0x7fe0ac139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 monclient: found mon.noname-i 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b0f3b640 1 -- 172.21.15.105:0/3382574991 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237912658 0 0) 0x7fe0ac06ebc0 con 0x7fe0ac1396a0 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3382574991 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac0a3690 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.271+0000 7fe0b0f3b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b1f3d640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 0x7fe0ac0a6cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009dc0 tx=0x7fe0a801c850 comp rx=0 tx=0).ready entity=mon.7 client_cookie=32acf7fea8e7a258 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 1 -- 172.21.15.105:0/3382574991 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a80b2070 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: dump: 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 1 -- 172.21.15.105:0/3382574991 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a80a59c0 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 5 monclient: authenticate success, global_id 34540 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 1 -- 172.21.15.105:0/3382574991 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a80a5bb0 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 4 set_mon_vals no callback set 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b0f3b640 10 monclient: dump: 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 10 monclient: shutdown 2024-09-17T16:33:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 msgr2=0x7fe0ac0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 0x7fe0ac0a6cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009dc0 tx=0x7fe0a801c850 comp rx=0 tx=0).stop 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3382574991 shutdown_connections 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe0ac0a4910 0x7fe0ac0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3382574991 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe0ac0a68e0 0x7fe0ac0a6cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3382574991 >> 172.21.15.105:0/3382574991 conn(0x7fe0ac068ba0 msgr2=0x7fe0ac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3382574991 shutdown_connections 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3382574991 wait complete. 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 10 monclient: monmap: 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 librados: starting msgr at 2024-09-17T16:33:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.275+0000 7fe0b41a1640 1 librados: starting objecter 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 Processor -- start 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- start start 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 librados: setting wanted keys 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 librados: calling monclient init 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: init 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _add_conns ranks=[8,3,0,6,2,7,4,1,5] 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 0x7fe0ac1d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-e con 0x7fe0ac0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac0a68e0 0x7fe0ac1d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-g con 0x7fe0ac0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0ac1396a0 0x7fe0ac13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): picked mon.noname-a con 0x7fe0ac1396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1444b0 con 0x7fe0ac1396a0 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac144330 con 0x7fe0ac0a68e0 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe0ac144630 con 0x7fe0ac0a4910 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): authenticate will time out at 1267.544559s 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 0x7fe0ac1d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0ac1396a0 0x7fe0ac13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 0x7fe0ac1d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36604/0 (socket says 172.21.15.105:36604) 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3509373475 learned_addr learned my addr 172.21.15.105:0/3509373475 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): get_auth_request con 0x7fe0ac0a4910 auth_method 0 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): get_auth_request con 0x7fe0ac1396a0 auth_method 0 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac0a68e0 0x7fe0ac1d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): get_auth_request con 0x7fe0ac0a68e0 auth_method 0 2024-09-17T16:33:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient(hunting): handle_auth_done global_id 23984 payload 824 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac0a68e0 msgr2=0x7fe0ac1d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac0a68e0 0x7fe0ac1d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0ac1396a0 msgr2=0x7fe0ac13d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0ac1396a0 0x7fe0ac13d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 monclient: found mon.noname-e 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac13fc00 con 0x7fe0ac0a4910 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0ac1396a0 0x7fe0ac13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b1f3d640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 0x7fe0ac1d1800 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80b0040 tx=0x7fe0a80195f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.?.objecter ms_handle_connect 0x7fe0ac0a4910 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a80b2050 con 0x7fe0ac0a4910 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: dump: 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a80be070 con 0x7fe0ac0a4910 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a8003e10 con 0x7fe0ac0a4910 2024-09-17T16:33:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 monclient: dump: 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.300 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 5 monclient: authenticate success, global_id 23984 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _renew_subs 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a3fff640 4 set_mon_vals no callback set 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac1229f0 con 0x7fe0ac0a4910 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 client.23984.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _renew_subs 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:04.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac0a9f00 con 0x7fe0ac0a4910 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 librados: init done 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 0x7fe0ac02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac051e80 con 0x7fe0ac13dfb0 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe0a80039a0 con 0x7fe0ac0a4910 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 0x7fe0ac02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient: get_auth_request con 0x7fe0ac13dfb0 auth_method 0 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe09803e550 0x7fe098040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(218..218 src has 1..218) ==== 6577+0+0 (secure 0 0 0) 0x7fe0a80eafa0 con 0x7fe0ac0a4910 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.23984.objecter ms_dispatch 0x7fe0ac000ba0 osd_map(218..218 src has 1..218) 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 3 client.23984.objecter handle_osd_map got epochs [218,218] > 0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 3 client.23984.objecter handle_osd_map decoding full epoch 218 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 20 client.23984.objecter dump_active .. 0 homeless 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe09803e550 0x7fe098040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 10 monclient: get_auth_request con 0x7fe09803e550 auth_method 0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 10 monclient(hunting): handle_auth_done global_id 19479 payload 824 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0ac0ab950 con 0x7fe0ac13dfb0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.23984.objecter ms_handle_connect 0x7fe0ac13dfb0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.23984.objecter resend_mon_ops 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 0x7fe0ac02d450 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac074500 tx=0x7fe0a4011a60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0a27fc640 10 client.23984.objecter ms_handle_connect 0x7fe09803e550 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.279+0000 7fe0b273e640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe09803e550 0x7fe098040a10 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 crypto rx=0x7fe09c007bc0 tx=0x7fe09c007b10 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a40044a0 con 0x7fe0ac13dfb0 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 10 monclient: dump: 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.304 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe0a4004030 con 0x7fe0ac13dfb0 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 msgr2=0x7fe0ac02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 0x7fe0ac02d450 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac074500 tx=0x7fe0a4011a60 comp rx=0 tx=0).stop 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 msgr2=0x7fe0ac02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.283+0000 7fe0a27fc640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13dfb0 0x7fe0ac02d450 secure :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac074500 tx=0x7fe0a4011a60 comp rx=0 tx=0).stop 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b41a1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 0x7fe0ac02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b41a1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac139ae0 con 0x7fe0ac13e390 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 0x7fe0ac02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient: get_auth_request con 0x7fe0ac13e390 auth_method 0 2024-09-17T16:33:04.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 10 monclient(hunting): handle_auth_done global_id 19488 payload 824 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.287+0000 7fe0b173c640 1 -- 172.21.15.105:0/3509373475 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe0ac0a1120 con 0x7fe0ac13e390 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 client.23984.objecter ms_handle_connect 0x7fe0ac13e390 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 client.23984.objecter resend_mon_ops 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b173c640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 0x7fe0ac02d990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4008010 tx=0x7fe0a40046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe0a400fdb0 con 0x7fe0ac13e390 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 monclient: dump: 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.309 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9289 (secure 0 0 0) 0x7fe0a400fdb0 con 0x7fe0ac13e390 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 msgr2=0x7fe0ac02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 0x7fe0ac02d990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4008010 tx=0x7fe0a40046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 msgr2=0x7fe0ac02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0a27fc640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0ac13e390 0x7fe0ac02d990 secure :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4008010 tx=0x7fe0a40046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 331915, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 9, 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.312 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:04.314 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.315 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.317 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:04.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.322 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:04.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.328 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:04.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:04.332 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 10 librados: watch_flush enter 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 10 librados: watch_flush exit 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 20 client.23984.objecter shutdown clearing up homeless session... 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 10 client.23984.objecter successfully canceled tick 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe09803e550 msgr2=0x7fe098040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe09803e550 0x7fe098040a10 secure :-1 s=READY pgs=3445 cs=0 l=1 rev1=1 crypto rx=0x7fe09c007bc0 tx=0x7fe09c007b10 comp rx=0 tx=0).stop 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 10 monclient: shutdown 2024-09-17T16:33:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 msgr2=0x7fe0ac1d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 --2- 172.21.15.105:0/3509373475 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe0ac0a4910 0x7fe0ac1d1800 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80b0040 tx=0x7fe0a80195f0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b273e640 1 -- 172.21.15.105:0/3509373475 reap_dead start 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b273e640 1 -- 172.21.15.105:0/3509373475 reap_dead start 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 shutdown_connections 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 >> 172.21.15.105:0/3509373475 conn(0x7fe0ac068ba0 msgr2=0x7fe0ac141370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 shutdown_connections 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 -- 172.21.15.105:0/3509373475 wait complete. 2024-09-17T16:33:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.291+0000 7fe0b41a1640 1 librados: shutdown 2024-09-17T16:33:04.335 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:04.336 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:33:04.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: get_monmap_and_config 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: monmap: 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:04.452324+0000 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:04.452324+0000 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:04.457 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 Processor -- start 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 -- start start 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: init 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient: _add_conns ranks=[6,8,5,3,10,4,7,1,9,0,2] 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-d con 0x7f4d4c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-e con 0x7f4d4c0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 0x7f4d4c139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-h con 0x7f4d4c1396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4d4c0a3fc0 con 0x7f4d4c1396a0 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f4d4c0a63c0 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4d4c0a7740 con 0x7f4d4c0a4910 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d52036640 10 monclient(hunting): authenticate will time out at 1267.718535s 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:36610/0 (socket says 172.21.15.105:36610) 2024-09-17T16:33:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 -- 172.21.15.105:0/284087216 learned_addr learned my addr 172.21.15.105:0/284087216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): get_auth_request con 0x7f4d4c0a4910 auth_method 0 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request con 0x7f4d4c0a68e0 auth_method 0 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4bfff640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 0x7f4d4c139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient(hunting): handle_auth_done global_id 23993 payload 274 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 -- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 msgr2=0x7f4d4c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 -- 172.21.15.105:0/284087216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 msgr2=0x7f4d4c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 0x7f4d4c139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 monclient: found mon.noname-e 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4affd640 1 -- 172.21.15.105:0/284087216 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c06ec10 con 0x7f4d4c0a4910 2024-09-17T16:33:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4bfff640 10 monclient: get_auth_request con 0x7f4d4c1396a0 auth_method 0 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.451+0000 7f4d4bfff640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 0x7f4d4c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4d400098d0 tx=0x7f4d4001cea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6d3b27c6d8760cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 1 -- 172.21.15.105:0/284087216 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d400b0070 con 0x7f4d4c0a4910 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: dump: 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.471 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 1 -- 172.21.15.105:0/284087216 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d400aa810 con 0x7f4d4c0a4910 2024-09-17T16:33:04.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 1 -- 172.21.15.105:0/284087216 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d400a9300 con 0x7f4d4c0a4910 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 5 monclient: authenticate success, global_id 23993 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 4 set_mon_vals no callback set 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4a7fc640 10 monclient: dump: 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.473 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: shutdown 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 msgr2=0x7f4d4c0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4d400098d0 tx=0x7f4d4001cea0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- 172.21.15.105:0/284087216 shutdown_connections 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c1396a0 0x7f4d4c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4d4c0a4910 0x7f4d4c0a7200 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- 172.21.15.105:0/284087216 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- 172.21.15.105:0/284087216 >> 172.21.15.105:0/284087216 conn(0x7f4d4c068ba0 msgr2=0x7f4d4c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- 172.21.15.105:0/284087216 shutdown_connections 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- 172.21.15.105:0/284087216 wait complete. 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: monmap: 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.475 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.476 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 librados: starting msgr at 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 librados: starting objecter 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 Processor -- start 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- start start 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 librados: setting wanted keys 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 librados: calling monclient init 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: init 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient: _add_conns ranks=[3,6,2,1,4,8,5,7,0] 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4d4c0a4910 0x7f4d4c1d1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-g con 0x7f4d4c0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c1d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-d con 0x7f4d4c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d4c1396a0 0x7f4d4c13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): picked mon.noname-b con 0x7f4d4c1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c143320 con 0x7f4d4c1396a0 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c06e630 con 0x7f4d4c0a4910 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f4d4c1434a0 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d52036640 10 monclient(hunting): authenticate will time out at 1267.723753s 2024-09-17T16:33:04.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c1d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c1d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36578/0 (socket says 172.21.15.105:36578) 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 1 -- 172.21.15.105:0/1715185501 learned_addr learned my addr 172.21.15.105:0/1715185501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): get_auth_request con 0x7f4d4c0a68e0 auth_method 0 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4bfff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d4c1396a0 0x7f4d4c13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4bfff640 10 monclient(hunting): get_auth_request con 0x7f4d4c1396a0 auth_method 0 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.455+0000 7f4d4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4d4c0a4910 0x7f4d4c1d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 10 monclient(hunting): handle_auth_done global_id 24027 payload 824 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4d4c0a4910 msgr2=0x7f4d4c1d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4d4c0a4910 0x7f4d4c1d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d4c1396a0 msgr2=0x7f4d4c13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d4c1396a0 0x7f4d4c13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 monclient: found mon.noname-d 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c123d10 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4affd640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c1d1db0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4d400098d0 tx=0x7f4d400191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.?.objecter ms_handle_connect 0x7f4d4c0a68e0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d400b9070 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: dump: 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.481 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d400044b0 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 5 monclient: authenticate success, global_id 24027 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 monclient: _renew_subs 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d400b0040 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c13f540 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d49ffb640 4 set_mon_vals no callback set 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: dump: 2024-09-17T16:33:04.483 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.484 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 client.24027.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 monclient: _renew_subs 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c123a30 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 1 librados: init done 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 0x7f4d4c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4d4c051e80 con 0x7f4d4c13db40 2024-09-17T16:33:04.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4d400aacd0 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 0x7f4d4c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4d3803e550 0x7f4d38040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(218..218 src has 1..218) ==== 6577+0+0 (secure 0 0 0) 0x7f4d400e9210 con 0x7f4d4c0a68e0 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient: get_auth_request con 0x7f4d4c13db40 auth_method 0 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.24027.objecter ms_dispatch 0x7f4d4c000ba0 osd_map(218..218 src has 1..218) 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 3 client.24027.objecter handle_osd_map got epochs [218,218] > 0 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 3 client.24027.objecter handle_osd_map decoding full epoch 218 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 20 client.24027.objecter dump_active .. 0 homeless 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4d3803e550 0x7f4d38040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 10 monclient: get_auth_request con 0x7f4d3803e550 auth_method 0 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_done global_id 19652 payload 824 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d4c0aa2a0 con 0x7f4d4c13db40 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.24027.objecter ms_handle_connect 0x7f4d4c13db40 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 0x7f4d4c13df20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009f60 tx=0x7f4d44009f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.24027.objecter resend_mon_ops 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 client.24027.objecter ms_handle_connect 0x7f4d3803e550 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d4bfff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4d3803e550 0x7f4d38040a10 secure :-1 s=READY pgs=3446 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c13fb90 tx=0x7f4d3c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d4401c070 con 0x7f4d4c13db40 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: dump: 2024-09-17T16:33:04.487 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.488 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4d4400fde0 con 0x7f4d4c13db40 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 msgr2=0x7f4d4c13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 0x7f4d4c13df20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009f60 tx=0x7f4d44009f90 comp rx=0 tx=0).stop 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 msgr2=0x7f4d4c13df20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.459+0000 7f4d33fff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13db40 0x7f4d4c13df20 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d52036640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d52036640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 0x7f4d4c0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d52036640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4d4c139ae0 con 0x7f4d4c13e460 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 0x7f4d4c0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient: get_auth_request con 0x7f4d4c13e460 auth_method 0 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 10 monclient(hunting): handle_auth_done global_id 19661 payload 824 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 1 -- 172.21.15.105:0/1715185501 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4d4c139480 con 0x7f4d4c13e460 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d4b7fe640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 0x7f4d4c0a0a20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4d44000f90 tx=0x7f4d44004a50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 10 client.24027.objecter ms_handle_connect 0x7f4d4c13e460 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 10 client.24027.objecter resend_mon_ops 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4d44025070 con 0x7f4d4c13e460 2024-09-17T16:33:04.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.467+0000 7f4d33fff640 10 monclient: dump: 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.493 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f4d44016720 con 0x7f4d4c13e460 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 msgr2=0x7f4d4c0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 0x7f4d4c0a0a20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4d44000f90 tx=0x7f4d44004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 msgr2=0x7f4d4c0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d33fff640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4d4c13e460 0x7f4d4c0a0a20 secure :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4d44000f90 tx=0x7f4d44004a50 comp rx=0 tx=0).stop 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 332067, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:04.495 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 9, 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.496 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.497 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:04.498 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.499 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:04.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.507 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.508 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.510 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.512 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.515 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.516 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:04.517 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:04.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 10 librados: watch_flush enter 2024-09-17T16:33:04.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 10 librados: watch_flush exit 2024-09-17T16:33:04.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 20 client.24027.objecter shutdown clearing up homeless session... 2024-09-17T16:33:04.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 10 client.24027.objecter successfully canceled tick 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4d3803e550 msgr2=0x7f4d38040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4d3803e550 0x7f4d38040a10 secure :-1 s=READY pgs=3446 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c13fb90 tx=0x7f4d3c008040 comp rx=0 tx=0).stop 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 10 monclient: shutdown 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 msgr2=0x7f4d4c1d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 --2- 172.21.15.105:0/1715185501 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4d4c0a68e0 0x7f4d4c1d1db0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4d400098d0 tx=0x7f4d400191e0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d4bfff640 1 -- 172.21.15.105:0/1715185501 reap_dead start 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d4bfff640 1 -- 172.21.15.105:0/1715185501 reap_dead start 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 shutdown_connections 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 >> 172.21.15.105:0/1715185501 conn(0x7f4d4c068ba0 msgr2=0x7f4d4c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 shutdown_connections 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 -- 172.21.15.105:0/1715185501 wait complete. 2024-09-17T16:33:04.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.471+0000 7f4d52036640 1 librados: shutdown 2024-09-17T16:33:04.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:04.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:33:04.521 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: get_monmap_and_config 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: monmap: 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:04.643829+0000 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:04.643829+0000 2024-09-17T16:33:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:04.647 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 Processor -- start 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 -- start start 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: init 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient: _add_conns ranks=[1,5,0,3,6,4,10,7,8,2,9] 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f588c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f588c0a4910 0x7f588c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-h con 0x7f588c0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f588c0a73d0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c06ebc0 con 0x7f588c0a73d0 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c0a3690 con 0x7f588c0a68e0 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f588c0a3fc0 con 0x7f588c0a4910 2024-09-17T16:33:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 10 monclient(hunting): authenticate will time out at 1267.909517s 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f588c0a73d0 0x7f588c141040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44168/0 (socket says 172.21.15.105:44168) 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 -- 172.21.15.105:0/1854356932 learned_addr learned my addr 172.21.15.105:0/1854356932 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f588c0a4910 0x7f588c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): get_auth_request con 0x7f588c0a4910 auth_method 0 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025307152 0 0) 0x7f588c06ebc0 con 0x7f588c0a73d0 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): _negotiate 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): my global_id is 34947 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5878003530 con 0x7f588c0a73d0 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1473349866 0 0) 0x7f5878003530 con 0x7f588c0a73d0 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f588c0a4910 msgr2=0x7f588c13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f588c0a4910 0x7f588c13eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 >> v1:172.21.15.160:6789/0 conn(0x7f588c0a68e0 legacy=0x7f588c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c0a3970 con 0x7f588c0a73d0 2024-09-17T16:33:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f5884001cf0 con 0x7f588c0a73d0 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: dump: 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 >> v1:172.21.15.105:6789/0 conn(0x7f588c0a73d0 legacy=0x7f588c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient: _add_conns ranks=[1,4,8,3,7,0,6,2,5] 2024-09-17T16:33:04.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878004240 0x7f5878004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): picked mon.f con 0x7f5878004240 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5878004da0 0x7f5878005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): picked mon.c con 0x7f5878004da0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 0x7f587800c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): picked mon.e con 0x7f5878009c60 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0a3fc0 con 0x7f5878004240 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f587800c9d0 con 0x7f5878004da0 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f587800cb80 con 0x7f5878009c60 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588959a640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:33:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 0x7f587800c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5878004da0 0x7f5878005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): get_auth_request con 0x7f5878009c60 auth_method 0 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): get_auth_request con 0x7f5878004da0 auth_method 0 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878004240 0x7f5878004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): get_auth_request con 0x7f5878004240 auth_method 0 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient(hunting): handle_auth_done global_id 34947 payload 293 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 -- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5878004da0 msgr2=0x7f5878005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5878004da0 0x7f5878005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 -- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878004240 msgr2=0x7f5878004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878004240 0x7f5878004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 monclient: found mon.e 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 1 -- 172.21.15.105:0/1854356932 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5878003530 con 0x7f5878009c60 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f588ad9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5878004da0 0x7f5878005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 5 monclient: authenticate success, global_id 34947 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.643+0000 7f5890da1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 1 -- 172.21.15.105:0/1854356932 reap_dead start 2024-09-17T16:33:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 0x7f587800c0f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f588400a640 tx=0x7f588401d0a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1e571286d3a6d0cc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f58740096b0 con 0x7f5878009c60 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: dump: 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58840a9070 con 0x7f5878009c60 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 1 -- 172.21.15.105:0/1854356932 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f588400a260 con 0x7f5878009c60 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 4 set_mon_vals no callback set 2024-09-17T16:33:04.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588959a640 10 monclient: dump: 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.670 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: shutdown 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 msgr2=0x7f587800c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 0x7f587800c0f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f588400a640 tx=0x7f588401d0a0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- 172.21.15.105:0/1854356932 shutdown_connections 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 --2- 172.21.15.105:0/1854356932 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f5878009c60 0x7f587800c0f0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- 172.21.15.105:0/1854356932 >> 172.21.15.105:0/1854356932 conn(0x7f588c068ba0 msgr2=0x7f588c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- 172.21.15.105:0/1854356932 shutdown_connections 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- 172.21.15.105:0/1854356932 wait complete. 2024-09-17T16:33:04.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: monmap: 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.672 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 librados: starting msgr at 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 librados: starting objecter 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 Processor -- start 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- start start 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 librados: setting wanted keys 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 librados: calling monclient init 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: init 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient: _add_conns ranks=[1,0,6,2,8,3,4,7,5] 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878009c60 0x7f588c137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-f con 0x7f5878009c60 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 0x7f588c13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-a con 0x7f588c138300 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 0x7f588c13dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): picked mon.noname-d con 0x7f588c138740 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0a7050 con 0x7f588c138300 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0a6ed0 con 0x7f5878009c60 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f588c0a71d0 con 0x7f588c138740 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5890da1640 10 monclient(hunting): authenticate will time out at 1267.915729s 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 0x7f588c13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 0x7f588c13d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49670/0 (socket says 172.21.15.105:49670) 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 1 -- 172.21.15.105:0/1148520189 learned_addr learned my addr 172.21.15.105:0/1148520189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 10 monclient(hunting): get_auth_request con 0x7f588c138300 auth_method 0 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 0x7f588c13dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f5889d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): get_auth_request con 0x7f588c138740 auth_method 0 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.647+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878009c60 0x7f588c137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): get_auth_request con 0x7f5878009c60 auth_method 0 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 10 monclient(hunting): handle_auth_done global_id 24036 payload 824 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878009c60 msgr2=0x7f588c137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878009c60 0x7f588c137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 msgr2=0x7f588c13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 0x7f588c13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 monclient: found mon.noname-d 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c1410b0 con 0x7f588c138740 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5878009c60 0x7f588c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c138300 0x7f588c13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588ad9d640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 0x7f588c13dc10 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a77b0 tx=0x7f5878005840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.?.objecter ms_handle_connect 0x7f588c138740 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f588400a260 con 0x7f588c138740 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: dump: 2024-09-17T16:33:04.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884001cb0 con 0x7f588c138740 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f58840013b0 con 0x7f588c138740 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:04.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: dump: 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.680 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5888d99640 4 set_mon_vals no callback set 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 5 monclient: authenticate success, global_id 24036 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 monclient: _renew_subs 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c141390 con 0x7f588c138740 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 client.24036.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 monclient: _renew_subs 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c0abb10 con 0x7f588c138740 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 1 librados: init done 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 0x7f588c13e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f588c051e80 con 0x7f588c13e150 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5884004b70 con 0x7f588c138740 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 0x7f588c13e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient: get_auth_request con 0x7f588c13e150 auth_method 0 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f586c03e540 0x7f586c040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f58840e8ec0 con 0x7f588c138740 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.24036.objecter ms_dispatch 0x7f588c000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 3 client.24036.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:04.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 3 client.24036.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 20 client.24036.objecter dump_active .. 0 homeless 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f586c03e540 0x7f586c040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 10 monclient: get_auth_request con 0x7f586c03e540 auth_method 0 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 10 monclient(hunting): handle_auth_done global_id 34549 payload 824 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f588c13a410 con 0x7f588c13e150 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.24036.objecter ms_handle_connect 0x7f588c13e150 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.24036.objecter resend_mon_ops 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 0x7f588c13e530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001960 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f587c006e60 con 0x7f588c13e150 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 monclient: dump: 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5882ffd640 10 client.24036.objecter ms_handle_connect 0x7f586c03e540 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.651+0000 7f5889d9b640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f586c03e540 0x7f586c040a00 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 crypto rx=0x7f588c13d5e0 tx=0x7f5874009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f587c0069f0 con 0x7f588c13e150 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 msgr2=0x7f588c13e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 0x7f588c13e530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001960 comp rx=0 tx=0).stop 2024-09-17T16:33:04.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 msgr2=0x7f588c13e530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.655+0000 7f5882ffd640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c13e150 0x7f588c13e530 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001960 comp rx=0 tx=0).stop 2024-09-17T16:33:04.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5890da1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5890da1640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 0x7f588c0a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5890da1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f588c1396a0 con 0x7f588c0a11a0 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 0x7f588c0a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient: get_auth_request con 0x7f588c0a11a0 auth_method 0 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 10 monclient(hunting): handle_auth_done global_id 34558 payload 824 2024-09-17T16:33:04.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 1 -- 172.21.15.105:0/1148520189 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f588c141890 con 0x7f588c0a11a0 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f588a59c640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 0x7f588c0a1580 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001130 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 10 client.24036.objecter ms_handle_connect 0x7f588c0a11a0 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 10 client.24036.objecter resend_mon_ops 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f587c013440 con 0x7f588c0a11a0 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.659+0000 7f5882ffd640 10 monclient: dump: 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.689 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f587c01ea30 con 0x7f588c0a11a0 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 msgr2=0x7f588c0a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 0x7f588c0a1580 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001130 comp rx=0 tx=0).stop 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 msgr2=0x7f588c0a1580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5882ffd640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f588c0a11a0 0x7f588c0a1580 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c001130 comp rx=0 tx=0).stop 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 46, 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 175458, 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:04.691 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 9, 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:04.692 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:04.693 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:04.694 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:04.695 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.696 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:04.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:04.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:04.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.706 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.709 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:04.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:04.711 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:04.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:33:04.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:04.714 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 10 librados: watch_flush enter 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 10 librados: watch_flush exit 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 20 client.24036.objecter shutdown clearing up homeless session... 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 10 client.24036.objecter successfully canceled tick 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f586c03e540 msgr2=0x7f586c040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f586c03e540 0x7f586c040a00 secure :-1 s=READY pgs=3448 cs=0 l=1 rev1=1 crypto rx=0x7f588c13d5e0 tx=0x7f5874009190 comp rx=0 tx=0).stop 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 10 monclient: shutdown 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 msgr2=0x7f588c13dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f588ad9d640 1 -- 172.21.15.105:0/1148520189 reap_dead start 2024-09-17T16:33:04.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 0x7f588c13dc10 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f588c0a77b0 tx=0x7f5878005840 comp rx=0 tx=0).stop 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 shutdown_connections 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 --2- 172.21.15.105:0/1148520189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f588c138740 0x7f588c13dc10 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 >> 172.21.15.105:0/1148520189 conn(0x7f588c068ba0 msgr2=0x7f588c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 shutdown_connections 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 -- 172.21.15.105:0/1148520189 wait complete. 2024-09-17T16:33:04.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.663+0000 7f5890da1640 1 librados: shutdown 2024-09-17T16:33:04.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:33:04.717 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'g', 'e', 'c'] 2024-09-17T16:33:04.718 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-17T16:33:04.718 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-17T16:33:04.718 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f store 2024-09-17T16:33:04.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f sync_force --yes-i-really-mean-it 2024-09-17T16:33:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: get_monmap_and_config 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: monmap: 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:04.837229+0000 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:04.837229+0000 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:04.841 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 Processor -- start 2024-09-17T16:33:04.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 -- start start 2024-09-17T16:33:04.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: init 2024-09-17T16:33:04.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient: _add_conns ranks=[3,4,2,6,0,1,7,8,9,10,5] 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4750 0x7fdbd40a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-g con 0x7fdbd40a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-c con 0x7fdbd40a4b30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-b con 0x7fdbd4136500 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd409de80 con 0x7fdbd4136500 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd409e850 con 0x7fdbd40a4750 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdbd409f570 con 0x7fdbd40a4b30 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdc77f640 10 monclient(hunting): authenticate will time out at 1268.103270s 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:45782/0 (socket says 172.21.15.105:45782) 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53140/0 (socket says 172.21.15.105:53140) 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 -- 172.21.15.105:0/155691411 learned_addr learned my addr 172.21.15.105:0/155691411 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): get_auth_request con 0x7fdbd40a4b30 auth_method 0 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): get_auth_request con 0x7fdbd4136500 auth_method 0 2024-09-17T16:33:04.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4750 0x7fdbd40a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): get_auth_request con 0x7fdbd40a4750 auth_method 0 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient(hunting): handle_auth_done global_id 23134 payload 274 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 -- 172.21.15.105:0/155691411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4750 msgr2=0x7fdbd40a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4750 0x7fdbd40a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 -- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 msgr2=0x7fdbd4136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 monclient: found mon.noname-c 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.835+0000 7fdbd9d1a640 1 -- 172.21.15.105:0/155691411 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd409f850 con 0x7fdbd40a4b30 2024-09-17T16:33:04.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdad1c640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400bb70 tx=0x7fdbc401e7a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=839337235dc72b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 1 -- 172.21.15.105:0/155691411 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbc40b3070 con 0x7fdbd40a4b30 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: dump: 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 5 monclient: authenticate success, global_id 23134 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 1 -- 172.21.15.105:0/155691411 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc40a6c70 con 0x7fdbd40a4b30 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 4 set_mon_vals no callback set 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 1 -- 172.21.15.105:0/155691411 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbc40acc20 con 0x7fdbd40a4b30 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbd9519640 10 monclient: dump: 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 10 monclient: shutdown 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 -- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 msgr2=0x7fdbd40a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400bb70 tx=0x7fdbc401e7a0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 -- 172.21.15.105:0/155691411 shutdown_connections 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdbd4136500 0x7fdbd4136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdbd40a4b30 0x7fdbd40a4f10 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/155691411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4750 0x7fdbd40a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 -- 172.21.15.105:0/155691411 >> 172.21.15.105:0/155691411 conn(0x7fdbd40a93d0 msgr2=0x7fdbd40a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 -- 172.21.15.105:0/155691411 shutdown_connections 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 -- 172.21.15.105:0/155691411 wait complete. 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 10 monclient: build_initial_monmap 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 10 monclient: monmap: 2024-09-17T16:33:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:04.859 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 librados: starting msgr at 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.839+0000 7fdbdc77f640 1 librados: starting objecter 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 Processor -- start 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- start start 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 librados: setting wanted keys 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 librados: calling monclient init 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: init 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _add_conns ranks=[1,3,6,2,5,7,0,8,4] 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd40a4750 0x7fdbd4134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-f con 0x7fdbd40a4750 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4b30 0x7fdbd41353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-g con 0x7fdbd40a4b30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 0x7fdbd413a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): picked mon.noname-d con 0x7fdbd4136500 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd413ff70 con 0x7fdbd40a4750 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd406cf50 con 0x7fdbd40a4b30 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdbd41400f0 con 0x7fdbd4136500 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): authenticate will time out at 1268.108603s 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 0x7fdbd413a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 0x7fdbd413a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36596/0 (socket says 172.21.15.105:36596) 2024-09-17T16:33:04.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 learned_addr learned my addr 172.21.15.105:0/4233133607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): get_auth_request con 0x7fdbd4136500 auth_method 0 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4b30 0x7fdbd41353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd40a4750 0x7fdbd4134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 10 monclient(hunting): get_auth_request con 0x7fdbd40a4b30 auth_method 0 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): get_auth_request con 0x7fdbd40a4750 auth_method 0 2024-09-17T16:33:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient(hunting): handle_auth_done global_id 24045 payload 824 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4b30 msgr2=0x7fdbd41353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdbd40a4b30 0x7fdbd41353e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd40a4750 msgr2=0x7fdbd4134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd40a4750 0x7fdbd4134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient: _finish_hunting 0 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 monclient: found mon.noname-d 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd4120760 con 0x7fdbd4136500 2024-09-17T16:33:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdad1c640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 0x7fdbd413a320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ef90 tx=0x7fdbcc00c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.?.objecter ms_handle_connect 0x7fdbd4136500 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbcc018020 con 0x7fdbd4136500 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: dump: 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc00cdf0 con 0x7fdbd4136500 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 5 monclient: authenticate success, global_id 24045 2024-09-17T16:33:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _renew_subs 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd4120a40 con 0x7fdbd4136500 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbcc014d60 con 0x7fdbd4136500 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: dump: 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.866 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 client.24045.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _renew_subs 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcbfff640 4 set_mon_vals no callback set 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:04.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd413c900 con 0x7fdbd4136500 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 librados: init done 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 0x7fdbd402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4051e80 con 0x7fdbd413b020 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdbcc01b020 con 0x7fdbd4136500 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 0x7fdbd402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient: get_auth_request con 0x7fdbd413b020 auth_method 0 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdbb803e550 0x7fdbb8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7fdbcc050d00 con 0x7fdbd4136500 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.24045.objecter ms_dispatch 0x7fdbd4000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 3 client.24045.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 3 client.24045.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 20 client.24045.objecter dump_active .. 0 homeless 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdbb803e550 0x7fdbb8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 10 monclient: get_auth_request con 0x7fdbb803e550 auth_method 0 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 10 monclient(hunting): handle_auth_done global_id 25246 payload 824 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdbd4136120 con 0x7fdbd413b020 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.24045.objecter ms_handle_connect 0x7fdbd413b020 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.24045.objecter resend_mon_ops 2024-09-17T16:33:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbda51b640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 0x7fdbd402d450 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0011600 tx=0x7fdbd0011ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbd00044a0 con 0x7fdbd413b020 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 monclient: dump: 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.870 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbcaffd640 10 client.24045.objecter ms_handle_connect 0x7fdbb803e550 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.843+0000 7fdbd9d1a640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdbb803e550 0x7fdbb8040a10 secure :-1 s=READY pgs=3449 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b0000 tx=0x7fdbc40a8070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdbd0004030 con 0x7fdbd413b020 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 msgr2=0x7fdbd402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 0x7fdbd402d450 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0011600 tx=0x7fdbd0011ac0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 msgr2=0x7fdbd402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.847+0000 7fdbcaffd640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b020 0x7fdbd402d450 secure :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0011600 tx=0x7fdbd0011ac0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.851+0000 7fdbdc77f640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.851+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 0x7fdbd402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.851+0000 7fdbdc77f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.851+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd413bda0 con 0x7fdbd413b400 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 0x7fdbd402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:04.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient: get_auth_request con 0x7fdbd413b400 auth_method 0 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 10 monclient(hunting): handle_auth_done global_id 25255 payload 824 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 1 -- 172.21.15.105:0/4233133607 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fdbd40a1dd0 con 0x7fdbd413b400 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbda51b640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 0x7fdbd402d990 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0008010 tx=0x7fdbd00046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 client.24045.objecter ms_handle_connect 0x7fdbd413b400 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 client.24045.objecter resend_mon_ops 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdbd002d080 con 0x7fdbd413b400 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:04.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 monclient: dump: 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:04.875 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fdbd000fcf0 con 0x7fdbd413b400 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 msgr2=0x7fdbd402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 0x7fdbd402d990 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0008010 tx=0x7fdbd00046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 msgr2=0x7fdbd402d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbcaffd640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdbd413b400 0x7fdbd402d990 secure :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0008010 tx=0x7fdbd00046d0 comp rx=0 tx=0).stop 2024-09-17T16:33:04.876 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:04.877 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:33:04.877 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:33:04.877 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:04.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 10 librados: watch_flush enter 2024-09-17T16:33:04.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 10 librados: watch_flush exit 2024-09-17T16:33:04.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 20 client.24045.objecter shutdown clearing up homeless session... 2024-09-17T16:33:04.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 10 client.24045.objecter successfully canceled tick 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdbb803e550 msgr2=0x7fdbb8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdbb803e550 0x7fdbb8040a10 secure :-1 s=READY pgs=3449 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b0000 tx=0x7fdbc40a8070 comp rx=0 tx=0).stop 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 10 monclient: shutdown 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 msgr2=0x7fdbd413a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 --2- 172.21.15.105:0/4233133607 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdbd4136500 0x7fdbd413a320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00ef90 tx=0x7fdbcc00c490 comp rx=0 tx=0).stop 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 reap_dead start 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdad1c640 1 -- 172.21.15.105:0/4233133607 reap_dead start 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 shutdown_connections 2024-09-17T16:33:04.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 >> 172.21.15.105:0/4233133607 conn(0x7fdbd40a93d0 msgr2=0x7fdbd40a73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 shutdown_connections 2024-09-17T16:33:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 -- 172.21.15.105:0/4233133607 wait complete. 2024-09-17T16:33:04.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:04.855+0000 7fdbdc77f640 1 librados: shutdown 2024-09-17T16:33:04.880 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-17T16:33:04.880 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-17T16:33:04.880 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:33:04.906 INFO:tasks.ceph.mon.f:Stopped 2024-09-17T16:33:04.907 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-17T16:33:04.907 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g store 2024-09-17T16:33:04.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g sync_force --yes-i-really-mean-it 2024-09-17T16:33:05.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: get_monmap_and_config 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: monmap: 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:05.021461+0000 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:05.021461+0000 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.024 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:05.025 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 Processor -- start 2024-09-17T16:33:05.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 -- start start 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: init 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient: _add_conns ranks=[6,5,1,4,10,2,0,9,7,8,3] 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-d con 0x7f52b40a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b40a5070 0x7f52b4137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-h con 0x7f52b40a5070 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f52b4137a50 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b406c590 con 0x7f52b4137a50 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f52b406cf10 con 0x7f52b40a5070 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f52b40a62d0 con 0x7f52b40a4750 2024-09-17T16:33:05.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52bbf7d640 10 monclient(hunting): authenticate will time out at 1268.287601s 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52ba51a640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f52b4137a50 legacy=0x7f52b413e970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36604/0 (socket says 172.21.15.105:36604) 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 1 -- 172.21.15.105:0/2942451482 learned_addr learned my addr 172.21.15.105:0/2942451482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): get_auth_request con 0x7f52b40a4750 auth_method 0 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.019+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9518640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b40a5070 0x7f52b4137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9518640 10 monclient(hunting): get_auth_request con 0x7f52b40a5070 auth_method 0 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9518640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9518640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9518640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 10 monclient(hunting): handle_auth_done global_id 24054 payload 274 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 -- 172.21.15.105:0/2942451482 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b40a5070 msgr2=0x7f52b4137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b40a5070 0x7f52b4137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 -- 172.21.15.105:0/2942451482 >> v1:172.21.15.160:6789/0 conn(0x7f52b4137a50 legacy=0x7f52b413e970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 monclient: found mon.noname-d 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:05.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 -- 172.21.15.105:0/2942451482 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b40a09d0 con 0x7f52b40a4750 2024-09-17T16:33:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b9d19640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098a0 tx=0x7f52b001c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7180cb0d3724156d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 1 -- 172.21.15.105:0/2942451482 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52b00b0070 con 0x7f52b40a4750 2024-09-17T16:33:05.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: dump: 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.037 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 1 -- 172.21.15.105:0/2942451482 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b00aa810 con 0x7f52b40a4750 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 1 -- 172.21.15.105:0/2942451482 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52b00a9300 con 0x7f52b40a4750 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 5 monclient: authenticate success, global_id 24054 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:05.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 4 set_mon_vals no callback set 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52b8d17640 10 monclient: dump: 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.039 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 10 monclient: shutdown 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 -- 172.21.15.105:0/2942451482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 msgr2=0x7f52b40a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098a0 tx=0x7f52b001c440 comp rx=0 tx=0).stop 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 -- 172.21.15.105:0/2942451482 shutdown_connections 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b40a5070 0x7f52b4137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/2942451482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a4750 0x7f52b40a4b30 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 -- 172.21.15.105:0/2942451482 >> 172.21.15.105:0/2942451482 conn(0x7f52b40a94d0 msgr2=0x7f52b40a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 -- 172.21.15.105:0/2942451482 shutdown_connections 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 1 -- 172.21.15.105:0/2942451482 wait complete. 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.023+0000 7f52bbf7d640 10 monclient: monmap: 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.041 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 librados: starting msgr at 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 librados: starting objecter 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 Processor -- start 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- start start 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 librados: setting wanted keys 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 librados: calling monclient init 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: init 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _add_conns ranks=[7,6,5,1,8,4,0,3,2] 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f52b40a4750 0x7f52b41d1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-i con 0x7f52b40a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 0x7f52b41d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-d con 0x7f52b40a5070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b4137a50 0x7f52b409da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): picked mon.noname-h con 0x7f52b4137a50 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f52b40a0550 con 0x7f52b4137a50 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f52b40a03d0 con 0x7f52b40a5070 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f52b406c810 con 0x7f52b40a4750 2024-09-17T16:33:05.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): authenticate will time out at 1268.293234s 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 0x7f52b41d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 0x7f52b41d1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36610/0 (socket says 172.21.15.105:36610) 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 -- 172.21.15.105:0/1692656213 learned_addr learned my addr 172.21.15.105:0/1692656213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): get_auth_request con 0x7f52b40a5070 auth_method 0 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f52b40a4750 0x7f52b41d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52ba51a640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b4137a50 0x7f52b409da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient(hunting): handle_auth_done global_id 24063 payload 824 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f52b40a4750 msgr2=0x7f52b41d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f52b40a4750 0x7f52b41d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b4137a50 msgr2=0x7f52b409da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f52b4137a50 0x7f52b409da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 monclient: found mon.noname-d 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b409f310 con 0x7f52b40a5070 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient: get_auth_request con 0x7f52b40a4750 auth_method 0 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f52b40a4750 0x7f52b41d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9518640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 0x7f52b41d1e60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f52a400d800 tx=0x7f52a400dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 client.?.objecter ms_handle_connect 0x7f52b40a5070 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52a400bb30 con 0x7f52b40a5070 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: dump: 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.046 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a400bcf0 con 0x7f52b40a5070 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52a40148a0 con 0x7f52b40a5070 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 monclient: dump: 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52abfff640 4 set_mon_vals no callback set 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 5 monclient: authenticate success, global_id 24063 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _renew_subs 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b413f090 con 0x7f52b40a5070 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 client.24063.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _renew_subs 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b40735f0 con 0x7f52b40a5070 2024-09-17T16:33:05.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 librados: init done 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 0x7f52b409e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f52b4051e80 con 0x7f52b409df80 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f52a401b030 con 0x7f52b40a5070 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 0x7f52b409e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient: get_auth_request con 0x7f52b409df80 auth_method 0 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f529803e580 0x7f5298040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f52a4019020 con 0x7f52b40a5070 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 10 client.24063.objecter ms_dispatch 0x7f52b4000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 3 client.24063.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 3 client.24063.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52aa7fc640 20 client.24063.objecter dump_active .. 0 homeless 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52ba51a640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f529803e580 0x7f5298040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 10 monclient(hunting): handle_auth_done global_id 19515 payload 824 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52b9d19640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52b40a2390 con 0x7f52b409df80 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.027+0000 7f52ba51a640 10 monclient: get_auth_request con 0x7f529803e580 auth_method 0 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 client.24063.objecter ms_handle_connect 0x7f52b409df80 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 client.24063.objecter resend_mon_ops 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 0x7f52b409e360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f52b409ffb0 tx=0x7f52b00191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52b00b0040 con 0x7f52b409df80 2024-09-17T16:33:05.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 monclient: dump: 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 client.24063.objecter ms_handle_connect 0x7f529803e580 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52ba51a640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f529803e580 0x7f5298040a40 secure :-1 s=READY pgs=3450 cs=0 l=1 rev1=1 crypto rx=0x7f52b4074580 tx=0x7f52ac008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f52b0003620 con 0x7f52b409df80 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 msgr2=0x7f52b409e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 0x7f52b409e360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f52b409ffb0 tx=0x7f52b00191e0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 msgr2=0x7f52b409e360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.031+0000 7f52aa7fc640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409df80 0x7f52b409e360 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 0x7f52b4138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f52b4136de0 con 0x7f52b409e8a0 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 0x7f52b4138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient: get_auth_request con 0x7f52b409e8a0 auth_method 0 2024-09-17T16:33:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 10 monclient(hunting): handle_auth_done global_id 19524 payload 824 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 1 -- 172.21.15.105:0/1692656213 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f52b40a4b30 con 0x7f52b409e8a0 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 client.24063.objecter ms_handle_connect 0x7f52b409e8a0 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52b9d19640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 0x7f52b4138730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f52b001c270 tx=0x7f52b00a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 client.24063.objecter resend_mon_ops 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f52b00b0040 con 0x7f52b409e8a0 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 monclient: dump: 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.057 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f52b00bd8c0 con 0x7f52b409e8a0 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 msgr2=0x7f52b4138730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 0x7f52b4138730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f52b001c270 tx=0x7f52b00a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 msgr2=0x7f52b4138730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52aa7fc640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52b409e8a0 0x7f52b4138730 secure :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f52b001c270 tx=0x7f52b00a8000 comp rx=0 tx=0).stop 2024-09-17T16:33:05.058 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:05.059 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:33:05.059 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:33:05.059 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 librados: watch_flush enter 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 librados: watch_flush exit 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 20 client.24063.objecter shutdown clearing up homeless session... 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 client.24063.objecter successfully canceled tick 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f529803e580 msgr2=0x7f5298040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f529803e580 0x7f5298040a40 secure :-1 s=READY pgs=3450 cs=0 l=1 rev1=1 crypto rx=0x7f52b4074580 tx=0x7f52ac008040 comp rx=0 tx=0).stop 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 10 monclient: shutdown 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 msgr2=0x7f52b41d1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 --2- 172.21.15.105:0/1692656213 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f52b40a5070 0x7f52b41d1e60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f52a400d800 tx=0x7f52a400dcc0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52ba51a640 1 -- 172.21.15.105:0/1692656213 reap_dead start 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52ba51a640 1 -- 172.21.15.105:0/1692656213 reap_dead start 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 shutdown_connections 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 >> 172.21.15.105:0/1692656213 conn(0x7f52b40a94d0 msgr2=0x7f52b40724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 shutdown_connections 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 -- 172.21.15.105:0/1692656213 wait complete. 2024-09-17T16:33:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.039+0000 7f52bbf7d640 1 librados: shutdown 2024-09-17T16:33:05.062 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-17T16:33:05.062 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-17T16:33:05.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:33:05.092 INFO:tasks.ceph.mon.g:Stopped 2024-09-17T16:33:05.092 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-17T16:33:05.093 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-17T16:33:05.093 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-17T16:33:05.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:33:05.106 INFO:tasks.ceph.mon.e:Stopped 2024-09-17T16:33:05.106 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-17T16:33:05.106 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c store 2024-09-17T16:33:05.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c sync_force --yes-i-really-mean-it 2024-09-17T16:33:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: get_monmap_and_config 2024-09-17T16:33:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: monmap: 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:05.270136+0000 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:05.270136+0000 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:05.274 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 Processor -- start 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 -- start start 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: init 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient: _add_conns ranks=[1,2,3,7,6,10,8,0,5,4,9] 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f30500a53c0 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): picked mon.noname-b con 0x7f30500a4810 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:05.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 0x7f3050140e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): picked mon.noname-g con 0x7f30500a4bf0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30500a8170 con 0x7f30500a53c0 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f30500a5eb0 con 0x7f30500a4810 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f30500a3590 con 0x7f30500a4bf0 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f3054977640 10 monclient(hunting): authenticate will time out at 1268.535865s 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.267+0000 7f304e59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304ed9d640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f30500a53c0 legacy=0x7f30500a57a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304f59e640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 msgr2=0x7f3050140e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53148/0 (socket says 172.21.15.105:53148) 2024-09-17T16:33:05.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 -- 172.21.15.105:0/3146700760 learned_addr learned my addr 172.21.15.105:0/3146700760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): get_auth_request con 0x7f30500a4810 auth_method 0 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304f59e640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 0x7f3050140e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient(hunting): handle_auth_done global_id 44282 payload 274 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 -- 172.21.15.105:0/3146700760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 msgr2=0x7f3050140e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 0x7f3050140e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 -- 172.21.15.105:0/3146700760 >> v1:172.21.15.160:6789/0 conn(0x7f30500a53c0 legacy=0x7f30500a57a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 monclient: found mon.noname-b 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 -- 172.21.15.105:0/3146700760 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30500a3870 con 0x7f30500a4810 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304e59c640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30400098a0 tx=0x7f304001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0258275bc9d2c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 1 -- 172.21.15.105:0/3146700760 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30400b0070 con 0x7f30500a4810 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:05.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: dump: 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 1 -- 172.21.15.105:0/3146700760 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30400aa810 con 0x7f30500a4810 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 5 monclient: authenticate success, global_id 44282 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 1 -- 172.21.15.105:0/3146700760 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30400a9300 con 0x7f30500a4810 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 4 set_mon_vals no callback set 2024-09-17T16:33:05.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f304dd9b640 10 monclient: dump: 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.284 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 10 monclient: shutdown 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 -- 172.21.15.105:0/3146700760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 msgr2=0x7f305013e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30400098a0 tx=0x7f304001c430 comp rx=0 tx=0).stop 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 -- 172.21.15.105:0/3146700760 shutdown_connections 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30500a4bf0 0x7f3050140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 --2- 172.21.15.105:0/3146700760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30500a4810 0x7f305013e8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 -- 172.21.15.105:0/3146700760 >> 172.21.15.105:0/3146700760 conn(0x7f3050068ba0 msgr2=0x7f3050068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 -- 172.21.15.105:0/3146700760 shutdown_connections 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.271+0000 7f3054977640 1 -- 172.21.15.105:0/3146700760 wait complete. 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: monmap: 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.286 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 librados: starting msgr at 2024-09-17T16:33:05.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 librados: starting objecter 2024-09-17T16:33:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 Processor -- start 2024-09-17T16:33:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- start start 2024-09-17T16:33:05.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 librados: setting wanted keys 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 librados: calling monclient init 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: init 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _add_conns ranks=[6,4,7,5,1,8,2,0,3] 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 0x7f3050138d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): picked mon.noname-d con 0x7f30500a4810 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30500a4bf0 0x7f30501392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): picked mon.noname-c con 0x7f30500a4bf0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f30500a53c0 0x7f305013e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): picked mon.noname-i con 0x7f30500a53c0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3050143d80 con 0x7f30500a4bf0 2024-09-17T16:33:05.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3050076050 con 0x7f30500a4810 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3050143f00 con 0x7f30500a53c0 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): authenticate will time out at 1268.541513s 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 0x7f3050138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 0x7f3050138d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:36614/0 (socket says 172.21.15.105:36614) 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 -- 172.21.15.105:0/2726599020 learned_addr learned my addr 172.21.15.105:0/2726599020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): get_auth_request con 0x7f30500a4810 auth_method 0 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30500a4bf0 0x7f30501392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): get_auth_request con 0x7f30500a4bf0 auth_method 0 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f30500a53c0 0x7f305013e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): get_auth_request con 0x7f30500a53c0 auth_method 0 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient(hunting): handle_auth_done global_id 23143 payload 824 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f30500a53c0 msgr2=0x7f305013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f30500a53c0 0x7f305013e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 msgr2=0x7f3050138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 0x7f3050138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 monclient: found mon.noname-c 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:33:05.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3050123ae0 con 0x7f30500a4bf0 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30500a4810 0x7f3050138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304e59c640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30500a4bf0 0x7f30501392b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f30500a8340 tx=0x7f30400191d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 client.?.objecter ms_handle_connect 0x7f30500a4bf0 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30400b0040 con 0x7f30500a4bf0 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: dump: 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.298 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 5 monclient: authenticate success, global_id 23143 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _renew_subs 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30400044a0 con 0x7f30500a4bf0 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3050123dc0 con 0x7f30500a4bf0 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3040003a90 con 0x7f30500a4bf0 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304d59a640 4 set_mon_vals no callback set 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 client.23143.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 monclient: dump: 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _renew_subs 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3050123800 con 0x7f30500a4bf0 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 librados: init done 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 0x7f305013e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f305006a200 con 0x7f3050140470 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 0x7f305013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient: get_auth_request con 0x7f3050140470 auth_method 0 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 10 monclient(hunting): handle_auth_done global_id 23152 payload 824 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30500a57a0 con 0x7f3050140470 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f304ed9d640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 0x7f305013e860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f30380057b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 client.23143.objecter ms_handle_connect 0x7f3050140470 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.275+0000 7f30477fe640 10 client.23143.objecter resend_mon_ops 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3038016070 con 0x7f3050140470 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 monclient: dump: 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.303 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f30400bb060 con 0x7f30500a4bf0 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f303003e7f0 0x7f3030040cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f30400e9080 con 0x7f30500a4bf0 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 client.23143.objecter ms_dispatch 0x7f3050000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 3 client.23143.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 3 client.23143.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 20 client.23143.objecter dump_active .. 0 homeless 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f303800eaf0 con 0x7f3050140470 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 msgr2=0x7f305013e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 0x7f305013e860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f30380057b0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 msgr2=0x7f305013e860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3050140470 0x7f305013e860 secure :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f30380057b0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f304f59e640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f303003e7f0 0x7f3030040cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f304f59e640 10 monclient: get_auth_request con 0x7f303003e7f0 auth_method 0 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f30477fe640 10 client.23143.objecter ms_handle_connect 0x7f303003e7f0 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.279+0000 7f304f59e640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f303003e7f0 0x7f3030040cb0 secure :-1 s=READY pgs=3451 cs=0 l=1 rev1=1 crypto rx=0x7f305013fe60 tx=0x7f30480095f0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f3054977640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f3054977640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 0x7f305013f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f3054977640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3050064860 con 0x7f305013eda0 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 0x7f305013f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient: get_auth_request con 0x7f305013eda0 auth_method 0 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.283+0000 7f304ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f304ed9d640 10 monclient(hunting): handle_auth_done global_id 23161 payload 824 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f304ed9d640 1 -- 172.21.15.105:0/2726599020 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f3050122920 con 0x7f305013eda0 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f304ed9d640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 0x7f305013f180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f3038005990 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 client.23143.objecter ms_handle_connect 0x7f305013eda0 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 client.23143.objecter resend_mon_ops 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f303802b080 con 0x7f305013eda0 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 monclient: dump: 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.308 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f3038012600 con 0x7f305013eda0 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 msgr2=0x7f305013f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 0x7f305013f180 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f3038005990 comp rx=0 tx=0).stop 2024-09-17T16:33:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 msgr2=0x7f305013f180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f30477fe640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f305013eda0 0x7f305013f180 secure :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f30380052f0 tx=0x7f3038005990 comp rx=0 tx=0).stop 2024-09-17T16:33:05.310 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:05.310 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:33:05.310 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:33:05.310 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 10 librados: watch_flush enter 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 10 librados: watch_flush exit 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 20 client.23143.objecter shutdown clearing up homeless session... 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 10 client.23143.objecter successfully canceled tick 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f303003e7f0 msgr2=0x7f3030040cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f303003e7f0 0x7f3030040cb0 secure :-1 s=READY pgs=3451 cs=0 l=1 rev1=1 crypto rx=0x7f305013fe60 tx=0x7f30480095f0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 10 monclient: shutdown 2024-09-17T16:33:05.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30500a4bf0 msgr2=0x7f30501392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 --2- 172.21.15.105:0/2726599020 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30500a4bf0 0x7f30501392b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f30500a8340 tx=0x7f30400191d0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f304f59e640 1 -- 172.21.15.105:0/2726599020 reap_dead start 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f304f59e640 1 -- 172.21.15.105:0/2726599020 reap_dead start 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 shutdown_connections 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 >> 172.21.15.105:0/2726599020 conn(0x7f3050068ba0 msgr2=0x7f30500a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 shutdown_connections 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 -- 172.21.15.105:0/2726599020 wait complete. 2024-09-17T16:33:05.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.287+0000 7f3054977640 1 librados: shutdown 2024-09-17T16:33:05.313 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-17T16:33:05.314 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T16:33:05.314 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:33:05.327 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T16:33:05.327 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-17T16:33:05.328 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:33:05.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: get_monmap_and_config 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: monmap: 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:05.477487+0000 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:05.477487+0000 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:05.480 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:05.481 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 Processor -- start 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 -- start start 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: init 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient: _add_conns ranks=[2,1,7,9,10,5,6,4,8,3,0] 2024-09-17T16:33:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-b con 0x7f04300a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f04300a4910 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 0x7f0430139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-i con 0x7f04300a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043006ebc0 con 0x7f04300a4910 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f04300a3fc0 con 0x7f04300a68e0 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f04300a63c0 con 0x7f04300a7200 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0437c78640 10 monclient(hunting): authenticate will time out at 1268.743280s 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435213640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f04300a4910 legacy=0x7f04301396a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:05.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:53158/0 (socket says 172.21.15.105:53158) 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 1 -- 172.21.15.105:0/648856174 learned_addr learned my addr 172.21.15.105:0/648856174 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): get_auth_request con 0x7f04300a68e0 auth_method 0 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0436215640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 0x7f0430139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0436215640 10 monclient(hunting): get_auth_request con 0x7f04300a7200 auth_method 0 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0436215640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0436215640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0436215640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 10 monclient(hunting): handle_auth_done global_id 44291 payload 274 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.475+0000 7f0435a14640 1 -- 172.21.15.105:0/648856174 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 msgr2=0x7f0430139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 0x7f0430139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 1 -- 172.21.15.105:0/648856174 >> v1:172.21.15.160:6789/0 conn(0x7f04300a4910 legacy=0x7f04301396a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 1 monclient: found mon.noname-b 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 1 -- 172.21.15.105:0/648856174 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300a3690 con 0x7f04300a68e0 2024-09-17T16:33:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0436215640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 0x7f0430139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.479+0000 7f0435a14640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098a0 tx=0x7f042c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3255933e57429fb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 1 -- 172.21.15.105:0/648856174 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f042c0b0070 con 0x7f04300a68e0 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: dump: 2024-09-17T16:33:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.493 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 1 -- 172.21.15.105:0/648856174 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c0aa810 con 0x7f04300a68e0 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 5 monclient: authenticate success, global_id 44291 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 1 -- 172.21.15.105:0/648856174 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f042c0a9300 con 0x7f04300a68e0 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 4 set_mon_vals no callback set 2024-09-17T16:33:05.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0434a12640 10 monclient: dump: 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.495 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: shutdown 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/648856174 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 msgr2=0x7f04300a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098a0 tx=0x7f042c01c410 comp rx=0 tx=0).stop 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/648856174 shutdown_connections 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04300a7200 0x7f0430139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- 172.21.15.105:0/648856174 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04300a68e0 0x7f04300a6cc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/648856174 >> 172.21.15.105:0/648856174 conn(0x7f0430068ba0 msgr2=0x7f0430068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/648856174 shutdown_connections 2024-09-17T16:33:05.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/648856174 wait complete. 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: build_initial_monmap 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: monmap: 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:05.497 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 librados: starting msgr at 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 librados: starting objecter 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 Processor -- start 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- start start 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 librados: setting wanted keys 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 librados: calling monclient init 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: init 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _add_conns ranks=[4,0,1,2,6,7,8,3,5] 2024-09-17T16:33:05.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 0x7f04301d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-c con 0x7f04300a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-a con 0x7f04300a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 0x7f043013e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): picked mon.noname-f con 0x7f04300a7200 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f043013a2f0 con 0x7f04300a68e0 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f04300a7610 con 0x7f04300a7200 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f043013a470 con 0x7f04300a4910 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient(hunting): authenticate will time out at 1268.750540s 2024-09-17T16:33:05.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435a14640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 msgr2=0x7f04301d1380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435a14640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 0x7f04301d1380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49686/0 (socket says 172.21.15.105:49686) 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 -- 172.21.15.105:0/2366200559 learned_addr learned my addr 172.21.15.105:0/2366200559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): get_auth_request con 0x7f04300a68e0 auth_method 0 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0436215640 1 -- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 msgr2=0x7f043013e230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0436215640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 0x7f043013e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:05.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient(hunting): handle_auth_done global_id 34965 payload 824 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 -- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 msgr2=0x7f04301d1380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 0x7f04301d1380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 -- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 msgr2=0x7f043013e230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 0x7f043013e230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient: _finish_hunting 0 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 monclient: found mon.noname-a 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 -- 172.21.15.105:0/2366200559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0430123330 con 0x7f04300a68e0 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0435213640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f042000b550 tx=0x7f042000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 client.?.objecter ms_handle_connect 0x7f04300a68e0 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04200092c0 con 0x7f04300a68e0 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: dump: 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420015070 con 0x7f04300a68e0 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 5 monclient: authenticate success, global_id 34965 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _renew_subs 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0430123610 con 0x7f04300a68e0 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04200054e0 con 0x7f04300a68e0 2024-09-17T16:33:05.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f04267fc640 10 monclient: dump: 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 client.34965.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _renew_subs 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04300a2060 con 0x7f04300a68e0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0427fff640 4 set_mon_vals no callback set 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.483+0000 7f0437c78640 1 librados: init done 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0437c78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0437c78640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0437c78640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043006ddb0 con 0x7f04300a68e0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0420005070 con 0x7f04300a68e0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 0x7f0414040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f042000d040 con 0x7f04300a68e0 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 10 client.34965.objecter ms_dispatch 0x7f0430000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 3 client.34965.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 3 client.34965.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 20 client.34965.objecter dump_active .. 0 homeless 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0435a14640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 0x7f0414040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0435a14640 10 monclient: get_auth_request con 0x7f041403e550 auth_method 0 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f04267fc640 10 client.34965.objecter ms_handle_connect 0x7f041403e550 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.487+0000 7f0435a14640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 0x7f0414040a10 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f042c0ad000 tx=0x7f042c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.491+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f042000f380 con 0x7f04300a68e0 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.491+0000 7f04267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.491+0000 7f04267fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f0437c78640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:33:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f0437c78640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f0437c78640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:05.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f04300a6cc0 con 0x7f04300a68e0 2024-09-17T16:33:05.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f04267fc640 1 -- 172.21.15.105:0/2366200559 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7f0420010e10 con 0x7f04300a68e0 2024-09-17T16:33:05.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f04267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:05.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.719+0000 7f04267fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:05.725 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":46,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":10,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:33:05.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 10 librados: watch_flush enter 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 10 librados: watch_flush exit 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 20 client.34965.objecter shutdown clearing up homeless session... 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 10 client.34965.objecter successfully canceled tick 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 msgr2=0x7f0414040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 0x7f0414040a10 secure :-1 s=READY pgs=3452 cs=0 l=1 rev1=1 crypto rx=0x7f042c0ad000 tx=0x7f042c008040 comp rx=0 tx=0).stop 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 10 monclient: shutdown 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 msgr2=0x7f04301d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f042000b550 tx=0x7f042000ba10 comp rx=0 tx=0).stop 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 shutdown_connections 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f041403e550 0x7f0414040a10 unknown :-1 s=CLOSED pgs=3452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04300a7200 0x7f043013e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04300a68e0 0x7f04301d18c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 --2- 172.21.15.105:0/2366200559 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f04300a4910 0x7f04301d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 >> 172.21.15.105:0/2366200559 conn(0x7f0430068ba0 msgr2=0x7f043009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 shutdown_connections 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 -- 172.21.15.105:0/2366200559 wait complete. 2024-09-17T16:33:05.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:05.723+0000 7f0437c78640 1 librados: shutdown 2024-09-17T16:33:08.743 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: get_monmap_and_config 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: build_initial_monmap 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: monmap: 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:08.869528+0000 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:08.869528+0000 2024-09-17T16:33:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:08.873 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:08.874 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:08.874 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 Processor -- start 2024-09-17T16:33:08.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 -- start start 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: init 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient: _add_conns ranks=[1,5,2,4,7,6,8,0,10,9,3] 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5c1c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-h con 0x7f5c1c0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-b con 0x7f5c1c0a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1c06ebc0 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1c0a3fc0 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5c1c0a63c0 con 0x7f5c1c0a4910 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c20f32640 10 monclient(hunting): authenticate will time out at 1272.135267s 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1a59c640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f5c1c0a68e0 legacy=0x7f5c1c0a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56510/0 (socket says 172.21.15.105:56510) 2024-09-17T16:33:08.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 -- 172.21.15.105:0/593722953 learned_addr learned my addr 172.21.15.105:0/593722953 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): get_auth_request con 0x7f5c1c0a73d0 auth_method 0 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c19d9b640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c19d9b640 10 monclient(hunting): get_auth_request con 0x7f5c1c0a4910 auth_method 0 2024-09-17T16:33:08.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c19d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient(hunting): handle_auth_done global_id 44300 payload 274 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 -- 172.21.15.105:0/593722953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5c1c0a4910 msgr2=0x7f5c1c13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c13eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 -- 172.21.15.105:0/593722953 >> v1:172.21.15.160:6789/0 conn(0x7f5c1c0a68e0 legacy=0x7f5c1c0a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 monclient: found mon.noname-b 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.867+0000 7f5c1ad9d640 1 -- 172.21.15.105:0/593722953 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c0a3690 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1ad9d640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009e80 tx=0x7f5c1401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79db06b6cdc15b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 1 -- 172.21.15.105:0/593722953 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c140090d0 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: dump: 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: _finish_auth 0 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: _check_auth_tickets 2024-09-17T16:33:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 1 -- 172.21.15.105:0/593722953 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c140a5ac0 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 1 -- 172.21.15.105:0/593722953 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c140a5cb0 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 5 monclient: authenticate success, global_id 44300 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 4 set_mon_vals no callback set 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c1959a640 10 monclient: dump: 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 10 monclient: shutdown 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 -- 172.21.15.105:0/593722953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 msgr2=0x7f5c1c141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009e80 tx=0x7f5c1401c8e0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 -- 172.21.15.105:0/593722953 shutdown_connections 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5c1c0a73d0 0x7f5c1c141040 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 --2- 172.21.15.105:0/593722953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c13eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 -- 172.21.15.105:0/593722953 >> 172.21.15.105:0/593722953 conn(0x7f5c1c068ba0 msgr2=0x7f5c1c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 -- 172.21.15.105:0/593722953 shutdown_connections 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 -- 172.21.15.105:0/593722953 wait complete. 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 10 monclient: build_initial_monmap 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 10 monclient: monmap: 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 librados: starting msgr at 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.871+0000 7f5c20f32640 1 librados: starting objecter 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 Processor -- start 2024-09-17T16:33:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- start start 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 librados: setting wanted keys 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 librados: calling monclient init 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: init 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _add_conns ranks=[4,0,7,1,5,8,2,6,3] 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c1396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-c con 0x7f5c1c0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-a con 0x7f5c1c0a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 0x7f5c1c13e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): picked mon.noname-i con 0x7f5c1c0a73d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c143f20 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f5c1c143da0 con 0x7f5c1c0a4910 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5c1c1440a0 con 0x7f5c1c0a73d0 2024-09-17T16:33:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient(hunting): authenticate will time out at 1272.140623s 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1a59c640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 msgr2=0x7f5c1c1396d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1a59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c1396d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58336/0 (socket says 172.21.15.105:58336) 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 -- 172.21.15.105:0/3618383090 learned_addr learned my addr 172.21.15.105:0/3618383090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): get_auth_request con 0x7f5c1c0a68e0 auth_method 0 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1ad9d640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 0x7f5c1c13e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient(hunting): handle_auth_done global_id 34974 payload 824 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 -- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 msgr2=0x7f5c1c13e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 0x7f5c1c13e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 -- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 msgr2=0x7f5c1c1396d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c1396d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 monclient: found mon.noname-a 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 -- 172.21.15.105:0/3618383090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c0a2160 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1ad9d640 10 monclient: get_auth_request con 0x7f5c1c0a73d0 auth_method 0 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1ad9d640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 0x7f5c1c13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c19d9b640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d7d0 tx=0x7f5c0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 client.?.objecter ms_handle_connect 0x7f5c1c0a68e0 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c0c00bb30 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: dump: 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: _finish_auth 0 2024-09-17T16:33:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 5 monclient: authenticate success, global_id 34974 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c00bcf0 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _renew_subs 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c123f80 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c0c0148a0 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 client.34974.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c18d99640 4 set_mon_vals no callback set 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 monclient: dump: 2024-09-17T16:33:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _renew_subs 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c0a2600 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 librados: init done 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c06ddb0 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5c0c01b030 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 0x7f5bfc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f5c0c019020 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 client.34974.objecter ms_dispatch 0x7f5c1c000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 3 client.34974.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 3 client.34974.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 20 client.34974.objecter dump_active .. 0 homeless 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1a59c640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 0x7f5bfc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1a59c640 10 monclient: get_auth_request con 0x7f5bfc03e550 auth_method 0 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c0affd640 10 client.34974.objecter ms_handle_connect 0x7f5bfc03e550 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.875+0000 7f5c1a59c640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 0x7f5bfc040a10 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f5c10003940 tx=0x7f5c10006cf0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.879+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5c0c022060 con 0x7f5c1c0a68e0 2024-09-17T16:33:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.879+0000 7f5c0affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:08.879+0000 7f5c0affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.119+0000 7f5c20f32640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:33:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.119+0000 7f5c20f32640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.119+0000 7f5c20f32640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.119+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5c1c0a6cc0 con 0x7f5c1c0a68e0 2024-09-17T16:33:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c0affd640 1 -- 172.21.15.105:0/3618383090 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7f5c1c0a2160 con 0x7f5c1c0a68e0 2024-09-17T16:33:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c0affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c0affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:09.127 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":46,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 10 librados: watch_flush enter 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 10 librados: watch_flush exit 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 20 client.34974.objecter shutdown clearing up homeless session... 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 10 client.34974.objecter successfully canceled tick 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 msgr2=0x7f5bfc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 0x7f5bfc040a10 secure :-1 s=READY pgs=3453 cs=0 l=1 rev1=1 crypto rx=0x7f5c10003940 tx=0x7f5c10006cf0 comp rx=0 tx=0).stop 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 10 monclient: shutdown 2024-09-17T16:33:09.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 msgr2=0x7f5c1c139c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00d7d0 tx=0x7f5c0c00dc90 comp rx=0 tx=0).stop 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 shutdown_connections 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5bfc03e550 0x7f5bfc040a10 unknown :-1 s=CLOSED pgs=3453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5c1c0a73d0 0x7f5c1c13e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c1c0a68e0 0x7f5c1c139c10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 --2- 172.21.15.105:0/3618383090 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5c1c0a4910 0x7f5c1c1396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 >> 172.21.15.105:0/3618383090 conn(0x7f5c1c068ba0 msgr2=0x7f5c1c0aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 shutdown_connections 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 -- 172.21.15.105:0/3618383090 wait complete. 2024-09-17T16:33:09.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:09.123+0000 7f5c20f32640 1 librados: shutdown 2024-09-17T16:33:12.147 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: get_monmap_and_config 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: build_initial_monmap 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: monmap: 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:12.273353+0000 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:12.273353+0000 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:12.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:12.277 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 Processor -- start 2024-09-17T16:33:12.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 -- start start 2024-09-17T16:33:12.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: init 2024-09-17T16:33:12.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient: _add_conns ranks=[1,6,9,0,5,3,2,7,4,8,10] 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f885c0a4750 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-d con 0x7f885c09e9e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 0x7f885c140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f885c0a5240 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f885c09dd20 con 0x7f885c0a4750 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f885c06c470 con 0x7f885c0a5240 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f885c09e000 con 0x7f885c09e9e0 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:12.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8863d4a640 10 monclient(hunting): authenticate will time out at 1275.539488s 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f8861ae6640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f885c0a4750 legacy=0x7f885c0a4b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 0x7f885c140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:35478/0 (socket says 172.21.15.105:35478) 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 1 -- 172.21.15.105:0/2288032998 learned_addr learned my addr 172.21.15.105:0/2288032998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): get_auth_request con 0x7f885c09e9e0 auth_method 0 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): get_auth_request con 0x7f885c0a5240 auth_method 0 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:12.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 10 monclient(hunting): handle_auth_done global_id 24081 payload 274 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.271+0000 7f88612e5640 1 -- 172.21.15.105:0/2288032998 >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 msgr2=0x7f885c140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 1 --2- 172.21.15.105:0/2288032998 >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 0x7f885c140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 1 -- 172.21.15.105:0/2288032998 >> v1:172.21.15.160:6789/0 conn(0x7f885c0a4750 legacy=0x7f885c0a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 10 monclient: _finish_hunting 0 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 1 monclient: found mon.noname-d 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:12.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 1 -- 172.21.15.105:0/2288032998 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c09e5c0 con 0x7f885c09e9e0 2024-09-17T16:33:12.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88622e7640 1 --2- 172.21.15.105:0/2288032998 >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 0x7f885c140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f88612e5640 1 --2- 172.21.15.105:0/2288032998 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8850009e50 tx=0x7f885001c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b232e6828eae2d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 1 -- 172.21.15.105:0/2288032998 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88500090d0 con 0x7f885c09e9e0 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: dump: 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:12.288 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: _finish_auth 0 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: _check_auth_tickets 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:12.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 1 -- 172.21.15.105:0/2288032998 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88500a5a70 con 0x7f885c09e9e0 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 5 monclient: authenticate success, global_id 24081 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 4 set_mon_vals no callback set 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 1 -- 172.21.15.105:0/2288032998 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88500a5c60 con 0x7f885c09e9e0 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8860ae4640 10 monclient: dump: 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 10 monclient: shutdown 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:12.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 -- 172.21.15.105:0/2288032998 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 msgr2=0x7f885c13e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2288032998 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8850009e50 tx=0x7f885001c8f0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 -- 172.21.15.105:0/2288032998 shutdown_connections 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2288032998 >> v2:172.21.15.105:3300/0 conn(0x7f885c0a5240 0x7f885c140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2288032998 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f885c09e9e0 0x7f885c13e9f0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 -- 172.21.15.105:0/2288032998 >> 172.21.15.105:0/2288032998 conn(0x7f885c0a5840 msgr2=0x7f885c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 -- 172.21.15.105:0/2288032998 shutdown_connections 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 -- 172.21.15.105:0/2288032998 wait complete. 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 10 monclient: build_initial_monmap 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 10 monclient: monmap: 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:12.292 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 librados: starting msgr at 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.275+0000 7f8863d4a640 1 librados: starting objecter 2024-09-17T16:33:12.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 Processor -- start 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- start start 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 librados: setting wanted keys 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 librados: calling monclient init 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: init 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _add_conns ranks=[4,8,0,5,3,6,1,7,2] 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 0x7f885c135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-c con 0x7f885c09e9e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 0x7f885c136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-e con 0x7f885c0a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f885c0a5240 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:12.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f885c143e20 con 0x7f885c0a5240 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f885c143ca0 con 0x7f885c09e9e0 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f885c143fa0 con 0x7f885c0a4750 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient(hunting): authenticate will time out at 1275.544635s 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8861ae6640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 msgr2=0x7f885c135d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88612e5640 1 -- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 msgr2=0x7f885c136240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3304/0 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8861ae6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 0x7f885c135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88612e5640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 0x7f885c136240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58350/0 (socket says 172.21.15.105:58350) 2024-09-17T16:33:12.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 -- 172.21.15.105:0/2342055541 learned_addr learned my addr 172.21.15.105:0/2342055541 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): get_auth_request con 0x7f885c0a5240 auth_method 0 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient(hunting): handle_auth_done global_id 34992 payload 824 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 -- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 msgr2=0x7f885c136240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 0x7f885c136240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 -- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 msgr2=0x7f885c135d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 0x7f885c135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient: _finish_hunting 0 2024-09-17T16:33:12.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 monclient: found mon.noname-a 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 -- 172.21.15.105:0/2342055541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1406d0 con 0x7f885c0a5240 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f88622e7640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f885800e960 tx=0x7f885800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 client.?.objecter ms_handle_connect 0x7f885c0a5240 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8858017070 con 0x7f885c0a5240 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:12.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: dump: 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:12.298 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885800ce70 con 0x7f885c0a5240 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88580138e0 con 0x7f885c0a5240 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 5 monclient: authenticate success, global_id 34992 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884ffff640 4 set_mon_vals no callback set 2024-09-17T16:33:12.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 monclient: dump: 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:12.300 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _renew_subs 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c0a4500 con 0x7f885c0a5240 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 client.34992.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _renew_subs 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c0a3fb0 con 0x7f885c0a5240 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 librados: init done 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:12.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c0a4b30 con 0x7f885c0a5240 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8858013470 con 0x7f885c0a5240 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 0x7f8840040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6194+0+0 (secure 0 0 0) 0x7f885800f040 con 0x7f885c0a5240 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 client.34992.objecter ms_dispatch 0x7f885c000ba0 osd_map(219..219 src has 1..219) 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 3 client.34992.objecter handle_osd_map got epochs [219,219] > 0 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 3 client.34992.objecter handle_osd_map decoding full epoch 219 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 20 client.34992.objecter dump_active .. 0 homeless 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8861ae6640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 0x7f8840040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8861ae6640 10 monclient: get_auth_request con 0x7f884003e550 auth_method 0 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f884e7fc640 10 client.34992.objecter ms_handle_connect 0x7f884003e550 2024-09-17T16:33:12.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.279+0000 7f8861ae6640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 0x7f8840040a10 secure :-1 s=READY pgs=3454 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854006d40 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:12.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.283+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8858020070 con 0x7f885c0a5240 2024-09-17T16:33:12.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.283+0000 7f884e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:12.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.283+0000 7f884e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f8863d4a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f8863d4a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f8863d4a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f885c06e230 con 0x7f885c0a5240 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f884e7fc640 1 -- 172.21.15.105:0/2342055541 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7f885c0a4500 con 0x7f885c0a5240 2024-09-17T16:33:12.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f884e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:12.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.519+0000 7f884e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:12.526 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":46,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"g","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:33:12.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 10 librados: watch_flush enter 2024-09-17T16:33:12.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 10 librados: watch_flush exit 2024-09-17T16:33:12.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 20 client.34992.objecter shutdown clearing up homeless session... 2024-09-17T16:33:12.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 10 client.34992.objecter successfully canceled tick 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 msgr2=0x7f8840040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 0x7f8840040a10 secure :-1 s=READY pgs=3454 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854006d40 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 10 monclient: shutdown 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 msgr2=0x7f885c13e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f885800e960 tx=0x7f885800ee20 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 shutdown_connections 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f884003e550 0x7f8840040a10 unknown :-1 s=CLOSED pgs=3454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f885c0a5240 0x7f885c13e790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f885c0a4750 0x7f885c136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 --2- 172.21.15.105:0/2342055541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f885c09e9e0 0x7f885c135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 >> 172.21.15.105:0/2342055541 conn(0x7f885c0a5840 msgr2=0x7f885c0a1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:12.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 shutdown_connections 2024-09-17T16:33:12.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 -- 172.21.15.105:0/2342055541 wait complete. 2024-09-17T16:33:12.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:12.523+0000 7f8863d4a640 1 librados: shutdown 2024-09-17T16:33:15.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:33:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: get_monmap_and_config 2024-09-17T16:33:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: build_initial_monmap 2024-09-17T16:33:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: monmap: 2024-09-17T16:33:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:15.669372+0000 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:15.669372+0000 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:15.673 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:15.674 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:15.674 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:15.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 Processor -- start 2024-09-17T16:33:15.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 -- start start 2024-09-17T16:33:15.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: init 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient: _add_conns ranks=[4,5,0,7,9,6,8,3,10,2,1] 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-c con 0x7f8b600a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-h con 0x7f8b600a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b601396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6006ebc0 con 0x7f8b601396a0 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8b600a3fc0 con 0x7f8b600a68e0 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8b600a63c0 con 0x7f8b600a4910 2024-09-17T16:33:15.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b66c24640 10 monclient(hunting): authenticate will time out at 1278.935206s 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b649c0640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 msgr2=0x7f8b600a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b649c0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b651c1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b601396a0 0x7f8b60139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46772/0 (socket says 172.21.15.105:46772) 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 learned_addr learned my addr 172.21.15.105:0/1299109139 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b600a4910 auth_method 0 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.667+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:15.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 msgr2=0x7f8b600a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b600a4910 auth_method 0 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b649c0640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:15.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:15.871+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:33:16.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 msgr2=0x7f8b600a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:16.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b649c0640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:33:16.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:16.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b600a4910 auth_method 0 2024-09-17T16:33:16.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:16.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:16.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:16.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:16.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:16.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:16.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:16.275+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:33:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 msgr2=0x7f8b600a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b649c0640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b600a4910 auth_method 0 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:17.079+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:33:18.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): tick 2024-09-17T16:33:18.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:33:18.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:33:18.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 msgr2=0x7f8b600a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8b600a4910 0x7f8b600a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 msgr2=0x7f8b600a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8b600a68e0 0x7f8b600a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.105:6789/0 conn(0x7f8b601396a0 legacy=0x7f8b60139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient: _add_conns ranks=[1,10,9,3,7,8,2,6] 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b4c003850 addr v1:172.21.15.160:6789/0 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-f con 0x7f8b4c0074b0 addr v2:172.21.15.160:3300/0 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:18.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f8b4c00a060 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b600a3690 con 0x7f8b4c003850 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8b600a3fc0 con 0x7f8b4c00a060 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4c00dc00 con 0x7f8b4c0074b0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5e7fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.160:6789/0 conn(0x7f8b4c003850 legacy=0x7f8b4c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 msgr2=0x7f8b4c009970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b651c1640 10 monclient(hunting): get_auth_request con 0x7f8b4c00a060 auth_method 0 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b651c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:18.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b651c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.667+0000 7f8b651c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.671+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:33:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.671+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.671+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:18.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.671+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:18.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.871+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.160:6789/0 conn(0x7f8b4c003850 legacy=0x7f8b4c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:18.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.871+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 msgr2=0x7f8b4c009970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:18.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.871+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:33:18.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 10 monclient(hunting): get_auth_request con 0x7f8b4c00a060 auth_method 0 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:18.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:18.875+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:33:19.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.275+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 msgr2=0x7f8b4c009970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:19.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.275+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.160:6789/0 conn(0x7f8b4c003850 legacy=0x7f8b4c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:19.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.275+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 10 monclient(hunting): get_auth_request con 0x7f8b4c00a060 auth_method 0 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:19.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:19.279+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:33:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.079+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.160:6789/0 conn(0x7f8b4c003850 legacy=0x7f8b4c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:33:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.079+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 msgr2=0x7f8b4c009970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.079+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:33:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 10 monclient(hunting): get_auth_request con 0x7f8b4c00a060 auth_method 0 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:33:20.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:20.083+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:33:21.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): tick 2024-09-17T16:33:21.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:33:21.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:33:21.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 msgr2=0x7f8b4c009970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.160:3300/0 conn(0x7f8b4c0074b0 0x7f8b4c009970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 msgr2=0x7f8b4c00c500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> v2:172.21.15.105:3300/0 conn(0x7f8b4c00a060 0x7f8b4c00c500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 >> v1:172.21.15.160:6789/0 conn(0x7f8b4c003850 legacy=0x7f8b4c005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient: _add_conns ranks=[3,2,8,7,6] 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 reap_dead start 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 0x7f8b4c00d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-g con 0x7f8b4c00ccd0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 0x7f8b4c0122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f8b4c00d6e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 0x7f8b4c014d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.667+0000 7f8b5e7fc640 10 monclient(hunting): picked mon.noname-e con 0x7f8b4c0128d0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8b600a3fc0 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c00dc00 con 0x7f8b4c00ccd0 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1299109139 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8b4c015520 con 0x7f8b4c0128d0 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5e7fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 reap_dead start 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b649c0640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 msgr2=0x7f8b4c00d0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b651c1640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 msgr2=0x7f8b4c014d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3304/0 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b649c0640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 0x7f8b4c00d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b651c1640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 0x7f8b4c014d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:21.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 0x7f8b4c0122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b4c00d6e0 auth_method 0 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_done global_id 44309 payload 274 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 msgr2=0x7f8b4c014d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 0x7f8b4c014d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 msgr2=0x7f8b4c00d0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 0x7f8b4c00d0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:33:21.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 monclient: found mon.noname-b 2024-09-17T16:33:21.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:21.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.671+0000 7f8b5ffff640 1 -- 172.21.15.105:0/1299109139 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c00de10 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5ffff640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 0x7f8b4c0122c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b500036a0 tx=0x7f8b5001cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d292c9edfbc921d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 1 -- 172.21.15.105:0/1299109139 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b500ab070 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: dump: 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:21.682 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 1 -- 172.21.15.105:0/1299109139 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b500af400 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 1 -- 172.21.15.105:0/1299109139 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b500afa20 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b5f7fe640 10 monclient: dump: 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:21.684 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b66c24640 5 monclient: authenticate success, global_id 44309 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b66c24640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:21.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.675+0000 7f8b66c24640 4 set_mon_vals no callback set 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: shutdown 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 msgr2=0x7f8b4c0122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 0x7f8b4c0122c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b500036a0 tx=0x7f8b5001cfa0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- 172.21.15.105:0/1299109139 shutdown_connections 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c00d6e0 0x7f8b4c0122c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b4c00ccd0 0x7f8b4c00d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- 172.21.15.105:0/1299109139 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8b4c0128d0 0x7f8b4c014d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- 172.21.15.105:0/1299109139 >> 172.21.15.105:0/1299109139 conn(0x7f8b60068ba0 msgr2=0x7f8b60068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- 172.21.15.105:0/1299109139 shutdown_connections 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- 172.21.15.105:0/1299109139 wait complete. 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: build_initial_monmap 2024-09-17T16:33:21.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: monmap: 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:21.692 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 librados: starting msgr at 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 librados: starting objecter 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 Processor -- start 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- start start 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 librados: setting wanted keys 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 librados: calling monclient init 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: init 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient: _add_conns ranks=[6,0,2,3,5,7,1,4,8] 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b4c00ccd0 0x7f8b601d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-d con 0x7f8b4c00ccd0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 0x7f8b601d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-a con 0x7f8b4c00d6e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 0x7f8b60125220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): picked mon.noname-b con 0x7f8b4c0128d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6013a410 con 0x7f8b4c00d6e0 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6013a290 con 0x7f8b4c0128d0 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8b600a6cc0 con 0x7f8b4c00ccd0 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b66c24640 10 monclient(hunting): authenticate will time out at 1284.947732s 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.679+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 0x7f8b601d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:21.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 0x7f8b60125220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b4c00ccd0 0x7f8b601d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 0x7f8b601d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47336/0 (socket says 172.21.15.105:47336) 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): get_auth_request con 0x7f8b4c0128d0 auth_method 0 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 1 -- 172.21.15.105:0/2210612285 learned_addr learned my addr 172.21.15.105:0/2210612285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b4c00d6e0 auth_method 0 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): get_auth_request con 0x7f8b4c00ccd0 auth_method 0 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:21.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient(hunting): handle_auth_done global_id 44318 payload 824 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 -- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b4c00ccd0 msgr2=0x7f8b601d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b4c00ccd0 0x7f8b601d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 -- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 msgr2=0x7f8b601d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 0x7f8b601d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient: _finish_hunting 0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 monclient: found mon.noname-b 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 -- 172.21.15.105:0/2210612285 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b60138920 con 0x7f8b4c0128d0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b651c1640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 0x7f8b60125220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b600a7a80 tx=0x7f8b58007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 client.?.objecter ms_handle_connect 0x7f8b4c0128d0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b58004d80 con 0x7f8b4c0128d0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:21.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: dump: 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:21.698 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 5 monclient: authenticate success, global_id 44318 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5801c070 con 0x7f8b4c0128d0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _renew_subs 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b600a9dc0 con 0x7f8b4c0128d0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b58014dc0 con 0x7f8b4c0128d0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:21.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 monclient: dump: 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 client.44318.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _renew_subs 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b60126d10 con 0x7f8b4c0128d0 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 1 librados: init done 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5effd640 4 set_mon_vals no callback set 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b600a7050 con 0x7f8b4c0128d0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8b58014950 con 0x7f8b4c0128d0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 0x7f8b4c043020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(221..221 src has 1..221) ==== 6646+0+0 (secure 0 0 0) 0x7f8b58010070 con 0x7f8b4c0128d0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 10 client.44318.objecter ms_dispatch 0x7f8b60000ba0 osd_map(221..221 src has 1..221) 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 3 client.44318.objecter handle_osd_map got epochs [221,221] > 0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 3 client.44318.objecter handle_osd_map decoding full epoch 221 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b5d7fa640 20 client.44318.objecter dump_active .. 0 homeless 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 0x7f8b4c043020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.683+0000 7f8b649c0640 10 monclient: get_auth_request con 0x7f8b4c040b60 auth_method 0 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.687+0000 7f8b5d7fa640 10 client.44318.objecter ms_handle_connect 0x7f8b4c040b60 2024-09-17T16:33:21.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.687+0000 7f8b649c0640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 0x7f8b4c043020 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 crypto rx=0x7f8b54003be0 tx=0x7f8b54009a20 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:21.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.687+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b60138920 con 0x7f8b4c0128d0 2024-09-17T16:33:21.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.687+0000 7f8b5d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:21.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.687+0000 7f8b5d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:21.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b66c24640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:33:21.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b66c24640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b66c24640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8b600746d0 con 0x7f8b4c0128d0 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b5d7fa640 1 -- 172.21.15.105:0/2210612285 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3188 (secure 0 0 0) 0x7f8b58020cc0 con 0x7f8b4c0128d0 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b5d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.931+0000 7f8b5d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:21.937 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":48,"quorum":[0,2,5,6,7],"quorum_names":["a","b","h","d","i"],"quorum_leader_name":"h","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 10 librados: watch_flush enter 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 10 librados: watch_flush exit 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 20 client.44318.objecter shutdown clearing up homeless session... 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 10 client.44318.objecter successfully canceled tick 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 msgr2=0x7f8b4c043020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 0x7f8b4c043020 secure :-1 s=READY pgs=3455 cs=0 l=1 rev1=1 crypto rx=0x7f8b54003be0 tx=0x7f8b54009a20 comp rx=0 tx=0).stop 2024-09-17T16:33:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 10 monclient: shutdown 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 msgr2=0x7f8b60125220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 0x7f8b60125220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b600a7a80 tx=0x7f8b58007a40 comp rx=0 tx=0).stop 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 shutdown_connections 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b4c040b60 0x7f8b4c043020 unknown :-1 s=CLOSED pgs=3455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b4c00d6e0 0x7f8b601d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8b4c0128d0 0x7f8b60125220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 --2- 172.21.15.105:0/2210612285 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8b4c00ccd0 0x7f8b601d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 >> 172.21.15.105:0/2210612285 conn(0x7f8b60068ba0 msgr2=0x7f8b600a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 shutdown_connections 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 -- 172.21.15.105:0/2210612285 wait complete. 2024-09-17T16:33:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:21.935+0000 7f8b66c24640 1 librados: shutdown 2024-09-17T16:33:21.955 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-17T16:33:21.955 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-17T16:33:21.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:33:22.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.067+0000 7f8c91e89640 10 monclient: get_monmap_and_config 2024-09-17T16:33:22.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.067+0000 7f8c91e89640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.067+0000 7f8c91e89640 10 monclient: monmap: 2024-09-17T16:33:22.074 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.074 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.074 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:22.071350+0000 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:22.071350+0000 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.075 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.076 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.076 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:22.076 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:22.076 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 Processor -- start 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 -- start start 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient: init 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient: _add_conns ranks=[3,0,6,5,9,10,8,4,7,1,2] 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 0x7f8c8c0a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-g con 0x7f8c8c0a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8c8c09eec0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 0x7f8c8c139c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-d con 0x7f8c8c0a5070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8c09df90 con 0x7f8c8c09eec0 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c09dcb0 con 0x7f8c8c0a4750 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8c8c09e270 con 0x7f8c8c0a5070 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c91e89640 10 monclient(hunting): authenticate will time out at 1285.337238s 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8b7fe640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 msgr2=0x7f8c8c0a4b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8c8c09eec0 0x7f8c8c139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48068/0 (socket says 172.21.15.105:48068) 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8affd640 1 -- 172.21.15.105:0/4110862067 learned_addr learned my addr 172.21.15.105:0/4110862067 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 0x7f8c8c139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8b7fe640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 0x7f8c8c0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): get_auth_request con 0x7f8c8c0a5070 auth_method 0 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 1 -- 172.21.15.105:0/4110862067 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1304307568 0 0) 0x7f8c78001120 con 0x7f8c8c09eec0 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 10 monclient(hunting): _negotiate 2024-09-17T16:33:22.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 10 monclient(hunting): my global_id is 35019 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8a7fc640 1 -- 172.21.15.105:0/4110862067 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c70003530 con 0x7f8c8c09eec0 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient(hunting): handle_auth_done global_id 24099 payload 274 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 -- 172.21.15.105:0/4110862067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 msgr2=0x7f8c8c0a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 0x7f8c8c0a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 -- 172.21.15.105:0/4110862067 >> v1:172.21.15.105:6789/0 conn(0x7f8c8c09eec0 legacy=0x7f8c8c139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 monclient: found mon.noname-d 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 -- 172.21.15.105:0/4110862067 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c0a66e0 con 0x7f8c8c0a5070 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.071+0000 7f8c8bfff640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 0x7f8c8c139c40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009dc0 tx=0x7f8c8401c870 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a192dc25db1d8643 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 1 -- 172.21.15.105:0/4110862067 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c840b2070 con 0x7f8c8c0a5070 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: dump: 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.090 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 5 monclient: authenticate success, global_id 24099 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 1 -- 172.21.15.105:0/4110862067 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c840a5a70 con 0x7f8c8c0a5070 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:22.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 1 -- 172.21.15.105:0/4110862067 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c840a5c20 con 0x7f8c8c0a5070 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 4 set_mon_vals no callback set 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8a7fc640 10 monclient: dump: 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.092 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: shutdown 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4110862067 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 msgr2=0x7f8c8c139c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 0x7f8c8c139c40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009dc0 tx=0x7f8c8401c870 comp rx=0 tx=0).stop 2024-09-17T16:33:22.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4110862067 shutdown_connections 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c8c0a5070 0x7f8c8c139c40 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4110862067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c0a4750 0x7f8c8c0a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4110862067 >> 172.21.15.105:0/4110862067 conn(0x7f8c8c0a5840 msgr2=0x7f8c8c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4110862067 shutdown_connections 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4110862067 wait complete. 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: monmap: 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:22.094 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.095 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 librados: starting msgr at 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 librados: starting objecter 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 Processor -- start 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- start start 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 librados: setting wanted keys 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 librados: calling monclient init 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: init 2024-09-17T16:33:22.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: _add_conns ranks=[3,7,8,4,2,6,0,1,5] 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c09eec0 0x7f8c8c122030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-g con 0x7f8c8c09eec0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-i con 0x7f8c8c0a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8c8c0a5070 0x7f8c8c11fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): picked mon.noname-e con 0x7f8c8c0a5070 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c13a390 con 0x7f8c8c09eec0 2024-09-17T16:33:22.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8c8c1441c0 con 0x7f8c8c0a4750 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8c8c123550 con 0x7f8c8c0a5070 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient(hunting): authenticate will time out at 1285.342378s 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8bfff640 1 -- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8c8c0a5070 msgr2=0x7f8c8c11fcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3304/0 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8c8c0a5070 0x7f8c8c11fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8b7fe640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c09eec0 msgr2=0x7f8c8c122030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c09eec0 0x7f8c8c122030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:52242/0 (socket says 172.21.15.105:52242) 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 -- 172.21.15.105:0/4203978594 learned_addr learned my addr 172.21.15.105:0/4203978594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): get_auth_request con 0x7f8c8c0a4750 auth_method 0 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient(hunting): handle_auth_done global_id 34585 payload 824 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8c8c0a5070 msgr2=0x7f8c8c11fcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8c8c0a5070 0x7f8c8c11fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c09eec0 msgr2=0x7f8c8c122030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c8c09eec0 0x7f8c8c122030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 monclient: found mon.noname-i 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c123830 con 0x7f8c8c0a4750 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c8affd640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c730 tx=0x7f8c7800cbf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 client.?.objecter ms_handle_connect 0x7f8c8c0a4750 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c8c0a4750 2024-09-17T16:33:22.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: dump: 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.101 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 5 monclient: authenticate success, global_id 34585 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: _renew_subs 2024-09-17T16:33:22.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c78005d30 con 0x7f8c8c0a4750 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c0a3fb0 con 0x7f8c8c0a4750 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c780074e0 con 0x7f8c8c0a4750 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.075+0000 7f8c89ffb640 4 set_mon_vals no callback set 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 10 client.34585.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: dump: 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.103 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.104 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 10 monclient: _renew_subs 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c122f90 con 0x7f8c8c0a4750 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 1 librados: init done 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 0x7f8c8c1201f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c06cd40 con 0x7f8c8c121510 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 0x7f8c8c1201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient: get_auth_request con 0x7f8c8c121510 auth_method 0 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_done global_id 35028 payload 824 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c8c123190 con 0x7f8c8c121510 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8b7fe640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 0x7f8c8c1201f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b450 tx=0x7f8c7c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 client.34585.objecter ms_handle_connect 0x7f8c8c121510 2024-09-17T16:33:22.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 client.34585.objecter resend_mon_ops 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8c78007680 con 0x7f8c8c0a4750 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8c7403e550 0x7f8c74040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c7c003bd0 con 0x7f8c8c121510 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: dump: 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.107 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.108 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(222..222 src has 1..222) ==== 7069+0+0 (secure 0 0 0) 0x7f8c7804f370 con 0x7f8c8c0a4750 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 client.34585.objecter ms_dispatch 0x7f8c8c000ba0 osd_map(222..222 src has 1..222) 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 3 client.34585.objecter handle_osd_map got epochs [222,222] > 0 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 3 client.34585.objecter handle_osd_map decoding full epoch 222 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 20 client.34585.objecter dump_active .. 0 homeless 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8c7c015070 con 0x7f8c8c121510 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 msgr2=0x7f8c8c1201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 0x7f8c8c1201f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b450 tx=0x7f8c7c00b910 comp rx=0 tx=0).stop 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 msgr2=0x7f8c8c1201f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c121510 0x7f8c8c1201f0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b450 tx=0x7f8c7c00b910 comp rx=0 tx=0).stop 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8bfff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8c7403e550 0x7f8c74040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8bfff640 10 monclient: get_auth_request con 0x7f8c7403e550 auth_method 0 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c83fff640 10 client.34585.objecter ms_handle_connect 0x7f8c7403e550 2024-09-17T16:33:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.079+0000 7f8c8bfff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8c7403e550 0x7f8c74040a10 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7f8c840af000 tx=0x7f8c840a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.083+0000 7f8c91e89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 0x7f8c8c120730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c051e80 con 0x7f8c8c1d1bd0 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 0x7f8c8c120730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient: get_auth_request con 0x7f8c8c1d1bd0 auth_method 0 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 10 monclient(hunting): handle_auth_done global_id 35037 payload 824 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 1 -- 172.21.15.105:0/4203978594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8c8c0a4b30 con 0x7f8c8c1d1bd0 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8b7fe640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 0x7f8c8c120730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b450 tx=0x7f8c7c0059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 client.34585.objecter ms_handle_connect 0x7f8c8c1d1bd0 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 client.34585.objecter resend_mon_ops 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c7c0057b0 con 0x7f8c8c1d1bd0 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 monclient: dump: 2024-09-17T16:33:22.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.115 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9410 (secure 0 0 0) 0x7f8c7c005340 con 0x7f8c8c1d1bd0 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 msgr2=0x7f8c8c120730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 0x7f8c8c120730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b450 tx=0x7f8c7c0059e0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 msgr2=0x7f8c8c120730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c83fff640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c8c1d1bd0 0x7f8c8c120730 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 48, 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 192890, 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:22.117 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: 7 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.118 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.119 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.120 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:22.121 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.122 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.124 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.126 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.127 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:22.128 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.130 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.131 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:22.133 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:22.135 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:22.136 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:22.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:22.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:22.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:22.140 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:22.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 10 librados: watch_flush enter 2024-09-17T16:33:22.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 10 librados: watch_flush exit 2024-09-17T16:33:22.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 20 client.34585.objecter shutdown clearing up homeless session... 2024-09-17T16:33:22.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 10 client.34585.objecter successfully canceled tick 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8c7403e550 msgr2=0x7f8c74040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8c7403e550 0x7f8c74040a10 secure :-1 s=READY pgs=3456 cs=0 l=1 rev1=1 crypto rx=0x7f8c840af000 tx=0x7f8c840a7070 comp rx=0 tx=0).stop 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 10 monclient: shutdown 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 msgr2=0x7f8c8c1d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c730 tx=0x7f8c7800cbf0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c8bfff640 1 -- 172.21.15.105:0/4203978594 reap_dead start 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 shutdown_connections 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 --2- 172.21.15.105:0/4203978594 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c8c0a4750 0x7f8c8c1d1690 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 >> 172.21.15.105:0/4203978594 conn(0x7f8c8c0a5840 msgr2=0x7f8c8c0a1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 shutdown_connections 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 -- 172.21.15.105:0/4203978594 wait complete. 2024-09-17T16:33:22.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.087+0000 7f8c91e89640 1 librados: shutdown 2024-09-17T16:33:22.144 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-17T16:33:22.144 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon b 2024-09-17T16:33:22.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:33:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: get_monmap_and_config 2024-09-17T16:33:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: monmap: 2024-09-17T16:33:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:22.259979+0000 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:22.259979+0000 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 Processor -- start 2024-09-17T16:33:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 -- start start 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: init 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient: _add_conns ranks=[3,0,2,5,6,7,1,10,9,4,8] 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 0x7f187409b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): picked mon.noname-g con 0x7f187409f8e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18740a46f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f187413dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): picked mon.noname-b con 0x7f187413b980 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187409f3f0 con 0x7f18740a46f0 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f18740a1a50 con 0x7f187413b980 2024-09-17T16:33:22.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f18740a1ca0 con 0x7f187409f8e0 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1879337640 10 monclient(hunting): authenticate will time out at 1285.525720s 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1872ffd640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 msgr2=0x7f187409b050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f187413dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18727fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f18740a46f0 0x7f187409b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48082/0 (socket says 172.21.15.105:48082) 2024-09-17T16:33:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18727fc640 1 -- 172.21.15.105:0/330386389 learned_addr learned my addr 172.21.15.105:0/330386389 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1872ffd640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 0x7f187409b050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): get_auth_request con 0x7f187413b980 auth_method 0 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 1 -- 172.21.15.105:0/330386389 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852406145 0 0) 0x7f185c001120 con 0x7f18740a46f0 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 10 monclient(hunting): _negotiate 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 10 monclient(hunting): my global_id is 35046 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f1871ffb640 1 -- 172.21.15.105:0/330386389 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1860003530 con 0x7f18740a46f0 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient(hunting): handle_auth_done global_id 44327 payload 274 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 -- 172.21.15.105:0/330386389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 msgr2=0x7f187409b050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 0x7f187409b050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 -- 172.21.15.105:0/330386389 >> v1:172.21.15.105:6789/0 conn(0x7f18740a46f0 legacy=0x7f187409b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 monclient: found mon.noname-b 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.259+0000 7f18737fe640 1 -- 172.21.15.105:0/330386389 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740a27c0 con 0x7f187413b980 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f187413dd70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1864009820 tx=0x7f186401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427aa067f78bf4b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 1 -- 172.21.15.105:0/330386389 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18640b0070 con 0x7f187413b980 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: dump: 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.296 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.297 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.298 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 5 monclient: authenticate success, global_id 44327 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 1 -- 172.21.15.105:0/330386389 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18640aa810 con 0x7f187413b980 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:22.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:22.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 1 -- 172.21.15.105:0/330386389 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18640aae30 con 0x7f187413b980 2024-09-17T16:33:22.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 4 set_mon_vals no callback set 2024-09-17T16:33:22.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1871ffb640 10 monclient: dump: 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.301 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.302 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: shutdown 2024-09-17T16:33:22.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/330386389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 msgr2=0x7f187413dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f187413dd70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1864009820 tx=0x7f186401c3a0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/330386389 shutdown_connections 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f187413dd70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- 172.21.15.105:0/330386389 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f187409f8e0 0x7f187409b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/330386389 >> 172.21.15.105:0/330386389 conn(0x7f18740a83e0 msgr2=0x7f18740a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/330386389 shutdown_connections 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/330386389 wait complete. 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: monmap: 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.305 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:22.306 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.307 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 librados: starting msgr at 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 librados: starting objecter 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 Processor -- start 2024-09-17T16:33:22.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- start start 2024-09-17T16:33:22.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 librados: setting wanted keys 2024-09-17T16:33:22.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 librados: calling monclient init 2024-09-17T16:33:22.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: init 2024-09-17T16:33:22.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _add_conns ranks=[6,3,2,8,0,1,7,5,4] 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f187409f8e0 0x7f187409b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): picked mon.noname-d con 0x7f187409f8e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18740a46f0 0x7f18740710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): picked mon.noname-g con 0x7f18740a46f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): picked mon.noname-b con 0x7f187413b980 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:22.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f18740a3170 con 0x7f187413b980 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f18740a2ff0 con 0x7f18740a46f0 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f18740a32f0 con 0x7f187409f8e0 2024-09-17T16:33:22.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient(hunting): authenticate will time out at 1285.530804s 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:41046/0 (socket says 172.21.15.105:41046) 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 -- 172.21.15.105:0/3002156240 learned_addr learned my addr 172.21.15.105:0/3002156240 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): get_auth_request con 0x7f187413b980 auth_method 0 2024-09-17T16:33:22.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f187409f8e0 0x7f187409b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18727fc640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18740a46f0 msgr2=0x7f18740710c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): get_auth_request con 0x7f187409f8e0 auth_method 0 2024-09-17T16:33:22.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18727fc640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18740a46f0 0x7f18740710c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient(hunting): handle_auth_done global_id 44336 payload 824 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f187409f8e0 msgr2=0x7f187409b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f187409f8e0 0x7f187409b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18740a46f0 msgr2=0x7f18740710c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18740a46f0 0x7f18740710c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 monclient: found mon.noname-b 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1874072ba0 con 0x7f187413b980 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f187409f8e0 0x7f187409b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18737fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18640ae040 tx=0x7f1864019140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 client.?.objecter ms_handle_connect 0x7f187413b980 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18640b0050 con 0x7f187413b980 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: dump: 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.316 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18640bc070 con 0x7f187413b980 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18640044b0 con 0x7f187413b980 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f186b7fe640 10 monclient: dump: 2024-09-17T16:33:22.317 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.318 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f18717fa640 4 set_mon_vals no callback set 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 5 monclient: authenticate success, global_id 44336 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _renew_subs 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1874074a60 con 0x7f187413b980 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 client.44336.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _renew_subs 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18740a07b0 con 0x7f187413b980 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.263+0000 7f1879337640 1 librados: init done 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1879337640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1879337640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 0x7f1874074400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1874136fb0 con 0x7f1874071b40 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 0x7f1874074400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient: get_auth_request con 0x7f1874071b40 auth_method 0 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1864004040 con 0x7f187413b980 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 10 monclient(hunting): handle_auth_done global_id 44345 payload 824 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18740a0410 con 0x7f1874071b40 2024-09-17T16:33:22.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f185403e580 0x7f1854040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(222..222 src has 1..222) ==== 7069+0+0 (secure 0 0 0) 0x7f18640ea8f0 con 0x7f187413b980 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 client.44336.objecter ms_dispatch 0x7f1874000ba0 osd_map(222..222 src has 1..222) 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 3 client.44336.objecter handle_osd_map got epochs [222,222] > 0 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 3 client.44336.objecter handle_osd_map decoding full epoch 222 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 0x7f1874074400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f186c007590 tx=0x7f186c007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 20 client.44336.objecter dump_active .. 0 homeless 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 client.44336.objecter ms_handle_connect 0x7f1874071b40 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 client.44336.objecter resend_mon_ops 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f186c01a070 con 0x7f1874071b40 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 monclient: dump: 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.321 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.322 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f18727fc640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f185403e580 0x7f1854040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f18727fc640 10 monclient: get_auth_request con 0x7f185403e580 auth_method 0 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f186c00a6b0 con 0x7f1874071b40 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 msgr2=0x7f1874074400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 0x7f1874074400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f186c007590 tx=0x7f186c007a50 comp rx=0 tx=0).stop 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 msgr2=0x7f1874074400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874071b40 0x7f1874074400 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f186c007590 tx=0x7f186c007a50 comp rx=0 tx=0).stop 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f18727fc640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f185403e580 0x7f1854040a40 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7f185c0012c0 tx=0x7f185c001250 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.267+0000 7f186b7fe640 10 client.44336.objecter ms_handle_connect 0x7f185403e580 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1879337640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:22.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1879337640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 0x7f1874138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1879337640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1874073260 con 0x7f1874138760 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 0x7f1874138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient: get_auth_request con 0x7f1874138760 auth_method 0 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 10 monclient(hunting): handle_auth_done global_id 44354 payload 824 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.271+0000 7f1872ffd640 1 -- 172.21.15.105:0/3002156240 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1874071f20 con 0x7f1874138760 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 client.44336.objecter ms_handle_connect 0x7f1874138760 2024-09-17T16:33:22.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 client.44336.objecter resend_mon_ops 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1872ffd640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 0x7f1874138b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f186c017000 tx=0x7f186c005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f186c01a070 con 0x7f1874138760 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 monclient: dump: 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.325 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9242 (secure 0 0 0) 0x7f186c01d980 con 0x7f1874138760 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 msgr2=0x7f1874138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 0x7f1874138b70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f186c017000 tx=0x7f186c005d60 comp rx=0 tx=0).stop 2024-09-17T16:33:22.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 msgr2=0x7f1874138b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f186b7fe640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1874138760 0x7f1874138b70 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f186c017000 tx=0x7f186c005d60 comp rx=0 tx=0).stop 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 48, 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 52927, 2024-09-17T16:33:22.327 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: 7 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.328 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.329 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:22.330 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.331 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:22.334 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:22.336 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:22.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.338 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.339 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.340 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.341 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.344 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.346 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.348 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.350 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:22.350 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 10 librados: watch_flush enter 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 10 librados: watch_flush exit 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 20 client.44336.objecter shutdown clearing up homeless session... 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 10 client.44336.objecter successfully canceled tick 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f185403e580 msgr2=0x7f1854040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f185403e580 0x7f1854040a40 secure :-1 s=READY pgs=3457 cs=0 l=1 rev1=1 crypto rx=0x7f185c0012c0 tx=0x7f185c001250 comp rx=0 tx=0).stop 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 10 monclient: shutdown 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 msgr2=0x7f1874071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18640ae040 tx=0x7f1864019140 comp rx=0 tx=0).stop 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f18737fe640 1 -- 172.21.15.105:0/3002156240 reap_dead start 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 shutdown_connections 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 --2- 172.21.15.105:0/3002156240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f187413b980 0x7f1874071600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 >> 172.21.15.105:0/3002156240 conn(0x7f18740a83e0 msgr2=0x7f187409c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 shutdown_connections 2024-09-17T16:33:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 -- 172.21.15.105:0/3002156240 wait complete. 2024-09-17T16:33:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.275+0000 7f1879337640 1 librados: shutdown 2024-09-17T16:33:22.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-17T16:33:22.353 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-17T16:33:22.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: get_monmap_and_config 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: monmap: 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:22.475846+0000 2024-09-17T16:33:22.478 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:22.475846+0000 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:22.479 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 Processor -- start 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 -- start start 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: init 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient: _add_conns ranks=[0,8,2,1,10,7,4,5,6,9,3] 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc0ec0a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 0x7fc0ec13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-e con 0x7fc0ec0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 0x7fc0ec141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-b con 0x7fc0ec0a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec06ebc0 con 0x7fc0ec0a68e0 2024-09-17T16:33:22.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eb7fe640 1 -- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 msgr2=0x7fc0ec13eaf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3304/0 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec0a3fc0 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eb7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 0x7fc0ec13eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc0ec0a63c0 con 0x7fc0ec0a4910 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f259e640 10 monclient(hunting): authenticate will time out at 1285.741742s 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0ebfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc0ec0a68e0 0x7fc0ec0a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48092/0 (socket says 172.21.15.105:48092) 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0ebfff640 1 -- 172.21.15.105:0/3675283561 learned_addr learned my addr 172.21.15.105:0/3675283561 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 0x7fc0ec141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): get_auth_request con 0x7fc0ec0a73d0 auth_method 0 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 1 -- 172.21.15.105:0/3675283561 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362968264 0 0) 0x7fc0ec06ebc0 con 0x7fc0ec0a68e0 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 10 monclient(hunting): _negotiate 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 10 monclient(hunting): my global_id is 35055 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0eaffd640 1 -- 172.21.15.105:0/3675283561 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0d4003530 con 0x7fc0ec0a68e0 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_done global_id 44363 payload 274 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 msgr2=0x7fc0ec13eaf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:22.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 0x7fc0ec13eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/3675283561 >> v1:172.21.15.105:6789/0 conn(0x7fc0ec0a68e0 legacy=0x7fc0ec0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 monclient: found mon.noname-b 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.475+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/3675283561 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0a3690 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 0x7fc0ec141040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009dc0 tx=0x7fc0e401c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba3781783b9ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 1 -- 172.21.15.105:0/3675283561 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e40b2070 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: dump: 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 5 monclient: authenticate success, global_id 44363 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 1 -- 172.21.15.105:0/3675283561 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e40a5ac0 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 1 -- 172.21.15.105:0/3675283561 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e40a5c70 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 4 set_mon_vals no callback set 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eaffd640 10 monclient: dump: 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.495 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: shutdown 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 msgr2=0x7fc0ec141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 0x7fc0ec141040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009dc0 tx=0x7fc0e401c840 comp rx=0 tx=0).stop 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- 172.21.15.105:0/3675283561 shutdown_connections 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc0ec0a73d0 0x7fc0ec141040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- 172.21.15.105:0/3675283561 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc0ec0a4910 0x7fc0ec13eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- 172.21.15.105:0/3675283561 >> 172.21.15.105:0/3675283561 conn(0x7fc0ec068ba0 msgr2=0x7fc0ec068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- 172.21.15.105:0/3675283561 shutdown_connections 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- 172.21.15.105:0/3675283561 wait complete. 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: monmap: 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.497 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 librados: starting msgr at 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 librados: starting objecter 2024-09-17T16:33:22.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 Processor -- start 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- start start 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 librados: setting wanted keys 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 librados: calling monclient init 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: init 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient: _add_conns ranks=[5,1,7,4,6,8,2,3,0] 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 0x7fc0ec1d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-h con 0x7fc0ec0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec0a68e0 0x7fc0ec1d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-f con 0x7fc0ec0a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): picked mon.noname-i con 0x7fc0ec0a73d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec143f20 con 0x7fc0ec0a68e0 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc0ec143da0 con 0x7fc0ec0a4910 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc0ec1440a0 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f259e640 10 monclient(hunting): authenticate will time out at 1285.746973s 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eb7fe640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec0a68e0 msgr2=0x7fc0ec1d14f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0eb7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec0a68e0 0x7fc0ec1d14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 0x7fc0ec1d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 0x7fc0ec1d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:42150/0 (socket says 172.21.15.105:42150) 2024-09-17T16:33:22.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:52244/0 (socket says 172.21.15.105:52244) 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 1 -- 172.21.15.105:0/2100833437 learned_addr learned my addr 172.21.15.105:0/2100833437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): get_auth_request con 0x7fc0ec0a4910 auth_method 0 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): get_auth_request con 0x7fc0ec0a73d0 auth_method 0 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.479+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 10 monclient(hunting): handle_auth_done global_id 34594 payload 824 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 msgr2=0x7fc0ec1d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 0x7fc0ec1d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec0a68e0 msgr2=0x7fc0ec1d14f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0ec0a68e0 0x7fc0ec1d14f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 monclient: found mon.noname-i 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec129d30 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc0ec0a4910 0x7fc0ec1d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:33:22.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f0b3b640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40ac000 tx=0x7fc0e40195e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.?.objecter ms_handle_connect 0x7fc0ec0a73d0 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e40b2050 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: dump: 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.503 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e40be070 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e4003e10 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 5 monclient: authenticate success, global_id 34594 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: dump: 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.505 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ea7fc640 4 set_mon_vals no callback set 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 monclient: _renew_subs 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec12a010 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 client.34594.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 monclient: _renew_subs 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec141230 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 1 librados: init done 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 0x7fc0ec126690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc0ec051e80 con 0x7fc0ec1262b0 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 0x7fc0ec126690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient: get_auth_request con 0x7fc0ec1262b0 auth_method 0 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_done global_id 24117 payload 824 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0ec12a510 con 0x7fc0ec1262b0 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0ebfff640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 0x7fc0ec126690 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e000d150 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.34594.objecter ms_handle_connect 0x7fc0ec1262b0 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.34594.objecter resend_mon_ops 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc0e40039a0 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc0d803e550 0x7fc0d8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(222..222 src has 1..222) ==== 7069+0+0 (secure 0 0 0) 0x7fc0e40eb0e0 con 0x7fc0ec0a73d0 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.34594.objecter ms_dispatch 0x7fc0ec000ba0 osd_map(222..222 src has 1..222) 2024-09-17T16:33:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 3 client.34594.objecter handle_osd_map got epochs [222,222] > 0 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 3 client.34594.objecter handle_osd_map decoding full epoch 222 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 20 client.34594.objecter dump_active .. 0 homeless 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e000d880 con 0x7fc0ec1262b0 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: dump: 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc0e0018070 con 0x7fc0ec1262b0 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 msgr2=0x7fc0ec126690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 0x7fc0ec126690 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e000d150 comp rx=0 tx=0).stop 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 msgr2=0x7fc0ec126690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec1262b0 0x7fc0ec126690 secure :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e000d150 comp rx=0 tx=0).stop 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0eb7fe640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc0d803e550 0x7fc0d8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0eb7fe640 10 monclient: get_auth_request con 0x7fc0d803e550 auth_method 0 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0eb7fe640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc0d803e550 0x7fc0d8040a10 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec127a10 tx=0x7fc0dc006d10 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.483+0000 7fc0e8ff9640 10 client.34594.objecter ms_handle_connect 0x7fc0d803e550 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 0x7fc0ec122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc0ec06df80 con 0x7fc0ec126bd0 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 0x7fc0ec122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient: get_auth_request con 0x7fc0ec126bd0 auth_method 0 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 10 monclient(hunting): handle_auth_done global_id 24126 payload 824 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 1 -- 172.21.15.105:0/2100833437 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc0ec0a6cc0 con 0x7fc0ec126bd0 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 client.34594.objecter ms_handle_connect 0x7fc0ec126bd0 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 client.34594.objecter resend_mon_ops 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0ebfff640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 0x7fc0ec122530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e0005800 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0e0007750 con 0x7fc0ec126bd0 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 monclient: dump: 2024-09-17T16:33:22.514 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9410 (secure 0 0 0) 0x7fc0e00072e0 con 0x7fc0ec126bd0 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 msgr2=0x7fc0ec122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 0x7fc0ec122530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e0005800 comp rx=0 tx=0).stop 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 msgr2=0x7fc0ec122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0e8ff9640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc0ec126bd0 0x7fc0ec122530 secure :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005890 tx=0x7fc0e0005800 comp rx=0 tx=0).stop 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:22.516 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 48, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 350130, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: 7 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:22.520 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.521 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:22.522 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.525 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.528 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:22.532 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:22.534 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.535 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.536 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.538 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.539 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:22.540 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 librados: watch_flush enter 2024-09-17T16:33:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 librados: watch_flush exit 2024-09-17T16:33:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 20 client.34594.objecter shutdown clearing up homeless session... 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 client.34594.objecter successfully canceled tick 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc0d803e550 msgr2=0x7fc0d8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc0d803e550 0x7fc0d8040a10 secure :-1 s=READY pgs=3458 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec127a10 tx=0x7fc0dc006d10 comp rx=0 tx=0).stop 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 10 monclient: shutdown 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f0b3b640 1 -- 172.21.15.105:0/2100833437 reap_dead start 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 msgr2=0x7fc0ec1d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40ac000 tx=0x7fc0e40195e0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 shutdown_connections 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 --2- 172.21.15.105:0/2100833437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc0ec0a73d0 0x7fc0ec1d1a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 >> 172.21.15.105:0/2100833437 conn(0x7fc0ec068ba0 msgr2=0x7fc0ec09fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 shutdown_connections 2024-09-17T16:33:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 -- 172.21.15.105:0/2100833437 wait complete. 2024-09-17T16:33:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.491+0000 7fc0f259e640 1 librados: shutdown 2024-09-17T16:33:22.543 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-17T16:33:22.543 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-09-17T16:33:22.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:33:22.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: get_monmap_and_config 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: monmap: 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:22.660241+0000 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:22.660241+0000 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.663 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:22.664 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 Processor -- start 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 -- start start 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: init 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient: _add_conns ranks=[4,2,3,8,6,7,5,0,1,9,10] 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 0x7f98ac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-c con 0x7f98ac0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-b con 0x7f98ac0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 0x7f98ac139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-g con 0x7f98ac1396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac0a3fc0 con 0x7f98ac0a4910 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac0a63c0 con 0x7f98ac1396a0 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f98ac0a7740 con 0x7f98ac0a68e0 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98b224e640 10 monclient(hunting): authenticate will time out at 1285.926003s 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98ab7fe640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 msgr2=0x7f98ac0a6cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98ab7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 0x7f98ac0a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98abfff640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 msgr2=0x7f98ac139ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:41088/0 (socket says 172.21.15.105:41088) 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 -- 172.21.15.105:0/3553537324 learned_addr learned my addr 172.21.15.105:0/3553537324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): get_auth_request con 0x7f98ac0a4910 auth_method 0 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98abfff640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 0x7f98ac139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient(hunting): handle_auth_done global_id 44372 payload 274 2024-09-17T16:33:22.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 -- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 msgr2=0x7f98ac0a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 0x7f98ac0a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 -- 172.21.15.105:0/3553537324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 msgr2=0x7f98ac139ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 0x7f98ac139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 monclient: found mon.noname-b 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:33:22.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.659+0000 7f98aaffd640 1 -- 172.21.15.105:0/3553537324 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac06ec10 con 0x7f98ac0a4910 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aaffd640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098d0 tx=0x7f98a001cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa3e672444effc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 1 -- 172.21.15.105:0/3553537324 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a00b0070 con 0x7f98ac0a4910 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: dump: 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.677 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 1 -- 172.21.15.105:0/3553537324 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a00aa810 con 0x7f98ac0a4910 2024-09-17T16:33:22.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 5 monclient: authenticate success, global_id 44372 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 4 set_mon_vals no callback set 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 1 -- 172.21.15.105:0/3553537324 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a00aaa40 con 0x7f98ac0a4910 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aa7fc640 10 monclient: dump: 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.679 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: shutdown 2024-09-17T16:33:22.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 msgr2=0x7f98ac0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098d0 tx=0x7f98a001cee0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/3553537324 shutdown_connections 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98ac1396a0 0x7f98ac139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98ac0a4910 0x7f98ac0a7200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- 172.21.15.105:0/3553537324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f98ac0a68e0 0x7f98ac0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/3553537324 >> 172.21.15.105:0/3553537324 conn(0x7f98ac068ba0 msgr2=0x7f98ac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/3553537324 shutdown_connections 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/3553537324 wait complete. 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: monmap: 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 librados: starting msgr at 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 librados: starting objecter 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 Processor -- start 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- start start 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 librados: setting wanted keys 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 librados: calling monclient init 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: init 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _add_conns ranks=[0,1,7,3,8,6,2,4,5] 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 0x7f98ac122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-a con 0x7f98ac0a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac0a68e0 0x7f98ac1d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-f con 0x7f98ac0a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98ac1396a0 0x7f98ac13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): picked mon.noname-i con 0x7f98ac1396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac143320 con 0x7f98ac0a4910 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac06e630 con 0x7f98ac0a68e0 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f98ac1434a0 con 0x7f98ac1396a0 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient(hunting): authenticate will time out at 1285.931031s 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 0x7f98ac122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 0x7f98ac122570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47364/0 (socket says 172.21.15.105:47364) 2024-09-17T16:33:22.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 -- 172.21.15.105:0/178193956 learned_addr learned my addr 172.21.15.105:0/178193956 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): get_auth_request con 0x7f98ac0a4910 auth_method 0 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aaffd640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac0a68e0 msgr2=0x7f98ac1d1a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98aaffd640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac0a68e0 0x7f98ac1d1a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98abfff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98ac1396a0 0x7f98ac13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient(hunting): handle_auth_done global_id 35064 payload 824 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98ac1396a0 msgr2=0x7f98ac13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98ac1396a0 0x7f98ac13d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac0a68e0 msgr2=0x7f98ac1d1a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98ac0a68e0 0x7f98ac1d1a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 monclient: found mon.noname-a 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac123a90 con 0x7f98ac0a4910 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98ab7fe640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 0x7f98ac122570 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b9e0 tx=0x7f98a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 client.?.objecter ms_handle_connect 0x7f98ac0a4910 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a400ce10 con 0x7f98ac0a4910 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: dump: 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.687 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a4010070 con 0x7f98ac0a4910 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a40158e0 con 0x7f98ac0a4910 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 5 monclient: authenticate success, global_id 35064 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _renew_subs 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:22.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac123d70 con 0x7f98ac0a4910 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98a9ffb640 4 set_mon_vals no callback set 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f9893fff640 10 monclient: dump: 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.689 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 client.35064.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _renew_subs 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac0aa230 con 0x7f98ac0a4910 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.663+0000 7f98b224e640 1 librados: init done 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98b224e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98b224e640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 0x7f98ac02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f98ac0a9de0 con 0x7f98ac1d1f70 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f98a4015470 con 0x7f98ac0a4910 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 0x7f98ac02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient: get_auth_request con 0x7f98ac1d1f70 auth_method 0 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f989803e480 0x7f9898040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 7069+0+0 (secure 0 0 0) 0x7f98a4050310 con 0x7f98ac0a4910 2024-09-17T16:33:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 client.35064.objecter ms_dispatch 0x7f98ac000ba0 osd_map(222..222 src has 1..222) 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 3 client.35064.objecter handle_osd_map got epochs [222,222] > 0 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 3 client.35064.objecter handle_osd_map decoding full epoch 222 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 20 client.35064.objecter dump_active .. 0 homeless 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98abfff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f989803e480 0x7f9898040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98abfff640 10 monclient: get_auth_request con 0x7f989803e480 auth_method 0 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 10 monclient(hunting): handle_auth_done global_id 19715 payload 824 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98ac13ebc0 con 0x7f98ac1d1f70 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 client.35064.objecter ms_handle_connect 0x7f98ac1d1f70 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 client.35064.objecter resend_mon_ops 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98aaffd640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 0x7f98ac02d450 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098a0 tx=0x7f98a001acc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 client.35064.objecter ms_handle_connect 0x7f989803e480 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f98abfff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f989803e480 0x7f9898040940 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7f989c006fd0 tx=0x7f989c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a00ba070 con 0x7f98ac1d1f70 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 monclient: dump: 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98a00aac20 con 0x7f98ac1d1f70 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 msgr2=0x7f98ac02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 0x7f98ac02d450 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098a0 tx=0x7f98a001acc0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 msgr2=0x7f98ac02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.667+0000 7f9893fff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac1d1f70 0x7f98ac02d450 secure :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098a0 tx=0x7f98a001acc0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98b224e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:22.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98b224e640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 0x7f98ac02dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98b224e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f98ac051e80 con 0x7f98ac02d990 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 0x7f98ac02dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient: get_auth_request con 0x7f98ac02d990 auth_method 0 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 10 monclient(hunting): handle_auth_done global_id 19724 payload 824 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 1 -- 172.21.15.105:0/178193956 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98ac139ae0 con 0x7f98ac02d990 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f98aaffd640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 0x7f98ac02dda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f98a001c2b0 tx=0x7f98a0019220 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 client.35064.objecter ms_handle_connect 0x7f98ac02d990 2024-09-17T16:33:22.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 client.35064.objecter resend_mon_ops 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98a00ba050 con 0x7f98ac02d990 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 monclient: dump: 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.698 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9412 (secure 0 0 0) 0x7f98a00bb040 con 0x7f98ac02d990 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 msgr2=0x7f98ac02dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 0x7f98ac02dda0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f98a001c2b0 tx=0x7f98a0019220 comp rx=0 tx=0).stop 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 msgr2=0x7f98ac02dda0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.675+0000 7f9893fff640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f98ac02d990 0x7f98ac02dda0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 48, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 350279, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:22.700 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: 7 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.701 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.702 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.703 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.708 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:22.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.712 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.713 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.714 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.715 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.718 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.719 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.720 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.721 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:22.722 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 10 librados: watch_flush enter 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 10 librados: watch_flush exit 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 20 client.35064.objecter shutdown clearing up homeless session... 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 10 client.35064.objecter successfully canceled tick 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f989803e480 msgr2=0x7f9898040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f989803e480 0x7f9898040940 secure :-1 s=READY pgs=3459 cs=0 l=1 rev1=1 crypto rx=0x7f989c006fd0 tx=0x7f989c008040 comp rx=0 tx=0).stop 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 10 monclient: shutdown 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 msgr2=0x7f98ac122570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 --2- 172.21.15.105:0/178193956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98ac0a4910 0x7f98ac122570 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b9e0 tx=0x7f98a400bea0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98abfff640 1 -- 172.21.15.105:0/178193956 reap_dead start 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98abfff640 1 -- 172.21.15.105:0/178193956 reap_dead start 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 shutdown_connections 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 >> 172.21.15.105:0/178193956 conn(0x7f98ac068ba0 msgr2=0x7f98ac0aa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 shutdown_connections 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 -- 172.21.15.105:0/178193956 wait complete. 2024-09-17T16:33:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.679+0000 7f98b224e640 1 librados: shutdown 2024-09-17T16:33:22.725 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-17T16:33:22.725 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon i 2024-09-17T16:33:22.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: get_monmap_and_config 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: monmap: 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:33:22.837389+0000 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:33:22.837389+0000 2024-09-17T16:33:22.840 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:33:22.841 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 Processor -- start 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 -- start start 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: init 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient: _add_conns ranks=[0,6,9,8,10,2,3,4,5,1,7] 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f15a40a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 0x7f15a413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-d con 0x7f15a409e9e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f15a40a5240 0x7f15a4140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-a con 0x7f15a40a5240 addr v2:172.21.15.105:3300/0 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a409dd20 con 0x7f15a40a4750 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f15a406c470 con 0x7f15a40a5240 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f15a409e000 con 0x7f15a409e9e0 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a975e640 10 monclient(hunting): authenticate will time out at 1286.103476s 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.835+0000 7f15a2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f15a40a4750 0x7f15a40a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48106/0 (socket says 172.21.15.105:48106) 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a2ffd640 1 -- 172.21.15.105:0/2653283484 learned_addr learned my addr 172.21.15.105:0/2653283484 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 --2- 172.21.15.105:0/2653283484 >> v2:172.21.15.105:3300/0 conn(0x7f15a40a5240 0x7f15a4140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): get_auth_request con 0x7f15a40a5240 auth_method 0 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a27fc640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 0x7f15a413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient(hunting): handle_auth_done global_id 35073 payload 274 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 -- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 msgr2=0x7f15a413e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 0x7f15a413e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 -- 172.21.15.105:0/2653283484 >> v1:172.21.15.105:6789/0 conn(0x7f15a40a4750 legacy=0x7f15a40a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 monclient: found mon.noname-a 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:33:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a37fe640 1 -- 172.21.15.105:0/2653283484 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a409e5c0 con 0x7f15a40a5240 2024-09-17T16:33:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a27fc640 10 monclient: get_auth_request con 0x7f15a409e9e0 auth_method 0 2024-09-17T16:33:22.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.839+0000 7f15a27fc640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 0x7f15a413e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: seed 29504 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (194 ms) 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-17T16:33:22.849 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10257 ms) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (45 ms) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10496 ms total) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (6944 ms) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8083 ms) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10017 ms) 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [3] 2024-09-17T16:33:22.850 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [4,3] 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [5,4,3] 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting some heads 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: promoting from clones for snap 3 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrubs... 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (50028 ms) 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (29027 ms) 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-17T16:33:22.851 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7927 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (7991 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8001 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10026 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8984 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10027 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13025 ms) 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-17T16:33:22.852 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7974 ms) 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8034 ms) 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (30963 ms) 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8013 ms) 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-17T16:33:22.853 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (22306 ms) 2024-09-17T16:33:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a37fe640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40a5240 0x7f15a4140f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f159c009d90 tx=0x7f159c01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a8ad7b518967ae server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 1 -- 172.21.15.105:0/2653283484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f159c0b2070 con 0x7f15a40a5240 2024-09-17T16:33:22.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: dump: 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 5 monclient: authenticate success, global_id 35073 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 1 -- 172.21.15.105:0/2653283484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c0a55c0 con 0x7f15a40a5240 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 10 monclient: get_monmap_and_config success 2024-09-17T16:33:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 4 set_mon_vals no callback set 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 1 -- 172.21.15.105:0/2653283484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f159c0a5c60 con 0x7f15a40a5240 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a1ffb640 10 monclient: dump: 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 10 monclient: shutdown 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 1 -- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40a5240 msgr2=0x7f15a4140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.847+0000 7f15a975e640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40a5240 0x7f15a4140f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f159c009d90 tx=0x7f159c01c830 comp rx=0 tx=0).stop 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 -- 172.21.15.105:0/2653283484 shutdown_connections 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a40a5240 0x7f15a4140f40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 --2- 172.21.15.105:0/2653283484 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f15a409e9e0 0x7f15a413e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 -- 172.21.15.105:0/2653283484 >> 172.21.15.105:0/2653283484 conn(0x7f15a40a5840 msgr2=0x7f15a40a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 -- 172.21.15.105:0/2653283484 shutdown_connections 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 1 -- 172.21.15.105:0/2653283484 wait complete. 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 10 monclient: build_initial_monmap 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.851+0000 7f15a975e640 10 monclient: monmap: 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:33:22.860 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 librados: starting msgr at 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 librados: starting objecter 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 Processor -- start 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 -- start start 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 librados: setting wanted keys 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 librados: calling monclient init 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient: init 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient: _reopen_session rank -1 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient: _add_conns ranks=[8,5,3,0,4,2,6,7,1] 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15a409e9e0 0x7f15a4126320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-e con 0x7f15a409e9e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 0x7f15a4126860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-h con 0x7f15a40a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15a40a5240 0x7f15a4126da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): picked mon.noname-g con 0x7f15a40a5240 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f15a4143e20 con 0x7f15a40a5240 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f15a4143ca0 con 0x7f15a40a4750 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f15a4143fa0 con 0x7f15a409e9e0 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): _renew_subs 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a975e640 10 monclient(hunting): authenticate will time out at 1286.121293s 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a2ffd640 1 -- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15a409e9e0 msgr2=0x7f15a4126320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3304/0 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a2ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15a409e9e0 0x7f15a4126320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a37fe640 1 -- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15a40a5240 msgr2=0x7f15a4126da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3301/0 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a37fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15a40a5240 0x7f15a4126da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:33:22.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 0x7f15a4126860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 0x7f15a4126860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:42172/0 (socket says 172.21.15.105:42172) 2024-09-17T16:33:22.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 1 -- 172.21.15.105:0/2670855011 learned_addr learned my addr 172.21.15.105:0/2670855011 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:33:22.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): get_auth_request con 0x7f15a40a4750 auth_method 0 2024-09-17T16:33:22.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.855+0000 7f15a27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 10 monclient(hunting): handle_auth_done global_id 19733 payload 824 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15a409e9e0 msgr2=0x7f15a4126320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f15a409e9e0 0x7f15a4126320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15a40a5240 msgr2=0x7f15a4126da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15a40a5240 0x7f15a4126da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 10 monclient: _finish_hunting 0 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 monclient: found mon.noname-h 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40a4220 con 0x7f15a40a4750 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f15a27fc640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 0x7f15a4126860 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7c0 tx=0x7f159800cc80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f158b7fe640 10 client.?.objecter ms_handle_connect 0x7f15a40a4750 2024-09-17T16:33:22.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.859+0000 7f158b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:33:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1598007c40 con 0x7f15a40a4750 2024-09-17T16:33:22.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: dump: 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.869 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598007e20 con 0x7f15a40a4750 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 5 monclient: authenticate success, global_id 19733 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 monclient: _renew_subs 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a4129ae0 con 0x7f15a40a4750 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:33:22.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1598011860 con 0x7f15a40a4750 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 monclient: dump: 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.871 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a17fa640 4 set_mon_vals no callback set 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 client.19733.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 monclient: _renew_subs 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a40a2b10 con 0x7f15a40a4750 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 1 librados: init done 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 0x7f15a402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f15a4127d40 con 0x7f15a41d1670 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 0x7f15a402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient: get_auth_request con 0x7f15a41d1670 auth_method 0 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f15980113f0 con 0x7f15a40a4750 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159003e550 0x7f1590040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(223..223 src has 1..223) ==== 6684+0+0 (secure 0 0 0) 0x7f15980503c0 con 0x7f15a40a4750 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 10 client.19733.objecter ms_dispatch 0x7f15a4000ba0 osd_map(223..223 src has 1..223) 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 3 client.19733.objecter handle_osd_map got epochs [223,223] > 0 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.863+0000 7f158b7fe640 3 client.19733.objecter handle_osd_map decoding full epoch 223 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 20 client.19733.objecter dump_active .. 0 homeless 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a37fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159003e550 0x7f1590040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a37fe640 10 monclient: get_auth_request con 0x7f159003e550 auth_method 0 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_done global_id 34603 payload 824 2024-09-17T16:33:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a2ffd640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15a4129230 con 0x7f15a41d1670 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 client.19733.objecter ms_handle_connect 0x7f15a41d1670 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a2ffd640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 0x7f15a402d450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1594004880 tx=0x7f159400c900 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 client.19733.objecter resend_mon_ops 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1594017070 con 0x7f15a41d1670 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 monclient: dump: 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 client.19733.objecter ms_handle_connect 0x7f159003e550 2024-09-17T16:33:22.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f15a37fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159003e550 0x7f1590040a10 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7f159c0af000 tx=0x7f159c0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1594005af0 con 0x7f15a41d1670 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 msgr2=0x7f15a402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 0x7f15a402d450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1594004880 tx=0x7f159400c900 comp rx=0 tx=0).stop 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 msgr2=0x7f15a402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.867+0000 7f158b7fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1670 0x7f15a402d450 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1594004880 tx=0x7f159400c900 comp rx=0 tx=0).stop 2024-09-17T16:33:22.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:33:22.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 0x7f15a402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:33:22.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f15a4051e80 con 0x7f15a41d1a50 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 0x7f15a402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient: get_auth_request con 0x7f15a41d1a50 auth_method 0 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 10 monclient(hunting): handle_auth_done global_id 34612 payload 824 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 1 -- 172.21.15.105:0/2670855011 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f15a40a4b30 con 0x7f15a41d1a50 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a2ffd640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 0x7f15a402d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1594014000 tx=0x7f1594001130 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 client.19733.objecter ms_handle_connect 0x7f15a41d1a50 2024-09-17T16:33:22.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 client.19733.objecter resend_mon_ops 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1594017040 con 0x7f15a41d1a50 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 monclient: dump: 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:33:22.883 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:33:22.884 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9577 (secure 0 0 0) 0x7f1594022410 con 0x7f15a41d1a50 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 msgr2=0x7f15a402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 0x7f15a402d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1594014000 tx=0x7f1594001130 comp rx=0 tx=0).stop 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 msgr2=0x7f15a402d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f158b7fe640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f15a41d1a50 0x7f15a402d990 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1594014000 tx=0x7f1594001130 comp rx=0 tx=0).stop 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.885 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 48, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 193677, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: 7 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:33:22.886 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.887 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.888 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:33:22.889 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:33:22.890 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.892 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.893 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:33:22.894 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.896 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.897 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.898 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:33:22.899 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.900 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:33:22.901 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:33:22.903 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:33:22.904 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.905 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.906 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.907 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:33:22.908 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:33:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 librados: watch_flush enter 2024-09-17T16:33:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 librados: watch_flush exit 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 20 client.19733.objecter shutdown clearing up homeless session... 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 client.19733.objecter successfully canceled tick 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159003e550 msgr2=0x7f1590040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f159003e550 0x7f1590040a10 secure :-1 s=READY pgs=3460 cs=0 l=1 rev1=1 crypto rx=0x7f159c0af000 tx=0x7f159c0a7070 comp rx=0 tx=0).stop 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 10 monclient: shutdown 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 msgr2=0x7f15a4126860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 --2- 172.21.15.105:0/2670855011 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f15a40a4750 0x7f15a4126860 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7c0 tx=0x7f159800cc80 comp rx=0 tx=0).stop 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a37fe640 1 -- 172.21.15.105:0/2670855011 reap_dead start 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a37fe640 1 -- 172.21.15.105:0/2670855011 reap_dead start 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 shutdown_connections 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 >> 172.21.15.105:0/2670855011 conn(0x7f15a40a5840 msgr2=0x7f15a40a11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:33:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 shutdown_connections 2024-09-17T16:33:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 -- 172.21.15.105:0/2670855011 wait complete. 2024-09-17T16:33:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:33:22.875+0000 7f15a975e640 1 librados: shutdown 2024-09-17T16:33:22.912 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-17T16:33:22.912 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: ok, hit_set contains 267:602f83fe:::foo:head 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (8342 ms) 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg_num = 32 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 0 ls 1726590818,0 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 1 ls 1726590818,0 2024-09-17T16:33:37.884 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 2 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 3 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 4 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 5 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 6 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 7 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 8 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 9 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 10 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 11 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 12 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 13 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 14 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 15 ls 1726590818,0 2024-09-17T16:33:37.885 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 16 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 17 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 18 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 19 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 20 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 21 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 22 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 23 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 24 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 25 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 26 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 27 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 28 ls 1726590818,0 2024-09-17T16:33:37.886 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 29 ls 1726590818,0 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 30 ls 1726590818,0 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg 31 ls 1726590818,0 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: pg_num = 32 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6cac518f:::0:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:02547ec2:::1:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f905c69b:::2:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:cfc208b3:::3:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d83876eb:::4:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b29083e3:::5:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c4fdafeb:::6:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:5c6b0b28:::7:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:bd63b0f1:::8:head 2024-09-17T16:33:37.887 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e960b815:::9:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:52ea6a34:::10:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:89d3ae78:::11:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:de5d7c5f:::12:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:566253c9:::13:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:62a1935d:::14:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:863748b0:::15:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3958e169:::16:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4d4dabf9:::17:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:8391935d:::18:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:28883081:::19:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:69259c59:::20:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4bdb80b7:::21:head 2024-09-17T16:33:37.888 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a11c5d71:::22:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:271af37b:::23:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:95b121be:::24:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:58d1031b:::25:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:0a050783:::26:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c709704c:::27:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:cbe56eaf:::28:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:86b4b162:::29:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:70d89383:::30:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:dd450c7c:::31:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6d5729b1:::32:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c388f3fb:::33:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:56cfea31:::34:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9dbc1bf7:::35:head 2024-09-17T16:33:37.889 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:40b74ccd:::36:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4d5aaf42:::37:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:920f362c:::38:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6cc53222:::39:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9cad833f:::40:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:1ea84d41:::41:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c4480ef6:::42:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a694361e:::43:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d1bd33e9:::44:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ddc2cd5d:::45:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:2b782207:::46:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7b187fca:::47:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:90ecdf6f:::48:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a5ed95fe:::49:head 2024-09-17T16:33:37.890 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ea0eaa55:::50:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f33ef17b:::51:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a0d1b2f6:::52:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:60c5229e:::53:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:edcbc575:::54:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:102cf253:::55:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:efb7fb0b:::56:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:50d0a326:::57:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d4dc5daf:::58:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3a130462:::59:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ec87ed71:::60:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d5bc9454:::61:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3ddfe313:::62:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7c2816b9:::63:head 2024-09-17T16:33:37.891 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:47e00e4d:::64:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c6410c18:::65:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b48ed237:::66:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:cd63ad31:::67:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b179e92b:::68:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:0d9f741a:::69:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6d3352ae:::70:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c6d5c19e:::71:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:bc4729c3:::72:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:77e930b9:::73:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:0abeecfd:::74:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b7c37e15:::75:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b6378398:::76:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:02bd68de:::77:head 2024-09-17T16:33:37.892 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:cc795d2d:::78:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:630d4fea:::79:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e0d29ef5:::80:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:fd6f13d2:::81:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:606461d5:::82:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:eadbdc43:::83:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:8761d0bb:::84:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9ef0186f:::85:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e0d41294:::86:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:961de695:::87:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:1423148f:::88:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:633a8fa2:::89:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a8653809:::90:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3dac8b33:::91:head 2024-09-17T16:33:37.893 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:35aad435:::92:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f6dcc343:::93:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:dbbdad87:::94:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:1cb48ce0:::95:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:03cd461c:::96:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:17a4ea99:::97:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9993c9a7:::98:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6394211c:::99:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:94c7ae57:::100:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6fdee5bb:::101:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9a477fd1:::102:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:eb850916:::103:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:affc56b9:::104:head 2024-09-17T16:33:37.894 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b42dc814:::105:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f319f8f0:::106:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9a40b9de:::107:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:8b524f28:::108:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e3de589f:::109:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:90f90a5b:::110:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a7b4f1d7:::111:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:af51766e:::112:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b6f90bd1:::113:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e0261208:::114:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c9569ef7:::115:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:61bebe50:::116:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:fe93412b:::117:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d3d38bee:::118:head 2024-09-17T16:33:37.895 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3100ba0c:::119:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d0560ada:::120:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f0ea8b35:::121:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:766f231a:::122:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a07a2582:::123:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:bd7c6b3a:::124:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:fb2ddaff:::125:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4408e1fe:::126:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ee1df7a7:::127:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c3002909:::128:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4f48ffa9:::129:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:edf38733:::130:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c08425c0:::131:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:5f902d98:::132:head 2024-09-17T16:33:37.896 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:41ea2c93:::133:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:813cee13:::134:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:0131818d:::135:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:26ba5a85:::136:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:381b8a5a:::137:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:28797e47:::138:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:bfca7f22:::139:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:36807075:::140:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:80b03975:::141:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:5c15709b:::142:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f39ea15e:::143:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ea992956:::144:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:48887b1c:::145:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:9f24a9dd:::146:head 2024-09-17T16:33:37.897 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:987f100b:::147:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d2dd3581:::148:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7fed1808:::149:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c80b70e9:::150:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:85ed90f9:::151:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:36428b24:::152:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d044c34a:::153:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7c18bf58:::154:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d1c21232:::155:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a7a3c575:::156:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:87da0633:::157:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d5ac3822:::158:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3f20522d:::159:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6ca26563:::160:head 2024-09-17T16:33:37.898 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:532ce135:::161:head 2024-09-17T16:33:37.899 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:c78863e6:::162:head 2024-09-17T16:34:14.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7462ddf6:io.RoundTripAppendPP (3008 ms) 2024-09-17T16:34:14.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-17T16:34:14.185 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2968 ms) 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3017 ms) 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3016 ms) 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: setting pool EIO 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: max_success 99, min_failed 100 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (3975 ms) 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-17T16:34:14.186 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3016 ms) 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (154861 ms total) 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2997 ms) 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9050 ms) 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14941 ms) 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-17T16:34:14.187 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3048 ms) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30036 ms total) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3039 ms) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3039 ms total) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13929 ms) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (21134 ms) 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-17T16:34:14.188 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (10913 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (6980 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2986 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (20620 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7035 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7009 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (6992 ms) 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-17T16:34:14.189 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7090 ms) 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (6924 ms) 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6973 ms) 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-17T16:34:14.190 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (6993 ms) 2024-09-17T16:34:33.285 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFul::163:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:5d165639:::164:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f43765fc:::165:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b4c720e9:::166:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e694b040:::167:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:afa38db2:::168:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:77ba9f53:::169:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:87495034:::170:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7c96bf0e:::171:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:dbe346cc:::172:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:e943ec24:::173:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f97a9c0c:::174:head 2024-09-17T16:34:33.286 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:6f26e74d:::175:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4f95e106:::176:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:0e6f2f8f:::177:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:05db05f1:::178:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:38a78d66:::179:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:d095610b:::180:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:a1a9d709:::181:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:1e5d39db:::182:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:f7df4fb9:::183:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:03a7f161:::184:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ba70721e:::185:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:28e5662d:::186:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:973d52de:::187:head 2024-09-17T16:34:33.287 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:4303eb1c:::188:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:b990b48e:::189:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:29b8165b:::190:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:3547f197:::191:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:7e260936:::192:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:1abec7b1:::193:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:10fdda93:::194:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:15817eea:::195:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:770bab57:::196:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:ed9e13e7:::197:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:71471a8f:::198:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: checking for 269:10fb1d02:::199:head 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (9030 ms) 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-17T16:34:33.288 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first is 1726590828 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,1726590836,1726590837,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,1726590836,1726590837,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590828,1726590830,1726590831,1726590833,1726590834,1726590836,1726590837,0 2024-09-17T16:34:33.289 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726590831,1726590833,1726590834,1726590836,1726590837,1726590839,1726590840,0 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first now 1726590831, trimmed 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21034 ms) 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: foo0 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14004 ms) 2024-09-17T16:34:33.290 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-17T16:34:52.913 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-17T16:34:52.913 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-17T16:34:52.913 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-17T16:34:52.916 INFO:tasks.ceph.mon.f:Started 2024-09-17T16:34:52.916 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-17T16:34:52.916 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-17T16:34:52.916 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-17T16:34:52.919 INFO:tasks.ceph.mon.g:Started 2024-09-17T16:34:52.919 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-17T16:34:52.919 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-17T16:34:52.919 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-17T16:34:52.922 INFO:tasks.ceph.mon.e:Started 2024-09-17T16:34:52.922 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-17T16:34:52.922 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T16:34:52.922 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T16:34:52.924 INFO:tasks.ceph.mon.c:Started 2024-09-17T16:34:52.924 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:34:52.925 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:34:53.016 INFO:tasks.ceph.mon.f.smithi160.stderr:2024-09-17T16:34:53.012+0000 7f4a20462e00 -1 mon.f@-1(???).log v0 log_external_backlog rewinding external_log_to from 228 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:34:53.020 INFO:tasks.ceph.mon.g.smithi160.stderr:2024-09-17T16:34:53.016+0000 7fe062140e00 -1 mon.g@-1(???).log v0 log_external_backlog rewinding external_log_to from 228 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:34:53.025 INFO:tasks.ceph.mon.c.smithi105.stderr:2024-09-17T16:34:53.018+0000 7f8159ec0e00 -1 mon.c@-1(???).log v0 log_external_backlog rewinding external_log_to from 228 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 10 monclient: get_monmap_and_config 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 10 monclient: build_initial_monmap 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 10 monclient: monmap: 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:34:53.050552+0000 2024-09-17T16:34:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:34:53.050552+0000 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:34:53.054 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 1 Processor -- start 2024-09-17T16:34:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 1 -- start start 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.046+0000 7f6808c28640 10 monclient: init 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient: _reopen_session rank -1 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient: _add_conns ranks=[3,6,7,0,2,9,8,5,1,10,4] 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-g con 0x7f680409b410 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-d con 0x7f680409b7f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409bbd0 0x7f68040a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-i con 0x7f680409bbd0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f680409f8e0 con 0x7f680409b410 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f68040a1060 con 0x7f680409b7f0 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f68040a1340 con 0x7f680409bbd0 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): _renew_subs 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient(hunting): authenticate will time out at 1376.316775s 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:37746/0 (socket says 172.21.15.105:37746) 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 -- 172.21.15.105:0/3546874680 learned_addr learned my addr 172.21.15.105:0/3546874680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): get_auth_request con 0x7f680409b7f0 auth_method 0 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409bbd0 0x7f68040a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): get_auth_request con 0x7f680409bbd0 auth_method 0 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 10 monclient(hunting): get_auth_request con 0x7f680409b410 auth_method 0 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:34:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 1 -- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 msgr2=0x7f68040a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 1 -- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 msgr2=0x7f68040a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680259c640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient(hunting): handle_auth_done global_id 24180 payload 274 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 -- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409bbd0 msgr2=0x7f68040a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409bbd0 0x7f68040a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 -- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 msgr2=0x7f68040a2300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 monclient: found mon.noname-d 2024-09-17T16:34:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:34:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:34:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 -- 172.21.15.105:0/3546874680 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68040a5c50 con 0x7f680409b7f0 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6801d9b640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f67f800bb20 tx=0x7f67f801e920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e4eba3f341067a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 1 -- 172.21.15.105:0/3546874680 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67f80ad070 con 0x7f680409b7f0 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: dump: 2024-09-17T16:34:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:34:53.068 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: _finish_auth 0 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: _check_auth_tickets 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 1 -- 172.21.15.105:0/3546874680 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f80044a0 con 0x7f680409b7f0 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 1 -- 172.21.15.105:0/3546874680 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67f80a7b80 con 0x7f680409b7f0 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 5 monclient: authenticate success, global_id 24180 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 10 monclient: get_monmap_and_config success 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f6808c28640 4 set_mon_vals no callback set 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:34:53.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.050+0000 7f680159a640 10 monclient: dump: 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:34:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: shutdown 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- 172.21.15.105:0/3546874680 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 msgr2=0x7f68040a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f67f800bb20 tx=0x7f67f801e920 comp rx=0 tx=0).stop 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- 172.21.15.105:0/3546874680 shutdown_connections 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409bbd0 0x7f68040a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f680409b7f0 0x7f68040a3d80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- 172.21.15.105:0/3546874680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f680409b410 0x7f68040a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- 172.21.15.105:0/3546874680 >> 172.21.15.105:0/3546874680 conn(0x7f68040a4e70 msgr2=0x7f68040a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:34:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- 172.21.15.105:0/3546874680 shutdown_connections 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- 172.21.15.105:0/3546874680 wait complete. 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: build_initial_monmap 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: monmap: 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:34:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 librados: starting msgr at 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 librados: starting objecter 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 Processor -- start 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- start start 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 librados: setting wanted keys 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 librados: calling monclient init 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: init 2024-09-17T16:34:53.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: _reopen_session rank -1 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient: _add_conns ranks=[7,5,0,3,2,1,6,8,4] 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409b410 0x7f680412f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-i con 0x7f680409b410 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f680409b7f0 0x7f680412f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-h con 0x7f680409b7f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): picked mon.noname-a con 0x7f680409bbd0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f680413ef40 con 0x7f680409bbd0 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f68040a4c10 con 0x7f680409b7f0 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:34:53.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f680413f0c0 con 0x7f680409b410 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): _renew_subs 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6808c28640 10 monclient(hunting): authenticate will time out at 1376.323563s 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35804/0 (socket says 172.21.15.105:35804) 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 1 -- 172.21.15.105:0/3888560641 learned_addr learned my addr 172.21.15.105:0/3888560641 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): get_auth_request con 0x7f680409bbd0 auth_method 0 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f680259c640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409b410 0x7f680412f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f680409b7f0 0x7f680412f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f680259c640 10 monclient(hunting): get_auth_request con 0x7f680409b410 auth_method 0 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f680259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f680259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f680259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): get_auth_request con 0x7f680409b7f0 auth_method 0 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6802d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.054+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6801d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:34:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 10 monclient(hunting): handle_auth_done global_id 35190 payload 824 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 -- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409b410 msgr2=0x7f680412f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409b410 0x7f680412f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 -- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f680409b7f0 msgr2=0x7f680412f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f680409b7f0 0x7f680412f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 monclient: found mon.noname-a 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 -- 172.21.15.105:0/3888560641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68040a75e0 con 0x7f680409bbd0 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6802d9d640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00efc0 tx=0x7f67fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 client.?.objecter ms_handle_connect 0x7f680409bbd0 2024-09-17T16:34:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67fc018020 con 0x7f680409bbd0 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: dump: 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:34:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 5 monclient: authenticate success, global_id 35190 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc00cd30 con 0x7f680409bbd0 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _renew_subs 2024-09-17T16:34:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6804132aa0 con 0x7f680409bbd0 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67fc009e60 con 0x7f680409bbd0 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6800d99640 4 set_mon_vals no callback set 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 client.35190.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 monclient: dump: 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:34:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _renew_subs 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:34:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6804131860 con 0x7f680409bbd0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 1 librados: init done 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68040a7ed0 con 0x7f680409bbd0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f67fc01b020 con 0x7f680409bbd0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 0x7f67e4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(313..313 src has 1..313) ==== 6541+0+0 (secure 0 0 0) 0x7f67fc051360 con 0x7f680409bbd0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 client.35190.objecter ms_dispatch 0x7f6804000ba0 osd_map(313..313 src has 1..313) 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 3 client.35190.objecter handle_osd_map got epochs [313,313] > 0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 3 client.35190.objecter handle_osd_map decoding full epoch 313 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 20 client.35190.objecter dump_active .. 0 homeless 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 0x7f67e4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:34:53.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 10 monclient: get_auth_request con 0x7f67e403e550 auth_method 0 2024-09-17T16:34:53.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f680259c640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 0x7f67e4040a10 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000ee0 tx=0x7f67f4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:34:53.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.058+0000 7f67f2ffd640 10 client.35190.objecter ms_handle_connect 0x7f67e403e550 2024-09-17T16:34:53.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.062+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f67fc022030 con 0x7f680409bbd0 2024-09-17T16:34:53.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.062+0000 7f67f2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:34:53.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.062+0000 7f67f2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:34:53.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.314+0000 7f6808c28640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:34:53.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.314+0000 7f6808c28640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:34:53.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.314+0000 7f6808c28640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:34:53.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:53.314+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6804133140 con 0x7f680409bbd0 2024-09-17T16:34:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:56.054+0000 7f67f1ffb640 10 monclient: tick 2024-09-17T16:34:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:56.054+0000 7f67f1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:34:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:56.054+0000 7f67f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:34:56.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:56.054+0000 7f67f1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:34:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:57.058+0000 7f67f1ffb640 10 monclient: tick 2024-09-17T16:34:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:57.058+0000 7f67f1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:34:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:57.058+0000 7f67f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:34:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.058+0000 7f67f1ffb640 10 monclient: tick 2024-09-17T16:34:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.058+0000 7f67f1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:34:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.058+0000 7f67f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:34:58.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.062+0000 7f67f37fe640 10 client.35190.objecter tick 2024-09-17T16:34:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.326+0000 7f67f2ffd640 1 -- 172.21.15.105:0/3888560641 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7f68040a75e0 con 0x7f680409bbd0 2024-09-17T16:34:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.326+0000 7f67f2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:34:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.326+0000 7f67f2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:34:58.334 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":50,"quorum":[0,2,5,6,7,8],"quorum_names":["a","b","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:34:58.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 10 librados: watch_flush enter 2024-09-17T16:34:58.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 10 librados: watch_flush exit 2024-09-17T16:34:58.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 20 client.35190.objecter shutdown clearing up homeless session... 2024-09-17T16:34:58.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 10 client.35190.objecter successfully canceled tick 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 msgr2=0x7f67e4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 0x7f67e4040a10 secure :-1 s=READY pgs=3476 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000ee0 tx=0x7f67f4008040 comp rx=0 tx=0).stop 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 10 monclient: shutdown 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 msgr2=0x7f680412fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00efc0 tx=0x7f67fc00c490 comp rx=0 tx=0).stop 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 shutdown_connections 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f67e403e550 0x7f67e4040a10 unknown :-1 s=CLOSED pgs=3476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:58.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f680409bbd0 0x7f680412fd80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f680409b7f0 0x7f680412f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 --2- 172.21.15.105:0/3888560641 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f680409b410 0x7f680412f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 >> 172.21.15.105:0/3888560641 conn(0x7f68040a4e70 msgr2=0x7f680409d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 shutdown_connections 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 -- 172.21.15.105:0/3888560641 wait complete. 2024-09-17T16:34:58.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:34:58.330+0000 7f6808c28640 1 librados: shutdown 2024-09-17T16:35:01.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: get_monmap_and_config 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: build_initial_monmap 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: monmap: 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:01.496199+0000 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:01.496199+0000 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:01.500 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 Processor -- start 2024-09-17T16:35:01.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 -- start start 2024-09-17T16:35:01.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: init 2024-09-17T16:35:01.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:01.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient: _add_conns ranks=[7,9,0,4,2,5,6,1,3,8,10] 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7b00a4750 0x7fe7b00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-i con 0x7fe7b00a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7b00a5070 0x7fe7b0137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-a con 0x7fe7b00a5070 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7b0137a50 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b006c590 con 0x7fe7b0137a50 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b006cf10 con 0x7fe7b00a5070 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe7b00a62d0 con 0x7fe7b00a4750 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b8338640 10 monclient(hunting): authenticate will time out at 1384.761811s 2024-09-17T16:35:01.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7b00a5070 0x7fe7b0137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b68d5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe7b0137a50 0x7fe7b013e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35394/0 (socket says 172.21.15.105:35394) 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b68d5640 1 -- 172.21.15.105:0/1643300475 learned_addr learned my addr 172.21.15.105:0/1643300475 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): get_auth_request con 0x7fe7b00a5070 auth_method 0 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b60d4640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7b00a4750 0x7fe7b00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b60d4640 10 monclient(hunting): get_auth_request con 0x7fe7b00a4750 auth_method 0 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b60d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:01.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b60d4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b60d4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_done global_id 35199 payload 274 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 -- 172.21.15.105:0/1643300475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7b00a4750 msgr2=0x7fe7b00a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7b00a4750 0x7fe7b00a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 -- 172.21.15.105:0/1643300475 >> v1:172.21.15.105:6789/0 conn(0x7fe7b0137a50 legacy=0x7fe7b013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient: _finish_hunting 0 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 monclient: found mon.noname-a 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b58d3640 1 -- 172.21.15.105:0/1643300475 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b00a09d0 con 0x7fe7b00a5070 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b50d2640 1 -- 172.21.15.105:0/1643300475 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701027307 0 0) 0x7fe7b006c590 con 0x7fe7b0137a50 2024-09-17T16:35:01.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.494+0000 7fe7b50d2640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7b00a5070 0x7fe7b0137510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00097c0 tx=0x7fe7a001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759b896a1648b785 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 1 -- 172.21.15.105:0/1643300475 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a00b0070 con 0x7fe7b00a5070 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: dump: 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:01.512 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: _finish_auth 0 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: _check_auth_tickets 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 1 -- 172.21.15.105:0/1643300475 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a00aa810 con 0x7fe7b00a5070 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 5 monclient: authenticate success, global_id 35199 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:01.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 4 set_mon_vals no callback set 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 1 -- 172.21.15.105:0/1643300475 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a00a9300 con 0x7fe7b00a5070 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b50d2640 10 monclient: dump: 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:01.514 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: shutdown 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/1643300475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7b00a5070 msgr2=0x7fe7b0137510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7b00a5070 0x7fe7b0137510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00097c0 tx=0x7fe7a001c360 comp rx=0 tx=0).stop 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/1643300475 shutdown_connections 2024-09-17T16:35:01.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7b00a5070 0x7fe7b0137510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- 172.21.15.105:0/1643300475 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7b00a4750 0x7fe7b00a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/1643300475 >> 172.21.15.105:0/1643300475 conn(0x7fe7b00a94d0 msgr2=0x7fe7b00a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/1643300475 shutdown_connections 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/1643300475 wait complete. 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: build_initial_monmap 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: monmap: 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 librados: starting msgr at 2024-09-17T16:35:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 librados: starting objecter 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 Processor -- start 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- start start 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 librados: setting wanted keys 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 librados: calling monclient init 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: init 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _add_conns ranks=[5,2,1,0,7,6,4,8,3] 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7b00a4750 0x7fe7b011f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-h con 0x7fe7b00a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-b con 0x7fe7b00a5070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0137a50 0x7fe7b011fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): picked mon.noname-f con 0x7fe7b0137a50 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b00a0550 con 0x7fe7b0137a50 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b00a03d0 con 0x7fe7b00a5070 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe7b006c810 con 0x7fe7b00a4750 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient(hunting): authenticate will time out at 1384.766825s 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47058/0 (socket says 172.21.15.105:47058) 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 -- 172.21.15.105:0/2377805624 learned_addr learned my addr 172.21.15.105:0/2377805624 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): get_auth_request con 0x7fe7b00a5070 auth_method 0 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b60d4640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7b00a4750 0x7fe7b011f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b60d4640 10 monclient(hunting): get_auth_request con 0x7fe7b00a4750 auth_method 0 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b60d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b60d4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b60d4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient(hunting): handle_auth_done global_id 44444 payload 824 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 -- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7b00a4750 msgr2=0x7fe7b011f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7b00a4750 0x7fe7b011f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b68d5640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0137a50 0x7fe7b011fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 -- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0137a50 msgr2=0x7fe7b011fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0137a50 0x7fe7b011fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient: _finish_hunting 0 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 monclient: found mon.noname-b 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 -- 172.21.15.105:0/2377805624 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01228c0 con 0x7fe7b00a5070 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b58d3640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ae040 tx=0x7fe7a0019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 client.?.objecter ms_handle_connect 0x7fe7b00a5070 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a00b0040 con 0x7fe7b00a5070 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: dump: 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:01.521 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:01.522 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a00044b0 con 0x7fe7b00a5070 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 5 monclient: authenticate success, global_id 44444 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _renew_subs 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b00a2670 con 0x7fe7b00a5070 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7a00aae60 con 0x7fe7b00a5070 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b48d1640 4 set_mon_vals no callback set 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7a6ffd640 10 monclient: dump: 2024-09-17T16:35:01.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:01.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 client.44444.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _renew_subs 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b0121a50 con 0x7fe7b00a5070 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.498+0000 7fe7b8338640 1 librados: init done 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe7a0004040 con 0x7fe7b00a5070 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b8338640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b8338640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b8338640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b00a4b30 con 0x7fe7b00a5070 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 0x7fe794040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(318..318 src has 1..318) ==== 6274+0+0 (secure 0 0 0) 0x7fe7a00e9bf0 con 0x7fe7b00a5070 2024-09-17T16:35:01.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 10 client.44444.objecter ms_dispatch 0x7fe7b0000ba0 osd_map(318..318 src has 1..318) 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 3 client.44444.objecter handle_osd_map got epochs [318,318] > 0 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 3 client.44444.objecter handle_osd_map decoding full epoch 318 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 20 client.44444.objecter dump_active .. 0 homeless 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b60d4640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 0x7fe794040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b60d4640 10 monclient: get_auth_request con 0x7fe79403e540 auth_method 0 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7a6ffd640 10 client.44444.objecter ms_handle_connect 0x7fe79403e540 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.502+0000 7fe7b60d4640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 0x7fe794040a00 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b9d0 tx=0x7fe7ac008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.506+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7a0016070 con 0x7fe7b00a5070 2024-09-17T16:35:01.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.506+0000 7fe7a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:01.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.506+0000 7fe7a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.738+0000 7fe7b8338640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:35:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.738+0000 7fe7b8338640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.738+0000 7fe7b8338640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.738+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe7b0120560 con 0x7fe7b00a5070 2024-09-17T16:35:01.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7a6ffd640 1 -- 172.21.15.105:0/2377805624 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7fe7a0003dd0 con 0x7fe7b00a5070 2024-09-17T16:35:01.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:01.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7a6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:01.747 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":50,"quorum":[0,2,5,6,7,8],"quorum_names":["a","b","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 10 librados: watch_flush enter 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 10 librados: watch_flush exit 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 20 client.44444.objecter shutdown clearing up homeless session... 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 10 client.44444.objecter successfully canceled tick 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 msgr2=0x7fe794040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 0x7fe794040a00 secure :-1 s=READY pgs=3481 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00b9d0 tx=0x7fe7ac008040 comp rx=0 tx=0).stop 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 10 monclient: shutdown 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 msgr2=0x7fe7b011f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:01.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ae040 tx=0x7fe7a0019100 comp rx=0 tx=0).stop 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 shutdown_connections 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe79403e540 0x7fe794040a00 unknown :-1 s=CLOSED pgs=3481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7b0137a50 0x7fe7b011fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7b00a5070 0x7fe7b011f8d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 --2- 172.21.15.105:0/2377805624 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7b00a4750 0x7fe7b011f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 >> 172.21.15.105:0/2377805624 conn(0x7fe7b00a94d0 msgr2=0x7fe7b013ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 shutdown_connections 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 -- 172.21.15.105:0/2377805624 wait complete. 2024-09-17T16:35:01.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:01.742+0000 7fe7b8338640 1 librados: shutdown 2024-09-17T16:35:04.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:35:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: get_monmap_and_config 2024-09-17T16:35:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: build_initial_monmap 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: monmap: 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:04.893614+0000 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:04.893614+0000 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:04.898 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 Processor -- start 2024-09-17T16:35:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 -- start start 2024-09-17T16:35:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: init 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient: _add_conns ranks=[6,2,3,7,5,4,0,10,8,1,9] 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 0x7fab080a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-d con 0x7fab080a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-b con 0x7fab080a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 0x7fab08139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-g con 0x7fab081396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fab080a3fc0 con 0x7fab080a4910 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fab080a63c0 con 0x7fab081396a0 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fab080a7740 con 0x7fab080a68e0 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab105c2640 10 monclient(hunting): authenticate will time out at 1388.159513s 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 0x7fab080a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47072/0 (socket says 172.21.15.105:47072) 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 1 -- 172.21.15.105:0/1924747088 learned_addr learned my addr 172.21.15.105:0/1924747088 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): get_auth_request con 0x7fab080a4910 auth_method 0 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:04.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): get_auth_request con 0x7fab080a68e0 auth_method 0 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0e35e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:04.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.890+0000 7fab0eb5f640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 0x7fab08139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 10 monclient(hunting): handle_auth_done global_id 44462 payload 274 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 -- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 msgr2=0x7fab080a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 0x7fab080a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 -- 172.21.15.105:0/1924747088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 msgr2=0x7fab08139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 0x7fab08139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 monclient: found mon.noname-b 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 -- 172.21.15.105:0/1924747088 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0806ec10 con 0x7fab080a4910 2024-09-17T16:35:04.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0eb5f640 10 monclient: get_auth_request con 0x7fab081396a0 auth_method 0 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0eb5f640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 0x7fab08139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0e35e640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 0x7fab080a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0db5d640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009eb0 tx=0x7faaf801d350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=381cae88659b09d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 1 -- 172.21.15.105:0/1924747088 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaf80090d0 con 0x7fab080a4910 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: dump: 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.906 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: _finish_auth 0 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: _check_auth_tickets 2024-09-17T16:35:04.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 5 monclient: authenticate success, global_id 44462 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 1 -- 172.21.15.105:0/1924747088 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf80a5a50 con 0x7fab080a4910 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 1 -- 172.21.15.105:0/1924747088 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaf80a5c40 con 0x7fab080a4910 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 4 set_mon_vals no callback set 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:04.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab0d35c640 10 monclient: dump: 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:04.909 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 10 monclient: shutdown 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 -- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 msgr2=0x7fab080a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009eb0 tx=0x7faaf801d350 comp rx=0 tx=0).stop 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 -- 172.21.15.105:0/1924747088 shutdown_connections 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fab081396a0 0x7fab08139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fab080a4910 0x7fab080a7200 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 --2- 172.21.15.105:0/1924747088 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fab080a68e0 0x7fab080a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 -- 172.21.15.105:0/1924747088 >> 172.21.15.105:0/1924747088 conn(0x7fab08068ba0 msgr2=0x7fab08068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:04.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 -- 172.21.15.105:0/1924747088 shutdown_connections 2024-09-17T16:35:04.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 -- 172.21.15.105:0/1924747088 wait complete. 2024-09-17T16:35:04.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 10 monclient: build_initial_monmap 2024-09-17T16:35:04.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 10 monclient: monmap: 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:04.916 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 librados: starting msgr at 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.894+0000 7fab105c2640 1 librados: starting objecter 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 Processor -- start 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- start start 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 librados: setting wanted keys 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 librados: calling monclient init 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: init 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _add_conns ranks=[4,8,1,2,6,3,5,7,0] 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-c con 0x7fab080a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 0x7fab081d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-e con 0x7fab080a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab081396a0 0x7fab0813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): picked mon.noname-f con 0x7fab081396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab08143320 con 0x7fab081396a0 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fab0806e630 con 0x7fab080a4910 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fab081434a0 con 0x7fab080a68e0 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient(hunting): authenticate will time out at 1388.164558s 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 0x7fab081d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40092/0 (socket says 172.21.15.105:40092) 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 -- 172.21.15.105:0/3705145641 learned_addr learned my addr 172.21.15.105:0/3705145641 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 10 monclient(hunting): get_auth_request con 0x7fab080a4910 auth_method 0 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): get_auth_request con 0x7fab080a68e0 auth_method 0 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 msgr2=0x7fab081d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:35:04.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 msgr2=0x7fab081d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0eb5f640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab081396a0 0x7fab0813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0eb5f640 10 monclient(hunting): get_auth_request con 0x7fab081396a0 auth_method 0 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0eb5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0eb5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0eb5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient(hunting): handle_auth_done global_id 54107 payload 824 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 msgr2=0x7fab081d16f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab081396a0 msgr2=0x7fab0813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab081396a0 0x7fab0813d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:04.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 monclient: found mon.noname-e 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 -- 172.21.15.105:0/3705145641 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0813ef60 con 0x7fab080a68e0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0db5d640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 0x7fab081d1c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab080a43e0 tx=0x7faaf8019690 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 client.?.objecter ms_handle_connect 0x7fab080a68e0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaf80bc020 con 0x7fab080a68e0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: dump: 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:04.921 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf8003e10 con 0x7fab080a68e0 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 5 monclient: authenticate success, global_id 54107 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaf80a57f0 con 0x7fab080a68e0 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 monclient: dump: 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:04.923 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0cb5b640 4 set_mon_vals no callback set 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _renew_subs 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:04.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab080ab900 con 0x7fab080a68e0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 client.54107.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _renew_subs 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fab08123cd0 con 0x7fab080a68e0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 librados: init done 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7faaf80bf020 con 0x7fab080a68e0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08139ae0 con 0x7fab080a68e0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 0x7faaf4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(321..321 src has 1..321) ==== 6607+0+0 (secure 0 0 0) 0x7faaf80ead30 con 0x7fab080a68e0 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 client.54107.objecter ms_dispatch 0x7fab08000ba0 osd_map(321..321 src has 1..321) 2024-09-17T16:35:04.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 3 client.54107.objecter handle_osd_map got epochs [321,321] > 0 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 3 client.54107.objecter handle_osd_map decoding full epoch 321 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 20 client.54107.objecter dump_active .. 0 homeless 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 0x7faaf4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 10 monclient: get_auth_request con 0x7faaf403e580 auth_method 0 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab0e35e640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 0x7faaf4040a40 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7faafc009650 tx=0x7faafc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.898+0000 7fab06ffd640 10 client.54107.objecter ms_handle_connect 0x7faaf403e580 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.902+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faaf80c6030 con 0x7fab080a68e0 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.902+0000 7fab06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:04.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:04.902+0000 7fab06ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab105c2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:35:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab105c2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab105c2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:05.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fab0813fac0 con 0x7fab080a68e0 2024-09-17T16:35:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab06ffd640 1 -- 172.21.15.105:0/3705145641 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7faaf8016600 con 0x7fab080a68e0 2024-09-17T16:35:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab06ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:05.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.138+0000 7fab06ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:05.145 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":50,"quorum":[0,2,5,6,7,8],"quorum_names":["a","b","h","d","i","e"],"quorum_leader_name":"h","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:35:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 10 librados: watch_flush enter 2024-09-17T16:35:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 10 librados: watch_flush exit 2024-09-17T16:35:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 20 client.54107.objecter shutdown clearing up homeless session... 2024-09-17T16:35:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 10 client.54107.objecter successfully canceled tick 2024-09-17T16:35:05.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 msgr2=0x7faaf4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 0x7faaf4040a40 secure :-1 s=READY pgs=3483 cs=0 l=1 rev1=1 crypto rx=0x7faafc009650 tx=0x7faafc008040 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 10 monclient: shutdown 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 msgr2=0x7fab081d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 0x7fab081d1c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab080a43e0 tx=0x7faaf8019690 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 shutdown_connections 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faaf403e580 0x7faaf4040a40 unknown :-1 s=CLOSED pgs=3483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab081396a0 0x7fab0813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fab080a68e0 0x7fab081d1c30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 --2- 172.21.15.105:0/3705145641 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fab080a4910 0x7fab081d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 >> 172.21.15.105:0/3705145641 conn(0x7fab08068ba0 msgr2=0x7fab081403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 shutdown_connections 2024-09-17T16:35:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 -- 172.21.15.105:0/3705145641 wait complete. 2024-09-17T16:35:05.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:05.142+0000 7fab105c2640 1 librados: shutdown 2024-09-17T16:35:08.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:35:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 10 monclient: get_monmap_and_config 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 10 monclient: monmap: 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:08.289835+0000 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:08.289835+0000 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:08.294 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 1 Processor -- start 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 1 -- start start 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.286+0000 7fd426416640 10 monclient: init 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient: _add_conns ranks=[8,1,9,0,10,7,3,4,5,6,2] 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): picked mon.noname-e con 0x7fd42009f8e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd4200a46f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd42013b980 0x7fd42013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): picked mon.noname-a con 0x7fd42013b980 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42009f3f0 con 0x7fd4200a46f0 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd4200a1a50 con 0x7fd42013b980 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fd4200a1ca0 con 0x7fd42009f8e0 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient(hunting): authenticate will time out at 1391.556425s 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd42013b980 0x7fd42013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:35372/0 (socket says 172.21.15.105:35372) 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 1 -- 172.21.15.105:0/333179442 learned_addr learned my addr 172.21.15.105:0/333179442 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): get_auth_request con 0x7fd42009f8e0 auth_method 0 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd42013b980 0x7fd42013dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52726/0 (socket says 172.21.15.105:52726) 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): get_auth_request con 0x7fd42013b980 auth_method 0 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient(hunting): handle_auth_done global_id 35217 payload 274 2024-09-17T16:35:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 -- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 msgr2=0x7fd42009b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 -- 172.21.15.105:0/333179442 >> v1:172.21.15.160:6789/0 conn(0x7fd4200a46f0 legacy=0x7fd42009b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 monclient: found mon.noname-a 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 -- 172.21.15.105:0/333179442 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200a27c0 con 0x7fd42013b980 2024-09-17T16:35:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41ffff640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd4249b3640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd42013b980 0x7fd42013dd70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd4180093f0 tx=0x7fd418020a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c70e681be98ef78 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 1 -- 172.21.15.105:0/333179442 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4180b6070 con 0x7fd42013b980 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: dump: 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 1 -- 172.21.15.105:0/333179442 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4180a9a10 con 0x7fd42013b980 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 5 monclient: authenticate success, global_id 35217 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 1 -- 172.21.15.105:0/333179442 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4180a9c40 con 0x7fd42013b980 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd426416640 4 set_mon_vals no callback set 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.290+0000 7fd41f7fe640 10 monclient: dump: 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: shutdown 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd42013b980 msgr2=0x7fd42013dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd42013b980 0x7fd42013dd70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd4180093f0 tx=0x7fd418020a50 comp rx=0 tx=0).stop 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/333179442 shutdown_connections 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd42013b980 0x7fd42013dd70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- 172.21.15.105:0/333179442 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd42009f8e0 0x7fd42009b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/333179442 >> 172.21.15.105:0/333179442 conn(0x7fd4200a83e0 msgr2=0x7fd4200a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/333179442 shutdown_connections 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/333179442 wait complete. 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: monmap: 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 librados: starting msgr at 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 librados: starting objecter 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 Processor -- start 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- start start 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 librados: setting wanted keys 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 librados: calling monclient init 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: init 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _add_conns ranks=[2,1,5,7,6,0,4,8,3] 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): picked mon.noname-b con 0x7fd42009f8e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4200a46f0 0x7fd420071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): picked mon.noname-f con 0x7fd4200a46f0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 0x7fd420071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): picked mon.noname-h con 0x7fd42013b980 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd4200a3980 con 0x7fd4200a46f0 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd4200a3800 con 0x7fd42009f8e0 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd4200a3b00 con 0x7fd42013b980 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient(hunting): authenticate will time out at 1391.561790s 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57174/0 (socket says 172.21.15.105:57174) 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 -- 172.21.15.105:0/2812157428 learned_addr learned my addr 172.21.15.105:0/2812157428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): get_auth_request con 0x7fd42009f8e0 auth_method 0 2024-09-17T16:35:08.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd4249b3640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 0x7fd420071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient(hunting): handle_auth_done global_id 44471 payload 824 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd417fff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4200a46f0 0x7fd420071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 -- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 msgr2=0x7fd420071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 0x7fd420071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 -- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4200a46f0 msgr2=0x7fd420071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4200a46f0 0x7fd420071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 monclient: found mon.noname-b 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 -- 172.21.15.105:0/2812157428 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200a6710 con 0x7fd42009f8e0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd4249b3640 10 monclient: get_auth_request con 0x7fd42013b980 auth_method 0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd4249b3640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 0x7fd420071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41ffff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e9d0 tx=0x7fd40800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 client.?.objecter ms_handle_connect 0x7fd42009f8e0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd408017070 con 0x7fd42009f8e0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: dump: 2024-09-17T16:35:08.314 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.315 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 5 monclient: authenticate success, global_id 44471 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd408013470 con 0x7fd42009f8e0 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd408013a90 con 0x7fd42009f8e0 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41effd640 4 set_mon_vals no callback set 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 10 monclient: dump: 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.316 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.317 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _renew_subs 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4200746f0 con 0x7fd42009f8e0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 client.44471.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _renew_subs 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4200a8ac0 con 0x7fd42009f8e0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 librados: init done 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42009b050 con 0x7fd42009f8e0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.294+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd40801a070 con 0x7fd42009f8e0 2024-09-17T16:35:08.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 0x7fd400040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(324..324 src has 1..324) ==== 6607+0+0 (secure 0 0 0) 0x7fd40800f040 con 0x7fd42009f8e0 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 10 client.44471.objecter ms_dispatch 0x7fd420000ba0 osd_map(324..324 src has 1..324) 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 3 client.44471.objecter handle_osd_map got epochs [324,324] > 0 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 3 client.44471.objecter handle_osd_map decoding full epoch 324 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 20 client.44471.objecter dump_active .. 0 homeless 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd417fff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 0x7fd400040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd417fff640 10 monclient: get_auth_request con 0x7fd40003e580 auth_method 0 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd41d7fa640 10 client.44471.objecter ms_handle_connect 0x7fd40003e580 2024-09-17T16:35:08.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.298+0000 7fd417fff640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 0x7fd400040a40 secure :-1 s=READY pgs=3484 cs=0 l=1 rev1=1 crypto rx=0x7fd410001600 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.302+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd408021070 con 0x7fd42009f8e0 2024-09-17T16:35:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.302+0000 7fd41d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.302+0000 7fd41d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd426416640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd426416640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd426416640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd42006d7a0 con 0x7fd42009f8e0 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd41d7fa640 1 -- 172.21.15.105:0/2812157428 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7fd408011350 con 0x7fd42009f8e0 2024-09-17T16:35:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd41d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.534+0000 7fd41d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:08.542 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 10 librados: watch_flush enter 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 10 librados: watch_flush exit 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 20 client.44471.objecter shutdown clearing up homeless session... 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 10 client.44471.objecter successfully canceled tick 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 msgr2=0x7fd400040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 0x7fd400040a40 secure :-1 s=READY pgs=3484 cs=0 l=1 rev1=1 crypto rx=0x7fd410001600 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 10 monclient: shutdown 2024-09-17T16:35:08.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 msgr2=0x7fd4200710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e9d0 tx=0x7fd40800ee90 comp rx=0 tx=0).stop 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 shutdown_connections 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd40003e580 0x7fd400040a40 unknown :-1 s=CLOSED pgs=3484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd42013b980 0x7fd420071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd4200a46f0 0x7fd420071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 --2- 172.21.15.105:0/2812157428 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd42009f8e0 0x7fd4200710c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 >> 172.21.15.105:0/2812157428 conn(0x7fd4200a83e0 msgr2=0x7fd42013bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 shutdown_connections 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 -- 172.21.15.105:0/2812157428 wait complete. 2024-09-17T16:35:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.538+0000 7fd426416640 1 librados: shutdown 2024-09-17T16:35:08.562 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:35:08.562 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:35:08.562 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: get_monmap_and_config 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: monmap: 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:08.688896+0000 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:08.688896+0000 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:08.692 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:08.693 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 Processor -- start 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 -- start start 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: init 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient: _add_conns ranks=[9,1,0,8,3,7,2,10,4,6,5] 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f230809b510 0x7f230809b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-a con 0x7f230809b510 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:08.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f23080a0540 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23080a4490 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23080696c0 con 0x7f23080a4490 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23080694c0 con 0x7f23080a0540 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f230809fa80 con 0x7f230809b510 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230d1ed640 10 monclient(hunting): authenticate will time out at 1391.954796s 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f230809b510 0x7f230809b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f23080a4490 0x7f230813f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41802/0 (socket says 172.21.15.105:41802) 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f230759e640 1 -- 172.21.15.105:0/2454142485 learned_addr learned my addr 172.21.15.105:0/2454142485 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): get_auth_request con 0x7f230809b510 auth_method 0 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient(hunting): handle_auth_done global_id 35226 payload 274 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 1 -- 172.21.15.105:0/2454142485 >> v1:172.21.15.160:6789/0 conn(0x7f23080a0540 legacy=0x7f23080a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 1 -- 172.21.15.105:0/2454142485 >> v1:172.21.15.105:6789/0 conn(0x7f23080a4490 legacy=0x7f230813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 1 monclient: found mon.noname-a 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2306d9d640 1 -- 172.21.15.105:0/2454142485 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23080a5cb0 con 0x7f230809b510 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2305d9b640 1 -- 172.21.15.105:0/2454142485 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3007507723 0 0) 0x7f23080696c0 con 0x7f23080a4490 2024-09-17T16:35:08.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.686+0000 7f2305d9b640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:08.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2306d9d640 1 --2- 172.21.15.105:0/2454142485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809b510 0x7f230809b8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f230001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb11d39772ab57b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 1 -- 172.21.15.105:0/2454142485 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23000b0070 con 0x7f230809b510 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: dump: 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.705 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 1 -- 172.21.15.105:0/2454142485 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000aa810 con 0x7f230809b510 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 5 monclient: authenticate success, global_id 35226 2024-09-17T16:35:08.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 1 -- 172.21.15.105:0/2454142485 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23000a9300 con 0x7f230809b510 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 4 set_mon_vals no callback set 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f2305d9b640 10 monclient: dump: 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.707 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: shutdown 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- 172.21.15.105:0/2454142485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809b510 msgr2=0x7f230809b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 --2- 172.21.15.105:0/2454142485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809b510 0x7f230809b8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f230001c360 comp rx=0 tx=0).stop 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- 172.21.15.105:0/2454142485 shutdown_connections 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 --2- 172.21.15.105:0/2454142485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809b510 0x7f230809b8f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- 172.21.15.105:0/2454142485 >> 172.21.15.105:0/2454142485 conn(0x7f23080a4e70 msgr2=0x7f23080a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- 172.21.15.105:0/2454142485 shutdown_connections 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- 172.21.15.105:0/2454142485 wait complete. 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: monmap: 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.709 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 librados: starting msgr at 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 librados: starting objecter 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 Processor -- start 2024-09-17T16:35:08.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- start start 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 librados: setting wanted keys 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 librados: calling monclient init 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: init 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient: _add_conns ranks=[1,6,2,8,7,4,5,0,3] 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f230809b510 0x7f230813a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-f con 0x7f230809b510 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 0x7f230813aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-d con 0x7f23080a0540 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23080a4490 0x7f230813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): picked mon.noname-b con 0x7f23080a4490 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2308143670 con 0x7f230809b510 2024-09-17T16:35:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f230809c040 con 0x7f23080a4490 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f23081437f0 con 0x7f23080a0540 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230d1ed640 10 monclient(hunting): authenticate will time out at 1391.959755s 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 0x7f230813aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23080a4490 0x7f230813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 0x7f230813aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42778/0 (socket says 172.21.15.105:42778) 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 1 -- 172.21.15.105:0/1625354508 learned_addr learned my addr 172.21.15.105:0/1625354508 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): get_auth_request con 0x7f23080a0540 auth_method 0 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): get_auth_request con 0x7f23080a4490 auth_method 0 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.690+0000 7f230759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f230809b510 0x7f230813a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): get_auth_request con 0x7f230809b510 auth_method 0 2024-09-17T16:35:08.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 10 monclient(hunting): handle_auth_done global_id 24198 payload 824 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23080a4490 msgr2=0x7f230813af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23080a4490 0x7f230813af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f230809b510 msgr2=0x7f230813a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f230809b510 0x7f230813a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 monclient: found mon.noname-d 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23080a77d0 con 0x7f23080a0540 2024-09-17T16:35:08.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230759e640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23080a4490 0x7f230813af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230659c640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 0x7f230813aa40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c450 tx=0x7f22fc00c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.?.objecter ms_handle_connect 0x7f23080a0540 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc012aa0 con 0x7f23080a0540 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: dump: 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.715 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 5 monclient: authenticate success, global_id 24198 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc017070 con 0x7f23080a0540 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 monclient: _renew_subs 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081402a0 con 0x7f23080a0540 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc012c80 con 0x7f23080a0540 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 client.24198.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230559a640 4 set_mon_vals no callback set 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: dump: 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.717 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 monclient: _renew_subs 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f230813ffc0 con 0x7f23080a0540 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 1 librados: init done 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 0x7f230809c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2308051e80 con 0x7f230809c370 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 0x7f230809c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient: get_auth_request con 0x7f230809c370 auth_method 0 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 10 monclient(hunting): handle_auth_done global_id 35244 payload 824 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f22fc012630 con 0x7f23080a0540 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23080a1500 con 0x7f230809c370 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f2306d9d640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 0x7f230809c750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f230806ed50 tx=0x7f2300019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f22f403e550 0x7f22f4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.24198.objecter ms_handle_connect 0x7f230809c370 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.24198.objecter resend_mon_ops 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(325..325 src has 1..325) ==== 6293+0+0 (secure 0 0 0) 0x7f22fc04fb40 con 0x7f23080a0540 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.24198.objecter ms_dispatch 0x7f2308000ba0 osd_map(325..325 src has 1..325) 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 3 client.24198.objecter handle_osd_map got epochs [325,325] > 0 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 3 client.24198.objecter handle_osd_map decoding full epoch 325 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 20 client.24198.objecter dump_active .. 0 homeless 2024-09-17T16:35:08.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23000b9070 con 0x7f230809c370 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: dump: 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.721 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2300003620 con 0x7f230809c370 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 msgr2=0x7f230809c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 0x7f230809c750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f230806ed50 tx=0x7f2300019100 comp rx=0 tx=0).stop 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 msgr2=0x7f230809c750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809c370 0x7f230809c750 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f230806ed50 tx=0x7f2300019100 comp rx=0 tx=0).stop 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230759e640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f22f403e550 0x7f22f4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230759e640 10 monclient: get_auth_request con 0x7f22f403e550 auth_method 0 2024-09-17T16:35:08.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f22ef7fe640 10 client.24198.objecter ms_handle_connect 0x7f22f403e550 2024-09-17T16:35:08.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.694+0000 7f230759e640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f22f403e550 0x7f22f4040a10 secure :-1 s=READY pgs=3485 cs=0 l=1 rev1=1 crypto rx=0x7f22f8007f40 tx=0x7f22f8007e90 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 0x7f2308072e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23080a5750 con 0x7f230809cc90 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 0x7f2308072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient: get_auth_request con 0x7f230809cc90 auth_method 0 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 10 monclient(hunting): handle_auth_done global_id 35253 payload 824 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 1 -- 172.21.15.105:0/1625354508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f230813fb90 con 0x7f230809cc90 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f2306d9d640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 0x7f2308072e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f23000a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 client.24198.objecter ms_handle_connect 0x7f230809cc90 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 client.24198.objecter resend_mon_ops 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23000b9050 con 0x7f230809cc90 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 monclient: dump: 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f23000b5bc0 con 0x7f230809cc90 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 msgr2=0x7f2308072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 0x7f2308072e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f23000a8000 comp rx=0 tx=0).stop 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 msgr2=0x7f2308072e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f22ef7fe640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230809cc90 0x7f2308072e20 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f23000a8000 comp rx=0 tx=0).stop 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:08.728 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 299505, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.729 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.730 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.731 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:08.732 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:08.733 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.734 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.739 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.744 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:08.746 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:08.747 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.749 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.750 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.751 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:08.752 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:08.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 librados: watch_flush enter 2024-09-17T16:35:08.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 librados: watch_flush exit 2024-09-17T16:35:08.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 20 client.24198.objecter shutdown clearing up homeless session... 2024-09-17T16:35:08.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 client.24198.objecter successfully canceled tick 2024-09-17T16:35:08.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f22f403e550 msgr2=0x7f22f4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f22f403e550 0x7f22f4040a10 secure :-1 s=READY pgs=3485 cs=0 l=1 rev1=1 crypto rx=0x7f22f8007f40 tx=0x7f22f8007e90 comp rx=0 tx=0).stop 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 10 monclient: shutdown 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 msgr2=0x7f230813aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 --2- 172.21.15.105:0/1625354508 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23080a0540 0x7f230813aa40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c450 tx=0x7f22fc00c910 comp rx=0 tx=0).stop 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230759e640 1 -- 172.21.15.105:0/1625354508 reap_dead start 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230759e640 1 -- 172.21.15.105:0/1625354508 reap_dead start 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 shutdown_connections 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 >> 172.21.15.105:0/1625354508 conn(0x7f23080a4e70 msgr2=0x7f23080a6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 shutdown_connections 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 -- 172.21.15.105:0/1625354508 wait complete. 2024-09-17T16:35:08.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.702+0000 7f230d1ed640 1 librados: shutdown 2024-09-17T16:35:08.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:08.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:35:08.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.862+0000 7f1785a82640 10 monclient: get_monmap_and_config 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.862+0000 7f1785a82640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.862+0000 7f1785a82640 10 monclient: monmap: 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:08.867098+0000 2024-09-17T16:35:08.870 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:08.867098+0000 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.871 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:08.872 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:08.872 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 Processor -- start 2024-09-17T16:35:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 -- start start 2024-09-17T16:35:08.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient: init 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient: _add_conns ranks=[1,9,2,5,8,10,4,3,7,0,6] 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f17800a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17800a4910 0x7f178013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-a con 0x7f17800a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a73d0 0x7f1780141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-b con 0x7f17800a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178006ebc0 con 0x7f17800a68e0 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f17800a3fc0 con 0x7f17800a4910 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f17800a63c0 con 0x7f17800a73d0 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient(hunting): authenticate will time out at 1392.132898s 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17800a4910 0x7f178013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f17800a4910 0x7f178013eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52762/0 (socket says 172.21.15.105:52762) 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 -- 172.21.15.105:0/549309126 learned_addr learned my addr 172.21.15.105:0/549309126 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): get_auth_request con 0x7f17800a4910 auth_method 0 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a73d0 0x7f1780141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): get_auth_request con 0x7f17800a73d0 auth_method 0 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient(hunting): handle_auth_done global_id 35262 payload 274 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 -- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a73d0 msgr2=0x7f1780141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a73d0 0x7f1780141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 -- 172.21.15.105:0/549309126 >> v1:172.21.15.160:6789/0 conn(0x7f17800a68e0 legacy=0x7f17800a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 monclient: found mon.noname-a 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 -- 172.21.15.105:0/549309126 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17800a3690 con 0x7f17800a4910 2024-09-17T16:35:08.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1777fff640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a4910 0x7f178013eaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1768009870 tx=0x7f176801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3246afa1344ede68 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 1 -- 172.21.15.105:0/549309126 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680b0070 con 0x7f17800a4910 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: dump: 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.892 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 1 -- 172.21.15.105:0/549309126 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680aa810 con 0x7f17800a4910 2024-09-17T16:35:08.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 1 -- 172.21.15.105:0/549309126 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680a9300 con 0x7f17800a4910 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 5 monclient: authenticate success, global_id 35262 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f1785a82640 4 set_mon_vals no callback set 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.866+0000 7f177effd640 10 monclient: dump: 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.895 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: shutdown 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a4910 msgr2=0x7f178013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a4910 0x7f178013eaf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1768009870 tx=0x7f176801c3f0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/549309126 shutdown_connections 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a73d0 0x7f1780141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- 172.21.15.105:0/549309126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a4910 0x7f178013eaf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/549309126 >> 172.21.15.105:0/549309126 conn(0x7f1780068ba0 msgr2=0x7f1780068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/549309126 shutdown_connections 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/549309126 wait complete. 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: build_initial_monmap 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: monmap: 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:08.897 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:08.898 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 librados: starting msgr at 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 librados: starting objecter 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 Processor -- start 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- start start 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 librados: setting wanted keys 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 librados: calling monclient init 2024-09-17T16:35:08.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: init 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _add_conns ranks=[2,0,4,8,3,5,6,7,1] 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 0x7f17801d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-b con 0x7f17800a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a68e0 0x7f17801d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-a con 0x7f17800a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17800a73d0 0x7f17801d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): picked mon.noname-c con 0x7f17800a73d0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1780143f20 con 0x7f17800a68e0 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1780143da0 con 0x7f17800a4910 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 0x7f17801d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a68e0 0x7f17801d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 0x7f17801d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57202/0 (socket says 172.21.15.105:57202) 2024-09-17T16:35:08.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 1 -- 172.21.15.105:0/1818198926 learned_addr learned my addr 172.21.15.105:0/1818198926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f17801440a0 con 0x7f17800a73d0 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): authenticate will time out at 1392.138337s 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17800a73d0 0x7f17801d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): get_auth_request con 0x7f17800a4910 auth_method 0 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): get_auth_request con 0x7f17800a73d0 auth_method 0 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): get_auth_request con 0x7f17800a68e0 auth_method 0 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient(hunting): handle_auth_done global_id 54103 payload 824 2024-09-17T16:35:08.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 msgr2=0x7f17801d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 0x7f17801d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a68e0 msgr2=0x7f17801d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a68e0 0x7f17801d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 monclient: found mon.noname-c 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17800a2530 con 0x7f17800a73d0 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1777fff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17800a68e0 0x7f17801d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17800a4910 0x7f17801d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177ffff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17800a73d0 0x7f17801d1a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f177800eba0 tx=0x7f177800c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 client.?.objecter ms_handle_connect 0x7f17800a73d0 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:08.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1778018020 con 0x7f17800a73d0 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: dump: 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.905 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778014480 con 0x7f17800a73d0 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1778014ae0 con 0x7f17800a73d0 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 5 monclient: authenticate success, global_id 54103 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _renew_subs 2024-09-17T16:35:08.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1780129b80 con 0x7f17800a73d0 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177e7fc640 4 set_mon_vals no callback set 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177cff9640 10 monclient: dump: 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.907 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 client.54103.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _renew_subs 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f178012a560 con 0x7f17800a73d0 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 librados: init done 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 0x7f17801267c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f178006db40 con 0x7f17801263e0 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 0x7f17801267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient: get_auth_request con 0x7f17801263e0 auth_method 0 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.870+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 10 monclient(hunting): handle_auth_done global_id 44507 payload 824 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17801292d0 con 0x7f17801263e0 2024-09-17T16:35:08.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f177801b020 con 0x7f17800a73d0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 0x7f17801267c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f177000fb10 tx=0x7f17700045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176003e580 0x7f1760040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 client.54103.objecter ms_handle_connect 0x7f17801263e0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 client.54103.objecter resend_mon_ops 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(325..325 src has 1..325) ==== 6293+0+0 (secure 0 0 0) 0x7f1778050b90 con 0x7f17800a73d0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 client.54103.objecter ms_dispatch 0x7f1780000ba0 osd_map(325..325 src has 1..325) 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 3 client.54103.objecter handle_osd_map got epochs [325,325] > 0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 3 client.54103.objecter handle_osd_map decoding full epoch 325 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 20 client.54103.objecter dump_active .. 0 homeless 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1770018030 con 0x7f17801263e0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 monclient: dump: 2024-09-17T16:35:08.910 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.911 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f177000db60 con 0x7f17801263e0 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 msgr2=0x7f17801267c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 0x7f17801267c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f177000fb10 tx=0x7f17700045a0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 msgr2=0x7f17801267c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17801263e0 0x7f17801267c0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f177000fb10 tx=0x7f17700045a0 comp rx=0 tx=0).stop 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f1777fff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176003e580 0x7f1760040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f1777fff640 10 monclient: get_auth_request con 0x7f176003e580 auth_method 0 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f1777fff640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176003e580 0x7f1760040a40 secure :-1 s=READY pgs=3486 cs=0 l=1 rev1=1 crypto rx=0x7f17680ad000 tx=0x7f17680a4f90 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.874+0000 7f177cff9640 10 client.54103.objecter ms_handle_connect 0x7f176003e580 2024-09-17T16:35:08.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 0x7f1780122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1780051e80 con 0x7f1780126d00 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 0x7f1780122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient: get_auth_request con 0x7f1780126d00 auth_method 0 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 10 monclient(hunting): handle_auth_done global_id 44516 payload 824 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 1 -- 172.21.15.105:0/1818198926 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f17800a6cc0 con 0x7f1780126d00 2024-09-17T16:35:08.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 client.54103.objecter ms_handle_connect 0x7f1780126d00 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 client.54103.objecter resend_mon_ops 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177f7fe640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 0x7f1780122530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1770018000 tx=0x7f1770007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17700045a0 con 0x7f1780126d00 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 monclient: dump: 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:08.914 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f177001fb70 con 0x7f1780126d00 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:08.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 msgr2=0x7f1780122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 0x7f1780122530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1770018000 tx=0x7f1770007f90 comp rx=0 tx=0).stop 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 msgr2=0x7f1780122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f177cff9640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1780126d00 0x7f1780122530 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1770018000 tx=0x7f1770007f90 comp rx=0 tx=0).stop 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:08.916 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 159530, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:08.920 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:08.921 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:08.922 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:08.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:08.930 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:08.931 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.933 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.934 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:08.935 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:08.936 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:08.937 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.938 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:08.939 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:08.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 10 librados: watch_flush enter 2024-09-17T16:35:08.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 10 librados: watch_flush exit 2024-09-17T16:35:08.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 20 client.54103.objecter shutdown clearing up homeless session... 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.878+0000 7f1785a82640 10 client.54103.objecter successfully canceled tick 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176003e580 msgr2=0x7f1760040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176003e580 0x7f1760040a40 secure :-1 s=READY pgs=3486 cs=0 l=1 rev1=1 crypto rx=0x7f17680ad000 tx=0x7f17680a4f90 comp rx=0 tx=0).stop 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 10 monclient: shutdown 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17800a73d0 msgr2=0x7f17801d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 --2- 172.21.15.105:0/1818198926 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f17800a73d0 0x7f17801d1a30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f177800eba0 tx=0x7f177800c490 comp rx=0 tx=0).stop 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f177ffff640 1 -- 172.21.15.105:0/1818198926 reap_dead start 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f177ffff640 1 -- 172.21.15.105:0/1818198926 reap_dead start 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 shutdown_connections 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 >> 172.21.15.105:0/1818198926 conn(0x7f1780068ba0 msgr2=0x7f178009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 shutdown_connections 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 -- 172.21.15.105:0/1818198926 wait complete. 2024-09-17T16:35:08.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:08.882+0000 7f1785a82640 1 librados: shutdown 2024-09-17T16:35:08.943 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:08.943 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:35:08.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:35:09.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: get_monmap_and_config 2024-09-17T16:35:09.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: monmap: 2024-09-17T16:35:09.055 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:09.052557+0000 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:09.052557+0000 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.056 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.057 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:09.057 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:09.057 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 Processor -- start 2024-09-17T16:35:09.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 -- start start 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: init 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient: _add_conns ranks=[10,7,0,9,3,5,6,2,1,4,8] 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdc580a6a30 0x7fdc580a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-f con 0x7fdc580a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc580a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-i con 0x7fdc580a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdc581397f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc580a4080 con 0x7fdc581397f0 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fdc580a7c40 con 0x7fdc580a6a30 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdc580a7890 con 0x7fdc580a4ab0 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5d1b8640 10 monclient(hunting): authenticate will time out at 1392.319048s 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5759e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdc581397f0 0x7fdc58139ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41816/0 (socket says 172.21.15.105:41816) 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5759e640 1 -- 172.21.15.105:0/1454331062 learned_addr learned my addr 172.21.15.105:0/1454331062 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc580a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/1454331062 >> v2:172.21.15.160:3300/0 conn(0x7fdc580a6a30 0x7fdc580a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request con 0x7fdc580a4ab0 auth_method 0 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request con 0x7fdc580a6a30 auth_method 0 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475857522 0 0) 0x7fdc580a4080 con 0x7fdc581397f0 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient(hunting): _negotiate 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient(hunting): my global_id is 35280 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc44003530 con 0x7fdc581397f0 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3786544651 0 0) 0x7fdc44003530 con 0x7fdc581397f0 2024-09-17T16:35:09.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 msgr2=0x7fdc580a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc580a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 >> v2:172.21.15.160:3300/0 conn(0x7fdc580a6a30 msgr2=0x7fdc580a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 --2- 172.21.15.105:0/1454331062 >> v2:172.21.15.160:3300/0 conn(0x7fdc580a6a30 0x7fdc580a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc580a37e0 con 0x7fdc581397f0 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fdc50001cf0 con 0x7fdc581397f0 2024-09-17T16:35:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.050+0000 7fdc5659c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: dump: 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.065 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 >> v1:172.21.15.105:6789/0 conn(0x7fdc581397f0 legacy=0x7fdc58139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: _add_conns ranks=[2,7,4,1,5,0,3,8,6] 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc44004210 0x7fdc44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): picked mon.b con 0x7fdc44004210 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc44004e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): picked mon.i con 0x7fdc580a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc440053b0 0x7fdc4400b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): picked mon.c con 0x7fdc440053b0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdc580a7c40 con 0x7fdc44004210 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdc580a7890 con 0x7fdc440053b0 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdc4400c270 con 0x7fdc580a4ab0 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc44004210 0x7fdc44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request con 0x7fdc44004210 auth_method 0 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:09.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc440053b0 0x7fdc4400b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): get_auth_request con 0x7fdc440053b0 auth_method 0 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc44004e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request con 0x7fdc580a4ab0 auth_method 0 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:09.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_done global_id 35280 payload 293 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 -- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 msgr2=0x7fdc44004e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc44004e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 -- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc440053b0 msgr2=0x7fdc4400b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc440053b0 0x7fdc4400b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 monclient: found mon.b 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 -- 172.21.15.105:0/1454331062 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44003530 con 0x7fdc44004210 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 5 monclient: authenticate success, global_id 35280 2024-09-17T16:35:09.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc580a4ab0 0x7fdc44004e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:09.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc44004210 0x7fdc44004810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ae0 tx=0x7fdc4001ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c501a710cb1d322 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc400b2070 con 0x7fdc44004210 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: dump: 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.079 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc400a5a80 con 0x7fdc44004210 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:09.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 1 -- 172.21.15.105:0/1454331062 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc400096b0 con 0x7fdc44004210 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 4 set_mon_vals no callback set 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5659c640 10 monclient: dump: 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.081 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 10 monclient: shutdown 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc44004210 msgr2=0x7fdc44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 --2- 172.21.15.105:0/1454331062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdc44004210 0x7fdc44004810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ae0 tx=0x7fdc4001ca60 comp rx=0 tx=0).stop 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5759e640 1 -- 172.21.15.105:0/1454331062 reap_dead start 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/1454331062 shutdown_connections 2024-09-17T16:35:09.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/1454331062 >> 172.21.15.105:0/1454331062 conn(0x7fdc580a8d00 msgr2=0x7fdc580a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/1454331062 shutdown_connections 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.054+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/1454331062 wait complete. 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: monmap: 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.083 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 librados: starting msgr at 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 librados: starting objecter 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 Processor -- start 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- start start 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 librados: setting wanted keys 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 librados: calling monclient init 2024-09-17T16:35:09.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: init 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _add_conns ranks=[7,4,5,6,0,8,3,2,1] 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc44004210 0x7fdc581d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-i con 0x7fdc44004210 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 0x7fdc58136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-c con 0x7fdc58136bd0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdc580a0030 0x7fdc580a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): picked mon.noname-h con 0x7fdc580a0030 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdc5806e770 con 0x7fdc58136bd0 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fdc5806b5b0 con 0x7fdc580a0030 2024-09-17T16:35:09.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdc58142260 con 0x7fdc44004210 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): authenticate will time out at 1392.325243s 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 0x7fdc58136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 0x7fdc58136ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36314/0 (socket says 172.21.15.105:36314) 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 -- 172.21.15.105:0/2891962573 learned_addr learned my addr 172.21.15.105:0/2891962573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request con 0x7fdc58136bd0 auth_method 0 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5759e640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdc580a0030 0x7fdc580a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient(hunting): handle_auth_done global_id 54112 payload 824 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc44004210 msgr2=0x7fdc581d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc44004210 0x7fdc581d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdc44004210 0x7fdc581d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdc580a0030 msgr2=0x7fdc580a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdc580a0030 0x7fdc580a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 monclient: found mon.noname-c 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc58140c00 con 0x7fdc58136bd0 2024-09-17T16:35:09.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5759e640 10 monclient: get_auth_request con 0x7fdc580a0030 auth_method 0 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5759e640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdc580a0030 0x7fdc580a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4ffff640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 0x7fdc58136ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc48011560 tx=0x7fdc48011a20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 client.?.objecter ms_handle_connect 0x7fdc58136bd0 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc48015ae0 con 0x7fdc58136bd0 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: dump: 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 5 monclient: authenticate success, global_id 54112 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48015cc0 con 0x7fdc58136bd0 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _renew_subs 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5813ae60 con 0x7fdc58136bd0 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc4800f8d0 con 0x7fdc58136bd0 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 client.54112.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc55d9b640 4 set_mon_vals no callback set 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: dump: 2024-09-17T16:35:09.090 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.091 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _renew_subs 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc58137530 con 0x7fdc58136bd0 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 librados: init done 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 0x7fdc580a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdc58123020 con 0x7fdc58123300 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 0x7fdc580a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient: get_auth_request con 0x7fdc58123300 auth_method 0 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdc4800f460 con 0x7fdc58136bd0 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_done global_id 54121 payload 824 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc580a6e10 con 0x7fdc58123300 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc3803e550 0x7fdc38040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(325..325 src has 1..325) ==== 6293+0+0 (secure 0 0 0) 0x7fdc48050770 con 0x7fdc58136bd0 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 client.54112.objecter ms_dispatch 0x7fdc58000ba0 osd_map(325..325 src has 1..325) 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 0x7fdc580a09d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ab0 tx=0x7fdc44005840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 3 client.54112.objecter handle_osd_map got epochs [325,325] > 0 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 3 client.54112.objecter handle_osd_map decoding full epoch 325 2024-09-17T16:35:09.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 20 client.54112.objecter dump_active .. 0 homeless 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 client.54112.objecter ms_handle_connect 0x7fdc58123300 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 client.54112.objecter resend_mon_ops 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc400bc030 con 0x7fdc58123300 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: dump: 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.094 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5759e640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc3803e550 0x7fdc38040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc5759e640 10 monclient: get_auth_request con 0x7fdc3803e550 auth_method 0 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdc400096b0 con 0x7fdc58123300 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:09.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 msgr2=0x7fdc580a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 0x7fdc580a09d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ab0 tx=0x7fdc44005840 comp rx=0 tx=0).stop 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 msgr2=0x7fdc580a09d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.058+0000 7fdc4f7fe640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58123300 0x7fdc580a09d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.062+0000 7fdc4f7fe640 10 client.54112.objecter ms_handle_connect 0x7fdc3803e550 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.062+0000 7fdc5759e640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc3803e550 0x7fdc38040a10 secure :-1 s=READY pgs=3487 cs=0 l=1 rev1=1 crypto rx=0x7fdc580a4ed0 tx=0x7fdc50006040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 0x7fdc5813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdc580a8970 con 0x7fdc5813ebe0 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 0x7fdc5813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient: get_auth_request con 0x7fdc5813ebe0 auth_method 0 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 10 monclient(hunting): handle_auth_done global_id 54130 payload 824 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 1 -- 172.21.15.105:0/2891962573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdc581236e0 con 0x7fdc5813ebe0 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc56d9d640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 0x7fdc5813eff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ae0 tx=0x7fdc400a7070 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 client.54112.objecter ms_handle_connect 0x7fdc5813ebe0 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 client.54112.objecter resend_mon_ops 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc400bc040 con 0x7fdc5813ebe0 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 monclient: dump: 2024-09-17T16:35:09.097 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.098 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7fdc400a5e50 con 0x7fdc5813ebe0 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 msgr2=0x7fdc5813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 0x7fdc5813eff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009ae0 tx=0x7fdc400a7070 comp rx=0 tx=0).stop 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 msgr2=0x7fdc5813eff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc4f7fe640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc5813ebe0 0x7fdc5813eff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.099 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16063, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:09.100 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.102 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:09.104 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.105 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.106 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.107 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.109 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:09.112 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:09.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:09.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.122 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:09.123 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.123 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.123 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:09.123 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 librados: watch_flush enter 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 librados: watch_flush exit 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 20 client.54112.objecter shutdown clearing up homeless session... 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 client.54112.objecter successfully canceled tick 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc3803e550 msgr2=0x7fdc38040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc3803e550 0x7fdc38040a10 secure :-1 s=READY pgs=3487 cs=0 l=1 rev1=1 crypto rx=0x7fdc580a4ed0 tx=0x7fdc50006040 comp rx=0 tx=0).stop 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 10 monclient: shutdown 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 msgr2=0x7fdc58136ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 --2- 172.21.15.105:0/2891962573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdc58136bd0 0x7fdc58136ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc48011560 tx=0x7fdc48011a20 comp rx=0 tx=0).stop 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5759e640 1 -- 172.21.15.105:0/2891962573 reap_dead start 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5759e640 1 -- 172.21.15.105:0/2891962573 reap_dead start 2024-09-17T16:35:09.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 shutdown_connections 2024-09-17T16:35:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 >> 172.21.15.105:0/2891962573 conn(0x7fdc580a8d00 msgr2=0x7fdc58140e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 shutdown_connections 2024-09-17T16:35:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 -- 172.21.15.105:0/2891962573 wait complete. 2024-09-17T16:35:09.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.066+0000 7fdc5d1b8640 1 librados: shutdown 2024-09-17T16:35:09.126 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:09.126 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:35:09.126 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:35:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: get_monmap_and_config 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: monmap: 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:09.256876+0000 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:09.256876+0000 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:09.261 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 Processor -- start 2024-09-17T16:35:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 -- start start 2024-09-17T16:35:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: init 2024-09-17T16:35:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient: _add_conns ranks=[10,0,1,9,2,6,5,7,4,3,8] 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65400a4750 0x7f65400a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-f con 0x7f65400a4750 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f654006cb00 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f65400a5070 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654013a350 con 0x7f654006cb00 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654006c7f0 con 0x7f65400a5070 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f65400a6610 con 0x7f65400a4750 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f65463f2640 10 monclient(hunting): authenticate will time out at 1392.522778s 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f654006cb00 0x7f6540139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41828/0 (socket says 172.21.15.105:41828) 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 1 -- 172.21.15.105:0/2980466204 learned_addr learned my addr 172.21.15.105:0/2980466204 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:09.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 --2- 172.21.15.105:0/2980466204 >> v2:172.21.15.160:3300/0 conn(0x7f65400a4750 0x7f65400a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710872197 0 0) 0x7f654013a350 con 0x7f654006cb00 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): _negotiate 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): my global_id is 35289 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003530 con 0x7f654006cb00 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): get_auth_request con 0x7f65400a4750 auth_method 0 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 645290740 0 0) 0x7f652c003530 con 0x7f654006cb00 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 >> v2:172.21.15.160:3300/0 conn(0x7f65400a4750 msgr2=0x7f65400a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 --2- 172.21.15.105:0/2980466204 >> v2:172.21.15.160:3300/0 conn(0x7f65400a4750 0x7f65400a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 >> v1:172.21.15.160:6789/0 conn(0x7f65400a5070 legacy=0x7f6540139c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654006c1f0 con 0x7f654006cb00 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f6528001970 con 0x7f654006cb00 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: dump: 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 >> v1:172.21.15.105:6789/0 conn(0x7f654006cb00 legacy=0x7f6540139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient: _add_conns ranks=[0,1,5,8,3,6,7,4,2] 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 0x7f652c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): picked mon.a con 0x7f652c004210 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004da0 0x7f652c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): picked mon.f con 0x7f652c004da0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f652c009c40 0x7f652c00c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): picked mon.h con 0x7f652c009c40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f652c00c980 con 0x7f652c004210 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f652c00cb30 con 0x7f652c004da0 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f652c00cce0 con 0x7f652c009c40 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653effd640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 0x7f652c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): get_auth_request con 0x7f652c004210 auth_method 0 2024-09-17T16:35:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004da0 0x7f652c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 10 monclient(hunting): get_auth_request con 0x7f652c004da0 auth_method 0 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient(hunting): handle_auth_done global_id 35289 payload 293 2024-09-17T16:35:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 -- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f652c009c40 msgr2=0x7f652c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f654498f640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f652c009c40 0x7f652c00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f652c009c40 0x7f652c00c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 -- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004da0 msgr2=0x7f652c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004da0 0x7f652c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.254+0000 7f653ffff640 1 monclient: found mon.a 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 1 -- 172.21.15.105:0/2980466204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c003530 con 0x7f652c004210 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f654498f640 1 -- 172.21.15.105:0/2980466204 reap_dead start 2024-09-17T16:35:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 5 monclient: authenticate success, global_id 35289 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653ffff640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 0x7f652c004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6530009ab0 tx=0x7f653001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b7d823e4c0a86d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65300096b0 con 0x7f652c004210 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: dump: 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65300a6880 con 0x7f652c004210 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 1 -- 172.21.15.105:0/2980466204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65300a73f0 con 0x7f652c004210 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 4 set_mon_vals no callback set 2024-09-17T16:35:09.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f653effd640 10 monclient: dump: 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 10 monclient: shutdown 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 msgr2=0x7f652c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 0x7f652c004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6530009ab0 tx=0x7f653001ca30 comp rx=0 tx=0).stop 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- 172.21.15.105:0/2980466204 shutdown_connections 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 --2- 172.21.15.105:0/2980466204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f652c004210 0x7f652c004810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- 172.21.15.105:0/2980466204 >> 172.21.15.105:0/2980466204 conn(0x7f65400a93d0 msgr2=0x7f65400a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- 172.21.15.105:0/2980466204 shutdown_connections 2024-09-17T16:35:09.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- 172.21.15.105:0/2980466204 wait complete. 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 10 monclient: monmap: 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.286 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 librados: starting msgr at 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 librados: starting objecter 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 Processor -- start 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 -- start start 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 librados: setting wanted keys 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 1 librados: calling monclient init 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.258+0000 7f65463f2640 10 monclient: init 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _add_conns ranks=[1,7,0,3,2,6,4,5,8] 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004210 0x7f6540127180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-f con 0x7f652c004210 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65401d17f0 0x7f65401d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-i con 0x7f65401d17f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 0x7f6540124c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): picked mon.noname-a con 0x7f65401d2170 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f654006b5b0 con 0x7f65401d2170 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f654009ecd0 con 0x7f652c004210 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6540139d40 con 0x7f65401d17f0 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): authenticate will time out at 1392.528281s 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 0x7f6540124c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 0x7f6540124c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52794/0 (socket says 172.21.15.105:52794) 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 learned_addr learned my addr 172.21.15.105:0/1688680467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): get_auth_request con 0x7f65401d2170 auth_method 0 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004210 0x7f6540127180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): get_auth_request con 0x7f652c004210 auth_method 0 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65401d17f0 0x7f65401d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 10 monclient(hunting): get_auth_request con 0x7f65401d17f0 auth_method 0 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient(hunting): handle_auth_done global_id 35298 payload 824 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65401d17f0 msgr2=0x7f65401d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f65401d17f0 0x7f65401d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004210 msgr2=0x7f6540127180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f652c004210 0x7f6540127180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 monclient: found mon.noname-a 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6540128980 con 0x7f65401d2170 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f654498f640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 0x7f6540124c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65400700f0 tx=0x7f653800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.?.objecter ms_handle_connect 0x7f65401d2170 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6538018070 con 0x7f65401d2170 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: dump: 2024-09-17T16:35:09.291 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.292 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6538003df0 con 0x7f65401d2170 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 5 monclient: authenticate success, global_id 35298 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65380054e0 con 0x7f65401d2170 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _renew_subs 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6540128e30 con 0x7f65401d2170 2024-09-17T16:35:09.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653e7fc640 4 set_mon_vals no callback set 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: dump: 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.294 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 client.35298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _renew_subs 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65400a8880 con 0x7f65401d2170 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 librados: init done 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 0x7f6540125590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6540051e80 con 0x7f65401251b0 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 0x7f6540125590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient: get_auth_request con 0x7f65401251b0 auth_method 0 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6538005680 con 0x7f65401d2170 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 10 monclient(hunting): handle_auth_done global_id 24207 payload 824 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6540127b70 con 0x7f65401251b0 2024-09-17T16:35:09.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652003e580 0x7f6520040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(325..325 src has 1..325) ==== 6293+0+0 (secure 0 0 0) 0x7f653804f570 con 0x7f65401d2170 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.35298.objecter ms_dispatch 0x7f6540000ba0 osd_map(325..325 src has 1..325) 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 3 client.35298.objecter handle_osd_map got epochs [325,325] > 0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 3 client.35298.objecter handle_osd_map decoding full epoch 325 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653ffff640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 0x7f6540125590 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f652c004790 tx=0x7f65300197d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 20 client.35298.objecter dump_active .. 0 homeless 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.35298.objecter ms_handle_connect 0x7f65401251b0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.35298.objecter resend_mon_ops 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652003e580 0x7f6520040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 10 monclient: get_auth_request con 0x7f652003e580 auth_method 0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65300096b0 con 0x7f65401251b0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: dump: 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.298 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6530003620 con 0x7f65401251b0 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 msgr2=0x7f6540125590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 0x7f6540125590 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f652c004790 tx=0x7f65300197d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 msgr2=0x7f6540125590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401251b0 0x7f6540125590 secure :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f652c004790 tx=0x7f65300197d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653cff9640 10 client.35298.objecter ms_handle_connect 0x7f652003e580 2024-09-17T16:35:09.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.262+0000 7f653f7fe640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652003e580 0x7f6520040a40 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7f6540136820 tx=0x7f6528009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:09.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 0x7f6540121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6540125ad0 con 0x7f65401213a0 2024-09-17T16:35:09.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 0x7f6540121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient: get_auth_request con 0x7f65401213a0 auth_method 0 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 10 monclient(hunting): handle_auth_done global_id 24216 payload 824 2024-09-17T16:35:09.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 1 -- 172.21.15.105:0/1688680467 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f654013a130 con 0x7f65401213a0 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 client.35298.objecter ms_handle_connect 0x7f65401213a0 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 client.35298.objecter resend_mon_ops 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653ffff640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 0x7f6540121780 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f653001c860 tx=0x7f652c005840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65300096b0 con 0x7f65401213a0 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 monclient: dump: 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.303 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f65300c3070 con 0x7f65401213a0 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 msgr2=0x7f6540121780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 0x7f6540121780 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f653001c860 tx=0x7f652c005840 comp rx=0 tx=0).stop 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 msgr2=0x7f6540121780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f653cff9640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f65401213a0 0x7f6540121780 secure :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f653001c860 tx=0x7f652c005840 comp rx=0 tx=0).stop 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 456909, 2024-09-17T16:35:09.305 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:09.306 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:09.307 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.308 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:09.309 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.310 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.312 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.313 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.315 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:09.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:09.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.328 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.328 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:09.328 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 librados: watch_flush enter 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 librados: watch_flush exit 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 20 client.35298.objecter shutdown clearing up homeless session... 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 client.35298.objecter successfully canceled tick 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652003e580 msgr2=0x7f6520040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f652003e580 0x7f6520040a40 secure :-1 s=READY pgs=3488 cs=0 l=1 rev1=1 crypto rx=0x7f6540136820 tx=0x7f6528009040 comp rx=0 tx=0).stop 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 10 monclient: shutdown 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 msgr2=0x7f6540124c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 --2- 172.21.15.105:0/1688680467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65401d2170 0x7f6540124c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65400700f0 tx=0x7f653800db80 comp rx=0 tx=0).stop 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 reap_dead start 2024-09-17T16:35:09.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f654498f640 1 -- 172.21.15.105:0/1688680467 reap_dead start 2024-09-17T16:35:09.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 shutdown_connections 2024-09-17T16:35:09.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 >> 172.21.15.105:0/1688680467 conn(0x7f65400a93d0 msgr2=0x7f65400a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 shutdown_connections 2024-09-17T16:35:09.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 -- 172.21.15.105:0/1688680467 wait complete. 2024-09-17T16:35:09.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.270+0000 7f65463f2640 1 librados: shutdown 2024-09-17T16:35:09.331 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:09.331 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:35:09.331 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:35:09.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: get_monmap_and_config 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: monmap: 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:09.440451+0000 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:09.440451+0000 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.444 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 Processor -- start 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 -- start start 2024-09-17T16:35:09.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: init 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient: _add_conns ranks=[6,2,0,5,3,1,7,9,8,10,4] 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04280a2620 0x7f04280a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-d con 0x7f04280a2620 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f0428137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-b con 0x7f04280a2f40 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0428137940 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042806f2e0 con 0x7f0428137940 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f042806e580 con 0x7f04280a2f40 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f04280a15e0 con 0x7f04280a2620 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042cae4640 10 monclient(hunting): authenticate will time out at 1392.705935s 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f0428137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0426d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0428137940 0x7f042813e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41840/0 (socket says 172.21.15.105:41840) 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04280a2620 0x7f04280a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0426d9d640 1 -- 172.21.15.105:0/3923637206 learned_addr learned my addr 172.21.15.105:0/3923637206 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): get_auth_request con 0x7f04280a2f40 auth_method 0 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): get_auth_request con 0x7f04280a2620 auth_method 0 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient(hunting): handle_auth_done global_id 44525 payload 274 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 -- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04280a2620 msgr2=0x7f04280a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 --2- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04280a2620 0x7f04280a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 -- 172.21.15.105:0/3923637206 >> v1:172.21.15.105:6789/0 conn(0x7f0428137940 legacy=0x7f042813e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 monclient: found mon.noname-b 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.438+0000 7f0425d9b640 1 -- 172.21.15.105:0/3923637206 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04280a3c30 con 0x7f04280a2f40 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 --2- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f0428137400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04180097f0 tx=0x7f041801bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839b2e53f392b2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 1 -- 172.21.15.105:0/3923637206 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04180a88d0 con 0x7f04280a2f40 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: dump: 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.452 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 1 -- 172.21.15.105:0/3923637206 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04180a8a90 con 0x7f04280a2f40 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 1 -- 172.21.15.105:0/3923637206 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04180a8c80 con 0x7f04280a2f40 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 5 monclient: authenticate success, global_id 44525 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:09.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042559a640 10 monclient: dump: 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 4 set_mon_vals no callback set 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: shutdown 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 msgr2=0x7f0428137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f0428137400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04180097f0 tx=0x7f041801bfa0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- 172.21.15.105:0/3923637206 shutdown_connections 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f0428137400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- 172.21.15.105:0/3923637206 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f04280a2620 0x7f04280a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- 172.21.15.105:0/3923637206 >> 172.21.15.105:0/3923637206 conn(0x7f0428068ba0 msgr2=0x7f0428068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- 172.21.15.105:0/3923637206 shutdown_connections 2024-09-17T16:35:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- 172.21.15.105:0/3923637206 wait complete. 2024-09-17T16:35:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: monmap: 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.464 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 librados: starting msgr at 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 librados: starting objecter 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 Processor -- start 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- start start 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 librados: setting wanted keys 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 librados: calling monclient init 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: init 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient: _add_conns ranks=[7,2,4,6,0,1,8,3,5] 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04280a2620 0x7f0428131fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-i con 0x7f04280a2620 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f04281324f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-b con 0x7f04280a2f40 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0428137940 0x7f0428070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): picked mon.noname-c con 0x7f0428137940 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f04281431b0 con 0x7f04280a2f40 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0428143030 con 0x7f0428137940 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0428143330 con 0x7f04280a2620 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042cae4640 10 monclient(hunting): authenticate will time out at 1392.711385s 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f04281324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0428137940 0x7f0428070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f04281324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57240/0 (socket says 172.21.15.105:57240) 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 -- 172.21.15.105:0/4185948133 learned_addr learned my addr 172.21.15.105:0/4185948133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): get_auth_request con 0x7f04280a2f40 auth_method 0 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): get_auth_request con 0x7f0428137940 auth_method 0 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient(hunting): handle_auth_done global_id 44534 payload 824 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04280a2620 msgr2=0x7f0428131fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f042659c640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04280a2620 0x7f0428131fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f04280a2620 0x7f0428131fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0428137940 msgr2=0x7f0428070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0428137940 0x7f0428070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 monclient: found mon.noname-b 2024-09-17T16:35:09.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0425d9b640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042809e5a0 con 0x7f04280a2f40 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.442+0000 7f0426d9d640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0428137940 0x7f0428070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0425d9b640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f04281324f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f041801c840 tx=0x7f041801a7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.?.objecter ms_handle_connect 0x7f04280a2f40 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04180be070 con 0x7f04280a2f40 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: dump: 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.469 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04180044b0 con 0x7f04280a2f40 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0418003a90 con 0x7f04280a2f40 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 5 monclient: authenticate success, global_id 44534 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 monclient: _renew_subs 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0424d99640 4 set_mon_vals no callback set 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042809e150 con 0x7f04280a2f40 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: dump: 2024-09-17T16:35:09.471 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.472 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 client.44534.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 monclient: _renew_subs 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f042811ea80 con 0x7f04280a2f40 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 1 librados: init done 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 0x7f0428070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0428051e80 con 0x7f0428070a40 2024-09-17T16:35:09.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 0x7f0428070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient: get_auth_request con 0x7f0428070a40 auth_method 0 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f04180a8900 con 0x7f04280a2f40 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f040803e550 0x7f0408040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 10 monclient(hunting): handle_auth_done global_id 54125 payload 824 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(325..325 src has 1..325) ==== 6293+0+0 (secure 0 0 0) 0x7f04180e9430 con 0x7f04280a2f40 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.44534.objecter ms_dispatch 0x7f0428000ba0 osd_map(325..325 src has 1..325) 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 3 client.44534.objecter handle_osd_map got epochs [325,325] > 0 2024-09-17T16:35:09.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 3 client.44534.objecter handle_osd_map decoding full epoch 325 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04280a2a00 con 0x7f0428070a40 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 20 client.44534.objecter dump_active .. 0 homeless 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.44534.objecter ms_handle_connect 0x7f0428070a40 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.44534.objecter resend_mon_ops 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f042659c640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 0x7f0428070e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c010c80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0426d9d640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f040803e550 0x7f0408040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f041c00f040 con 0x7f0428070a40 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: dump: 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.475 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0426d9d640 10 monclient: get_auth_request con 0x7f040803e550 auth_method 0 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f041c01c400 con 0x7f0428070a40 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 msgr2=0x7f0428070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 0x7f0428070e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c010c80 comp rx=0 tx=0).stop 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 msgr2=0x7f0428070e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428070a40 0x7f0428070e20 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c010c80 comp rx=0 tx=0).stop 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0416ffd640 10 client.44534.objecter ms_handle_connect 0x7f040803e550 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.446+0000 7f0426d9d640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f040803e550 0x7f0408040a10 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7f0428071b40 tx=0x7f04200017e0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:09.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 0x7f042813d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f042813fe60 con 0x7f0428071360 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 0x7f042813d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient: get_auth_request con 0x7f0428071360 auth_method 0 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 10 monclient(hunting): handle_auth_done global_id 54134 payload 824 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 1 -- 172.21.15.105:0/4185948133 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f042811e5c0 con 0x7f0428071360 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042659c640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 0x7f042813d0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 client.44534.objecter ms_handle_connect 0x7f0428071360 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 client.44534.objecter resend_mon_ops 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f041c006040 con 0x7f0428071360 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 monclient: dump: 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.479 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f041c00fc90 con 0x7f0428071360 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 msgr2=0x7f042813d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 0x7f042813d0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 msgr2=0x7f042813d0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0416ffd640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0428071360 0x7f042813d0a0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f041c0107c0 tx=0x7f041c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16451, 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:09.481 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:09.482 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:09.483 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:09.484 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:09.485 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.486 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.488 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.489 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.491 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.493 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.496 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:09.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.502 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:09.503 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 librados: watch_flush enter 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 librados: watch_flush exit 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 20 client.44534.objecter shutdown clearing up homeless session... 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 client.44534.objecter successfully canceled tick 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f040803e550 msgr2=0x7f0408040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f040803e550 0x7f0408040a10 secure :-1 s=READY pgs=3489 cs=0 l=1 rev1=1 crypto rx=0x7f0428071b40 tx=0x7f04200017e0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 10 monclient: shutdown 2024-09-17T16:35:09.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 msgr2=0x7f04281324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 --2- 172.21.15.105:0/4185948133 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04280a2f40 0x7f04281324f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f041801c840 tx=0x7f041801a7e0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0426d9d640 1 -- 172.21.15.105:0/4185948133 reap_dead start 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f0426d9d640 1 -- 172.21.15.105:0/4185948133 reap_dead start 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 shutdown_connections 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 >> 172.21.15.105:0/4185948133 conn(0x7f0428068ba0 msgr2=0x7f042809ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 shutdown_connections 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 -- 172.21.15.105:0/4185948133 wait complete. 2024-09-17T16:35:09.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.454+0000 7f042cae4640 1 librados: shutdown 2024-09-17T16:35:09.506 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:09.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:35:09.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ lPP2 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3113 ms) 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (6888 ms) 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-17T16:35:09.592 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7088 ms) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (12935 ms) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7050 ms) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (11237 ms) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7092 ms) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (180981 ms total) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (368918 ms total) 2024-09-17T16:35:09.593 INFO:tasks.workunit.client.0.smithi160.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: get_monmap_and_config 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: monmap: 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:09.628090+0000 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:09.628090+0000 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.631 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:09.632 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:09.633 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 Processor -- start 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 -- start start 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: init 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient: _add_conns ranks=[9,4,6,10,2,8,3,5,7,1,0] 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5780a6a30 0x7fb5780a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-a con 0x7fb5780a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 0x7fb5780a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-c con 0x7fb5780a4ab0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb5781397f0 0x7fb578139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-d con 0x7fb5781397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb5780a7c40 con 0x7fb5780a6a30 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb5780a7890 con 0x7fb5780a4ab0 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fb5780a4080 con 0x7fb5781397f0 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57cddd640 10 monclient(hunting): authenticate will time out at 1392.893949s 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5780a6a30 0x7fb5780a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5780a6a30 0x7fb5780a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52806/0 (socket says 172.21.15.105:52806) 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 -- 172.21.15.105:0/2737085829 learned_addr learned my addr 172.21.15.105:0/2737085829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): get_auth_request con 0x7fb5780a6a30 auth_method 0 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb5781397f0 0x7fb578139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 0x7fb5780a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): get_auth_request con 0x7fb5780a4ab0 auth_method 0 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): get_auth_request con 0x7fb5781397f0 auth_method 0 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb576d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient(hunting): handle_auth_done global_id 35316 payload 274 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 -- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb5781397f0 msgr2=0x7fb578139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb5781397f0 0x7fb578139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 -- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 msgr2=0x7fb5780a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 0x7fb5780a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 monclient: found mon.noname-a 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb57659c640 1 -- 172.21.15.105:0/2737085829 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780a37e0 con 0x7fb5780a6a30 2024-09-17T16:35:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.626+0000 7fb575d9b640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 0x7fb5780a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:09.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780a6a30 0x7fb5780a6e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb568009e80 tx=0x7fb56801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5acd218b37af1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 1 -- 172.21.15.105:0/2737085829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb5680090d0 con 0x7fb5780a6a30 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: dump: 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 1 -- 172.21.15.105:0/2737085829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5680a5b70 con 0x7fb5780a6a30 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 5 monclient: authenticate success, global_id 35316 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 1 -- 172.21.15.105:0/2737085829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb5680a5d60 con 0x7fb5780a6a30 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 4 set_mon_vals no callback set 2024-09-17T16:35:09.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57559a640 10 monclient: dump: 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: shutdown 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780a6a30 msgr2=0x7fb5780a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780a6a30 0x7fb5780a6e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb568009e80 tx=0x7fb56801c8a0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- 172.21.15.105:0/2737085829 shutdown_connections 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fb5781397f0 0x7fb578139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a4ab0 0x7fb5780a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- 172.21.15.105:0/2737085829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5780a6a30 0x7fb5780a6e10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- 172.21.15.105:0/2737085829 >> 172.21.15.105:0/2737085829 conn(0x7fb5780a8d00 msgr2=0x7fb5780a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- 172.21.15.105:0/2737085829 shutdown_connections 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- 172.21.15.105:0/2737085829 wait complete. 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: monmap: 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.651 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 librados: starting msgr at 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 librados: starting objecter 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 Processor -- start 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- start start 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 librados: setting wanted keys 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 librados: calling monclient init 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: init 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient: _add_conns ranks=[2,4,7,8,5,1,6,0,3] 2024-09-17T16:35:09.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 0x7fb5781d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-b con 0x7fb5780a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a6a30 0x7fb5781d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-c con 0x7fb5780a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5781397f0 0x7fb57813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): picked mon.noname-i con 0x7fb5781397f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb57806e770 con 0x7fb5780a4ab0 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb5780a2fc0 con 0x7fb5780a6a30 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb578143260 con 0x7fb5781397f0 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57cddd640 10 monclient(hunting): authenticate will time out at 1392.899206s 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 0x7fb5781d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a6a30 0x7fb5781d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 0x7fb5781d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57254/0 (socket says 172.21.15.105:57254) 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 -- 172.21.15.105:0/3023157398 learned_addr learned my addr 172.21.15.105:0/3023157398 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): get_auth_request con 0x7fb5780a4ab0 auth_method 0 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): get_auth_request con 0x7fb5780a6a30 auth_method 0 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb576d9d640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5781397f0 0x7fb57813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient(hunting): handle_auth_done global_id 44543 payload 824 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5781397f0 msgr2=0x7fb57813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5781397f0 0x7fb57813d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a6a30 msgr2=0x7fb5781d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb5780a6a30 0x7fb5781d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 monclient: found mon.noname-b 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb578123e20 con 0x7fb5780a4ab0 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb576d9d640 10 monclient: get_auth_request con 0x7fb5781397f0 auth_method 0 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb576d9d640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb5781397f0 0x7fb57813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb57659c640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 0x7fb5781d1410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb56801d140 tx=0x7fb568019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb56effd640 10 client.?.objecter ms_handle_connect 0x7fb5780a4ab0 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.630+0000 7fb56effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb5680bc020 con 0x7fb5780a4ab0 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: dump: 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.657 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568003e10 con 0x7fb5780a4ab0 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 5 monclient: authenticate success, global_id 44543 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 monclient: _renew_subs 2024-09-17T16:35:09.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57813fb10 con 0x7fb5780a4ab0 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb5680044b0 con 0x7fb5780a4ab0 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb574d99640 4 set_mon_vals no callback set 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 client.44543.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: dump: 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.659 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.660 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 monclient: _renew_subs 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5780abbf0 con 0x7fb5780a4ab0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 1 librados: init done 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 0x7fb57813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb578051e80 con 0x7fb5781242b0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb5680bf020 con 0x7fb5780a4ab0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 0x7fb57813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient: get_auth_request con 0x7fb5781242b0 auth_method 0 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb55803e550 0x7fb558040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(326..326 src has 1..326) ==== 5801+0+0 (secure 0 0 0) 0x7fb5680eaa10 con 0x7fb5780a4ab0 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 client.44543.objecter ms_dispatch 0x7fb578000ba0 osd_map(326..326 src has 1..326) 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 3 client.44543.objecter handle_osd_map got epochs [326,326] > 0 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 3 client.44543.objecter handle_osd_map decoding full epoch 326 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 20 client.44543.objecter dump_active .. 0 homeless 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 10 monclient(hunting): handle_auth_done global_id 54145 payload 824 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb578139c60 con 0x7fb5781242b0 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb576d9d640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb55803e550 0x7fb558040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb576d9d640 10 monclient: get_auth_request con 0x7fb55803e550 auth_method 0 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb575d9b640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 0x7fb57813db70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb57813d540 tx=0x7fb560011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 client.44543.objecter ms_handle_connect 0x7fb5781242b0 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 client.44543.objecter resend_mon_ops 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb560014440 con 0x7fb5781242b0 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: dump: 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.663 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 client.44543.objecter ms_handle_connect 0x7fb55803e550 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb576d9d640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb55803e550 0x7fb558040a10 secure :-1 s=READY pgs=3490 cs=0 l=1 rev1=1 crypto rx=0x7fb578123080 tx=0x7fb570008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb56000fb30 con 0x7fb5781242b0 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 msgr2=0x7fb57813db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 0x7fb57813db70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb57813d540 tx=0x7fb560011e30 comp rx=0 tx=0).stop 2024-09-17T16:35:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 msgr2=0x7fb57813db70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.634+0000 7fb56effd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5781242b0 0x7fb57813db70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:09.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 0x7fb57813e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb57806a200 con 0x7fb57813e0b0 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 0x7fb57813e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient: get_auth_request con 0x7fb57813e0b0 auth_method 0 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 10 monclient(hunting): handle_auth_done global_id 54154 payload 824 2024-09-17T16:35:09.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 1 -- 172.21.15.105:0/3023157398 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb57813ea20 con 0x7fb57813e0b0 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb575d9b640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 0x7fb57813e4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb57813d540 tx=0x7fb5600046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 client.44543.objecter ms_handle_connect 0x7fb57813e0b0 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 client.44543.objecter resend_mon_ops 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb560022280 con 0x7fb57813e0b0 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 monclient: dump: 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7fb560022280 con 0x7fb57813e0b0 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 msgr2=0x7fb57813e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 0x7fb57813e4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb57813d540 tx=0x7fb5600046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 msgr2=0x7fb57813e4e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb56effd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57813e0b0 0x7fb57813e4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16647, 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:09.670 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:09.671 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:09.672 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:09.673 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:09.674 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:09.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.677 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.682 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:09.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.688 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:09.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:09.692 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 librados: watch_flush enter 2024-09-17T16:35:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 librados: watch_flush exit 2024-09-17T16:35:09.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 20 client.44543.objecter shutdown clearing up homeless session... 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 client.44543.objecter successfully canceled tick 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb55803e550 msgr2=0x7fb558040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb55803e550 0x7fb558040a10 secure :-1 s=READY pgs=3490 cs=0 l=1 rev1=1 crypto rx=0x7fb578123080 tx=0x7fb570008040 comp rx=0 tx=0).stop 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 10 monclient: shutdown 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 msgr2=0x7fb5781d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 --2- 172.21.15.105:0/3023157398 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5780a4ab0 0x7fb5781d1410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb56801d140 tx=0x7fb568019640 comp rx=0 tx=0).stop 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb576d9d640 1 -- 172.21.15.105:0/3023157398 reap_dead start 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb576d9d640 1 -- 172.21.15.105:0/3023157398 reap_dead start 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 shutdown_connections 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 >> 172.21.15.105:0/3023157398 conn(0x7fb5780a8d00 msgr2=0x7fb5780a4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 shutdown_connections 2024-09-17T16:35:09.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 -- 172.21.15.105:0/3023157398 wait complete. 2024-09-17T16:35:09.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.642+0000 7fb57cddd640 1 librados: shutdown 2024-09-17T16:35:09.695 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:09.695 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:35:09.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:35:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.802+0000 7f6388367640 10 monclient: get_monmap_and_config 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.802+0000 7f6388367640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient: monmap: 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:09.807811+0000 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:09.807811+0000 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:09.811 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:09.812 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 Processor -- start 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 -- start start 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient: init 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient: _add_conns ranks=[3,6,5,8,0,2,4,10,9,1,7] 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a68e0 0x7f63800a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): picked mon.noname-g con 0x7f63800a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): picked mon.noname-d con 0x7f63800a6cc0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63801396a0 0x7f6380139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): picked mon.noname-h con 0x7f63801396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f63800a2ea0 con 0x7f63800a68e0 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6380069ab0 con 0x7f63801396a0 2024-09-17T16:35:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f63800a87d0 con 0x7f63800a6cc0 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6388367640 10 monclient(hunting): authenticate will time out at 1393.073479s 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42814/0 (socket says 172.21.15.105:42814) 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 -- 172.21.15.105:0/3927529110 learned_addr learned my addr 172.21.15.105:0/3927529110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): get_auth_request con 0x7f63800a6cc0 auth_method 0 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386904640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63801396a0 0x7f6380139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386103640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a68e0 0x7f63800a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386904640 10 monclient(hunting): get_auth_request con 0x7f63801396a0 auth_method 0 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386904640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386904640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386904640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386103640 10 monclient(hunting): get_auth_request con 0x7f63800a68e0 auth_method 0 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386103640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386103640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6386103640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient(hunting): handle_auth_done global_id 24243 payload 274 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 -- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63801396a0 msgr2=0x7f6380139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63801396a0 0x7f6380139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 -- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a68e0 msgr2=0x7f63800a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a68e0 0x7f63800a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 monclient: found mon.noname-d 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.806+0000 7f6385902640 1 -- 172.21.15.105:0/3927529110 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800a7b80 con 0x7f63800a6cc0 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f63700098a0 tx=0x7f637001c410 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3d85cc8ac0590396 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 1 -- 172.21.15.105:0/3927529110 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63700b0070 con 0x7f63800a6cc0 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: dump: 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.828 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 5 monclient: authenticate success, global_id 24243 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 1 -- 172.21.15.105:0/3927529110 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63700aa810 con 0x7f63800a6cc0 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 1 -- 172.21.15.105:0/3927529110 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63700a9300 con 0x7f63800a6cc0 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 4 set_mon_vals no callback set 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385101640 10 monclient: dump: 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: shutdown 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/3927529110 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 msgr2=0x7f63800a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f63700098a0 tx=0x7f637001c410 comp rx=0 tx=0).stop 2024-09-17T16:35:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/3927529110 shutdown_connections 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63801396a0 0x7f6380139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f63800a6cc0 0x7f63800a70a0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- 172.21.15.105:0/3927529110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a68e0 0x7f63800a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/3927529110 >> 172.21.15.105:0/3927529110 conn(0x7f63800a96e0 msgr2=0x7f6380068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/3927529110 shutdown_connections 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/3927529110 wait complete. 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: build_initial_monmap 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: monmap: 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 librados: starting msgr at 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 librados: starting objecter 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 Processor -- start 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- start start 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 librados: setting wanted keys 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 librados: calling monclient init 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: init 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:09.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: _add_conns ranks=[8,4,2,3,6,7,1,0,5] 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 0x7f6380137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): picked mon.noname-e con 0x7f63800a6cc0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f63801396a0 0x7f6380138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): picked mon.noname-c con 0x7f63801396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f638013d5f0 0x7f638013da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): picked mon.noname-b con 0x7f638013d5f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63800a75e0 con 0x7f638013d5f0 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6380069500 con 0x7f63801396a0 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f63800a4a10 con 0x7f63800a6cc0 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient(hunting): authenticate will time out at 1393.078538s 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 0x7f6380137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f638013d5f0 0x7f638013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 0x7f6380137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:35392/0 (socket says 172.21.15.105:35392) 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 1 -- 172.21.15.105:0/2845643750 learned_addr learned my addr 172.21.15.105:0/2845643750 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): get_auth_request con 0x7f63800a6cc0 auth_method 0 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f63801396a0 0x7f6380138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f638013d5f0 0x7f638013da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57256/0 (socket says 172.21.15.105:57256) 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): get_auth_request con 0x7f638013d5f0 auth_method 0 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): get_auth_request con 0x7f63801396a0 auth_method 0 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386904640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient(hunting): handle_auth_done global_id 54166 payload 824 2024-09-17T16:35:09.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 msgr2=0x7f6380137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 0x7f6380137e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f638013d5f0 msgr2=0x7f638013da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f638013d5f0 0x7f638013da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient: _finish_hunting 0 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 monclient: found mon.noname-c 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6380122f20 con 0x7f63801396a0 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6386103640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f63800a6cc0 0x7f6380137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6385902640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f63801396a0 0x7f6380138350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6380069c30 tx=0x7f63700191b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 client.?.objecter ms_handle_connect 0x7f63801396a0 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63700b9070 con 0x7f63801396a0 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: dump: 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.840 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63700044b0 con 0x7f63801396a0 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63700b0040 con 0x7f63801396a0 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6376ffd640 10 monclient: dump: 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6384900640 4 set_mon_vals no callback set 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 5 monclient: authenticate success, global_id 54166 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: _renew_subs 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.810+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6380123820 con 0x7f63801396a0 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 10 client.54166.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 10 monclient: _renew_subs 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f638013fcd0 con 0x7f63801396a0 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 1 librados: init done 2024-09-17T16:35:09.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 0x7f638013df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f63800a7760 con 0x7f63800a96e0 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6370004040 con 0x7f63801396a0 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f636403e550 0x7f6364040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 0x7f638013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(326..326 src has 1..326) ==== 5801+0+0 (secure 0 0 0) 0x7f63700e9260 con 0x7f63801396a0 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 client.54166.objecter ms_dispatch 0x7f6380000ba0 osd_map(326..326 src has 1..326) 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 3 client.54166.objecter handle_osd_map got epochs [326,326] > 0 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 3 client.54166.objecter handle_osd_map decoding full epoch 326 2024-09-17T16:35:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient: get_auth_request con 0x7f63800a96e0 auth_method 0 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 20 client.54166.objecter dump_active .. 0 homeless 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386904640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f636403e550 0x7f6364040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386904640 10 monclient: get_auth_request con 0x7f636403e550 auth_method 0 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 10 monclient(hunting): handle_auth_done global_id 54111 payload 824 2024-09-17T16:35:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6380139c90 con 0x7f63800a96e0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 client.54166.objecter ms_handle_connect 0x7f63800a96e0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 client.54166.objecter resend_mon_ops 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386103640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 0x7f638013df70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f637c010cb0 tx=0x7f637c00e5a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 client.54166.objecter ms_handle_connect 0x7f636403e550 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6386904640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f636403e550 0x7f6364040a10 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 crypto rx=0x7f63800a7a80 tx=0x7f6378009780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f637c018070 con 0x7f63800a96e0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 monclient: dump: 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f637c01d450 con 0x7f63800a96e0 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 msgr2=0x7f638013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 0x7f638013df70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f637c010cb0 tx=0x7f637c00e5a0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 msgr2=0x7f638013df70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.814+0000 7f6376ffd640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63800a96e0 0x7f638013df70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6388367640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6388367640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 0x7f63800a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6388367640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6380051e80 con 0x7f638013e4b0 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 0x7f63800a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient: get_auth_request con 0x7f638013e4b0 auth_method 0 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 10 monclient(hunting): handle_auth_done global_id 54120 payload 824 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 1 -- 172.21.15.105:0/2845643750 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f63800a70a0 con 0x7f638013e4b0 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6386103640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 0x7f63800a0a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f637c010cb0 tx=0x7f637c0046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 client.54166.objecter ms_handle_connect 0x7f638013e4b0 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 client.54166.objecter resend_mon_ops 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f637c004530 con 0x7f638013e4b0 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 monclient: dump: 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f637c01f2e0 con 0x7f638013e4b0 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 msgr2=0x7f63800a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 0x7f63800a0a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f637c010cb0 tx=0x7f637c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 msgr2=0x7f63800a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.822+0000 7f6376ffd640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f638013e4b0 0x7f63800a0a20 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f637c010cb0 tx=0x7f637c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16831, 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:35:09.857 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.858 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.859 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.860 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:09.861 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:09.862 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.863 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.864 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.865 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.869 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.871 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.876 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:09.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:09.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.879 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:09.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.881 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:09.884 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 10 librados: watch_flush enter 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 10 librados: watch_flush exit 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 20 client.54166.objecter shutdown clearing up homeless session... 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 10 client.54166.objecter successfully canceled tick 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f636403e550 msgr2=0x7f6364040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f636403e550 0x7f6364040a10 secure :-1 s=READY pgs=3491 cs=0 l=1 rev1=1 crypto rx=0x7f63800a7a80 tx=0x7f6378009780 comp rx=0 tx=0).stop 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 10 monclient: shutdown 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f63801396a0 msgr2=0x7f6380138350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:09.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 --2- 172.21.15.105:0/2845643750 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f63801396a0 0x7f6380138350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6380069c30 tx=0x7f63700191b0 comp rx=0 tx=0).stop 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6386904640 1 -- 172.21.15.105:0/2845643750 reap_dead start 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6386904640 1 -- 172.21.15.105:0/2845643750 reap_dead start 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 shutdown_connections 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 >> 172.21.15.105:0/2845643750 conn(0x7f63800a68e0 msgr2=0x7f63800a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 shutdown_connections 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 -- 172.21.15.105:0/2845643750 wait complete. 2024-09-17T16:35:09.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:09.826+0000 7f6388367640 1 librados: shutdown 2024-09-17T16:35:09.888 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:09.889 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:35:09.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: get_monmap_and_config 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: monmap: 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.007 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:10.004037+0000 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:10.004037+0000 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:10.009 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 Processor -- start 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 -- start start 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: init 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient: _add_conns ranks=[1,7,8,0,5,6,9,2,10,3,4] 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff5700a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 0x7ff57013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-i con 0x7ff5700a7520 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-e con 0x7ff57013f180 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5700a4010 con 0x7ff5700a6a30 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff5700a2ea0 con 0x7ff5700a7520 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff5700a8920 con 0x7ff57013f180 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff57821c640 10 monclient(hunting): authenticate will time out at 1393.269742s 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:35408/0 (socket says 172.21.15.105:35408) 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 -- 172.21.15.105:0/212201768 learned_addr learned my addr 172.21.15.105:0/212201768 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): get_auth_request con 0x7ff57013f180 auth_method 0 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 0x7ff57013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 10 monclient(hunting): get_auth_request con 0x7ff5700a7520 auth_method 0 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient(hunting): handle_auth_done global_id 54152 payload 274 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 -- 172.21.15.105:0/212201768 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 msgr2=0x7ff57013ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 0x7ff57013ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 -- 172.21.15.105:0/212201768 >> v1:172.21.15.160:6789/0 conn(0x7ff5700a6a30 legacy=0x7ff5700a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 monclient: found mon.noname-e 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5767b9640 1 -- 172.21.15.105:0/212201768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57006ed00 con 0x7ff57013f180 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.002+0000 7ff5757b7640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 0x7ff57013ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff560009e80 tx=0x7ff56001c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=457500cd733cb1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 1 -- 172.21.15.105:0/212201768 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5600090d0 con 0x7ff57013f180 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: dump: 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 1 -- 172.21.15.105:0/212201768 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5600a6970 con 0x7ff57013f180 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 5 monclient: authenticate success, global_id 54152 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 4 set_mon_vals no callback set 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 1 -- 172.21.15.105:0/212201768 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5600a6b60 con 0x7ff57013f180 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff574fb6640 10 monclient: dump: 2024-09-17T16:35:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.026 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: shutdown 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/212201768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 msgr2=0x7ff570141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff560009e80 tx=0x7ff56001c910 comp rx=0 tx=0).stop 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/212201768 shutdown_connections 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff57013f180 0x7ff570141570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- 172.21.15.105:0/212201768 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff5700a7520 0x7ff57013ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/212201768 >> 172.21.15.105:0/212201768 conn(0x7ff5700ac860 msgr2=0x7ff5700a8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/212201768 shutdown_connections 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/212201768 wait complete. 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: monmap: 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.028 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 librados: starting msgr at 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 librados: starting objecter 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 Processor -- start 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- start start 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 librados: setting wanted keys 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 librados: calling monclient init 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: init 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: _add_conns ranks=[0,1,7,8,2,4,5,6,3] 2024-09-17T16:35:10.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 0x7ff5701243d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-a con 0x7ff5700a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5700a7520 0x7ff570124910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-f con 0x7ff5700a7520 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff57013f180 0x7ff570124e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): picked mon.noname-i con 0x7ff57013f180 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5700a7c40 con 0x7ff5700a6a30 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff57006eac0 con 0x7ff5700a7520 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff5701442d0 con 0x7ff57013f180 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient(hunting): authenticate will time out at 1393.275050s 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 0x7ff5701243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 0x7ff5701243d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52818/0 (socket says 172.21.15.105:52818) 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 -- 172.21.15.105:0/981671692 learned_addr learned my addr 172.21.15.105:0/981671692 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): get_auth_request con 0x7ff5700a6a30 auth_method 0 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff57013f180 0x7ff570124e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 10 monclient(hunting): get_auth_request con 0x7ff57013f180 auth_method 0 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5700a7520 0x7ff570124910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5767b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient(hunting): handle_auth_done global_id 35325 payload 824 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff57013f180 msgr2=0x7ff570124e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff57013f180 0x7ff570124e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5700a7520 msgr2=0x7ff570124910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5700a7520 0x7ff570124910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 monclient: found mon.noname-a 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57006db80 con 0x7ff5700a6a30 2024-09-17T16:35:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5757b7640 10 monclient: get_auth_request con 0x7ff5700a7520 auth_method 0 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5700a7520 0x7ff570124910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff575fb8640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 0x7ff5701243d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff56800c450 tx=0x7ff56800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 client.?.objecter ms_handle_connect 0x7ff5700a6a30 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff568012aa0 con 0x7ff5700a6a30 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: dump: 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.033 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff568017070 con 0x7ff5700a6a30 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 5 monclient: authenticate success, global_id 35325 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: _renew_subs 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff570127ec0 con 0x7ff5700a6a30 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff568012c80 con 0x7ff5700a6a30 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff57821c640 10 client.35325.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56e7fc640 10 monclient: dump: 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.035 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.006+0000 7ff56ffff640 4 set_mon_vals no callback set 2024-09-17T16:35:10.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 10 monclient: _renew_subs 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff570140bd0 con 0x7ff5700a6a30 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 1 librados: init done 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 0x7ff5701d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff570051e80 con 0x7ff570127ae0 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff568012630 con 0x7ff5700a6a30 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 0x7ff5701d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient: get_auth_request con 0x7ff570127ae0 auth_method 0 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff55c03e580 0x7ff55c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(326..326 src has 1..326) ==== 5801+0+0 (secure 0 0 0) 0x7ff56804fa90 con 0x7ff5700a6a30 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 client.35325.objecter ms_dispatch 0x7ff570000ba0 osd_map(326..326 src has 1..326) 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 3 client.35325.objecter handle_osd_map got epochs [326,326] > 0 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 3 client.35325.objecter handle_osd_map decoding full epoch 326 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 20 client.35325.objecter dump_active .. 0 homeless 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5767b9640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff55c03e580 0x7ff55c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5767b9640 10 monclient: get_auth_request con 0x7ff55c03e580 auth_method 0 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 10 monclient(hunting): handle_auth_done global_id 19868 payload 824 2024-09-17T16:35:10.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5701286a0 con 0x7ff570127ae0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 client.35325.objecter ms_handle_connect 0x7ff55c03e580 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5767b9640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff55c03e580 0x7ff55c040a40 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7ff5600ae000 tx=0x7ff5600a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 client.35325.objecter ms_handle_connect 0x7ff570127ae0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 client.35325.objecter resend_mon_ops 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 0x7ff5701d19a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff5700a7bd0 tx=0x7ff56400f8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff564018070 con 0x7ff570127ae0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 monclient: dump: 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.039 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff564014690 con 0x7ff570127ae0 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 msgr2=0x7ff5701d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 0x7ff5701d19a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff5700a7bd0 tx=0x7ff56400f8f0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 msgr2=0x7ff5701d19a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:10.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.010+0000 7ff56e7fc640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff570127ae0 0x7ff5701d19a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff57821c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff57821c640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 0x7ff57002d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff57821c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff570077340 con 0x7ff57002d450 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 0x7ff57002d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient: get_auth_request con 0x7ff57002d450 auth_method 0 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 10 monclient(hunting): handle_auth_done global_id 19877 payload 824 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 1 -- 172.21.15.105:0/981671692 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff57006df40 con 0x7ff57002d450 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff5757b7640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 0x7ff57002d860 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff5700a7bd0 tx=0x7ff5640046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 client.35325.objecter ms_handle_connect 0x7ff57002d450 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 client.35325.objecter resend_mon_ops 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff564018050 con 0x7ff57002d450 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 monclient: dump: 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.044 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9289 (secure 0 0 0) 0x7ff5640135f0 con 0x7ff57002d450 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 msgr2=0x7ff57002d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 0x7ff57002d860 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff5700a7bd0 tx=0x7ff5640046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 msgr2=0x7ff57002d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.018+0000 7ff56e7fc640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff57002d450 0x7ff57002d860 secure :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff5700a7bd0 tx=0x7ff5640046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:10.046 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 457625, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.047 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.048 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.049 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:10.050 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.051 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:10.053 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.056 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:10.058 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.063 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:10.065 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.067 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:10.068 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:35:10.069 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.069 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.069 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:10.069 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 10 librados: watch_flush enter 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 10 librados: watch_flush exit 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 20 client.35325.objecter shutdown clearing up homeless session... 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 10 client.35325.objecter successfully canceled tick 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff55c03e580 msgr2=0x7ff55c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff55c03e580 0x7ff55c040a40 secure :-1 s=READY pgs=3492 cs=0 l=1 rev1=1 crypto rx=0x7ff5600ae000 tx=0x7ff5600a7000 comp rx=0 tx=0).stop 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 10 monclient: shutdown 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 msgr2=0x7ff5701243d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 --2- 172.21.15.105:0/981671692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5700a6a30 0x7ff5701243d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff56800c450 tx=0x7ff56800c910 comp rx=0 tx=0).stop 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff5767b9640 1 -- 172.21.15.105:0/981671692 reap_dead start 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff5767b9640 1 -- 172.21.15.105:0/981671692 reap_dead start 2024-09-17T16:35:10.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 shutdown_connections 2024-09-17T16:35:10.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 >> 172.21.15.105:0/981671692 conn(0x7ff5700ac860 msgr2=0x7ff5700aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 shutdown_connections 2024-09-17T16:35:10.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 -- 172.21.15.105:0/981671692 wait complete. 2024-09-17T16:35:10.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.022+0000 7ff57821c640 1 librados: shutdown 2024-09-17T16:35:10.072 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:10.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:35:10.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:35:10.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: get_monmap_and_config 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: monmap: 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:10.192802+0000 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:10.192802+0000 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.196 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:10.197 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 Processor -- start 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 -- start start 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: init 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient: _add_conns ranks=[7,6,1,3,5,4,2,10,8,0,9] 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb540a2de0 0x7ffb540a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-i con 0x7ffb540a2de0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-d con 0x7ffb540a3700 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffb540a3ae0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb540a20b0 con 0x7ffb540a3ae0 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5409c9a0 con 0x7ffb540a3700 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb540a2770 con 0x7ffb540a2de0 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb588f3640 10 monclient(hunting): authenticate will time out at 1393.459210s 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42830/0 (socket says 172.21.15.105:42830) 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 -- 172.21.15.105:0/961478242 learned_addr learned my addr 172.21.15.105:0/961478242 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): get_auth_request con 0x7ffb540a3700 auth_method 0 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb5259c640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb540a2de0 0x7ffb540a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb5259c640 10 monclient(hunting): get_auth_request con 0x7ffb540a2de0 auth_method 0 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_done global_id 24252 payload 274 2024-09-17T16:35:10.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 -- 172.21.15.105:0/961478242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb540a2de0 msgr2=0x7ffb540a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb540a2de0 0x7ffb540a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 -- 172.21.15.105:0/961478242 >> v1:172.21.15.160:6789/0 conn(0x7ffb540a3ae0 legacy=0x7ffb5409b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 monclient: found mon.noname-d 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.190+0000 7ffb51d9b640 1 -- 172.21.15.105:0/961478242 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb540a24f0 con 0x7ffb540a3700 2024-09-17T16:35:10.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb51d9b640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098d0 tx=0x7ffb4801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=394be89c67fa10cf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 1 -- 172.21.15.105:0/961478242 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480b0070 con 0x7ffb540a3700 2024-09-17T16:35:10.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:10.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: dump: 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 1 -- 172.21.15.105:0/961478242 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb480aa810 con 0x7ffb540a3700 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 5 monclient: authenticate success, global_id 24252 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 1 -- 172.21.15.105:0/961478242 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480a9300 con 0x7ffb540a3700 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 4 set_mon_vals no callback set 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb5159a640 10 monclient: dump: 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.213 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 10 monclient: shutdown 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 -- 172.21.15.105:0/961478242 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 msgr2=0x7ffb5409b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098d0 tx=0x7ffb4801c440 comp rx=0 tx=0).stop 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 -- 172.21.15.105:0/961478242 shutdown_connections 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ffb540a3700 0x7ffb5409b050 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 --2- 172.21.15.105:0/961478242 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb540a2de0 0x7ffb540a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 -- 172.21.15.105:0/961478242 >> 172.21.15.105:0/961478242 conn(0x7ffb540a83e0 msgr2=0x7ffb540a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 -- 172.21.15.105:0/961478242 shutdown_connections 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.194+0000 7ffb588f3640 1 -- 172.21.15.105:0/961478242 wait complete. 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: monmap: 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.215 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 librados: starting msgr at 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 librados: starting objecter 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 Processor -- start 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- start start 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 librados: setting wanted keys 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 librados: calling monclient init 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: init 2024-09-17T16:35:10.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _add_conns ranks=[3,4,5,0,2,8,6,7,1] 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb540a2de0 0x7ffb5409b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-g con 0x7ffb540a2de0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 0x7ffb5409b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-c con 0x7ffb540a3700 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb540a3ae0 0x7ffb5409f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): picked mon.noname-h con 0x7ffb540a3ae0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ffb541421d0 con 0x7ffb540a2de0 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ffb5409beb0 con 0x7ffb540a3700 2024-09-17T16:35:10.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ffb54142350 con 0x7ffb540a3ae0 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): authenticate will time out at 1393.465299s 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 0x7ffb5409b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 0x7ffb5409b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36380/0 (socket says 172.21.15.105:36380) 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 -- 172.21.15.105:0/1550722866 learned_addr learned my addr 172.21.15.105:0/1550722866 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): get_auth_request con 0x7ffb540a3700 auth_method 0 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb540a2de0 0x7ffb5409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb540a3ae0 0x7ffb5409f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): get_auth_request con 0x7ffb540a2de0 auth_method 0 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 10 monclient(hunting): get_auth_request con 0x7ffb540a3ae0 auth_method 0 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient(hunting): handle_auth_done global_id 54175 payload 824 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb540a3ae0 msgr2=0x7ffb5409f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb540a3ae0 0x7ffb5409f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb540a2de0 msgr2=0x7ffb5409b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb540a2de0 0x7ffb5409b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 monclient: found mon.noname-c 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5409bad0 con 0x7ffb540a3700 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb51d9b640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 0x7ffb5409b590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb4801c270 tx=0x7ffb480191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 client.?.objecter ms_handle_connect 0x7ffb540a3700 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480b9070 con 0x7ffb540a3700 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: dump: 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb480044b0 con 0x7ffb540a3700 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 5 monclient: authenticate success, global_id 54175 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _renew_subs 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5409d680 con 0x7ffb540a3700 2024-09-17T16:35:10.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480b0040 con 0x7ffb540a3700 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 monclient: dump: 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.223 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb50d99640 4 set_mon_vals no callback set 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 client.54175.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _renew_subs 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5409dbd0 con 0x7ffb540a3700 2024-09-17T16:35:10.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 librados: init done 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ffb48004040 con 0x7ffb540a3700 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 0x7ffb540a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb54051e80 con 0x7ffb5409fe90 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb3403e5a0 0x7ffb34040a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(326..326 src has 1..326) ==== 5801+0+0 (secure 0 0 0) 0x7ffb480e90d0 con 0x7ffb540a3700 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 10 client.54175.objecter ms_dispatch 0x7ffb54000ba0 osd_map(326..326 src has 1..326) 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 3 client.54175.objecter handle_osd_map got epochs [326,326] > 0 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 3 client.54175.objecter handle_osd_map decoding full epoch 326 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 0x7ffb540a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient: get_auth_request con 0x7ffb5409fe90 auth_method 0 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb42ffd640 20 client.54175.objecter dump_active .. 0 homeless 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb3403e5a0 0x7ffb34040a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.198+0000 7ffb52d9d640 10 monclient: get_auth_request con 0x7ffb3403e5a0 auth_method 0 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb5259c640 10 monclient(hunting): handle_auth_done global_id 34810 payload 824 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb5259c640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb5413c5b0 con 0x7ffb5409fe90 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 client.54175.objecter ms_handle_connect 0x7ffb5409fe90 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 client.54175.objecter resend_mon_ops 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb5259c640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 0x7ffb540a0270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb4400f970 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 client.54175.objecter ms_handle_connect 0x7ffb3403e5a0 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb52d9d640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb3403e5a0 0x7ffb34040a60 secure :-1 s=READY pgs=3493 cs=0 l=1 rev1=1 crypto rx=0x7ffb540a0f00 tx=0x7ffb4c009410 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb44019020 con 0x7ffb5409fe90 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 monclient: dump: 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.227 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ffb44014670 con 0x7ffb5409fe90 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 msgr2=0x7ffb540a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 0x7ffb540a0270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb4400f970 comp rx=0 tx=0).stop 2024-09-17T16:35:10.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 msgr2=0x7ffb540a0270 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:10.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.202+0000 7ffb42ffd640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5409fe90 0x7ffb540a0270 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb4400f970 comp rx=0 tx=0).stop 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb588f3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb588f3640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 0x7ffb54138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb588f3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5406a1c0 con 0x7ffb54138760 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb5259c640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 0x7ffb54138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb5259c640 10 monclient: get_auth_request con 0x7ffb54138760 auth_method 0 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.206+0000 7ffb5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 10 monclient(hunting): handle_auth_done global_id 34819 payload 824 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 1 -- 172.21.15.105:0/1550722866 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffb540a07b0 con 0x7ffb54138760 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 client.54175.objecter ms_handle_connect 0x7ffb54138760 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 client.54175.objecter resend_mon_ops 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb5259c640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 0x7ffb54138b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb440046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb44019040 con 0x7ffb54138760 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 monclient: dump: 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.232 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7ffb44004050 con 0x7ffb54138760 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 msgr2=0x7ffb54138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 0x7ffb54138b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb440046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 msgr2=0x7ffb54138b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:10.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb42ffd640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb54138760 0x7ffb54138b70 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400f4b0 tx=0x7ffb440046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 301012, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.236 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:10.237 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:10.238 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:10.241 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:10.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:10.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:10.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:10.254 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:10.255 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:10.256 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 10 librados: watch_flush enter 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 10 librados: watch_flush exit 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 20 client.54175.objecter shutdown clearing up homeless session... 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 10 client.54175.objecter successfully canceled tick 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb3403e5a0 msgr2=0x7ffb34040a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb3403e5a0 0x7ffb34040a60 secure :-1 s=READY pgs=3493 cs=0 l=1 rev1=1 crypto rx=0x7ffb540a0f00 tx=0x7ffb4c009410 comp rx=0 tx=0).stop 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 10 monclient: shutdown 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 msgr2=0x7ffb5409b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 --2- 172.21.15.105:0/1550722866 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ffb540a3700 0x7ffb5409b590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb4801c270 tx=0x7ffb480191e0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb52d9d640 1 -- 172.21.15.105:0/1550722866 reap_dead start 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb52d9d640 1 -- 172.21.15.105:0/1550722866 reap_dead start 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 shutdown_connections 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 >> 172.21.15.105:0/1550722866 conn(0x7ffb540a83e0 msgr2=0x7ffb5413f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 shutdown_connections 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 -- 172.21.15.105:0/1550722866 wait complete. 2024-09-17T16:35:10.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.210+0000 7ffb588f3640 1 librados: shutdown 2024-09-17T16:35:10.260 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:10.260 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:35:10.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:35:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 10 monclient: get_monmap_and_config 2024-09-17T16:35:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 10 monclient: monmap: 2024-09-17T16:35:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:10.370579+0000 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:10.370579+0000 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.374 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.375 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.375 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:10.375 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:10.375 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 1 Processor -- start 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 1 -- start start 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.366+0000 7f9334a37640 10 monclient: init 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient: _add_conns ranks=[9,10,7,8,2,6,3,4,5,0,1] 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93300a68e0 0x7f93300a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-a con 0x7f93300a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 0x7f93300a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-f con 0x7f93300a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f93301396a0 0x7f9330139b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-i con 0x7f93301396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f93300a3fc0 con 0x7f93300a68e0 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f93300a63c0 con 0x7f93300a4910 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f93300a7740 con 0x7f93301396a0 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient(hunting): authenticate will time out at 1393.636400s 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93300a68e0 0x7f93300a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f93300a68e0 0x7f93300a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52826/0 (socket says 172.21.15.105:52826) 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 -- 172.21.15.105:0/1467512506 learned_addr learned my addr 172.21.15.105:0/1467512506 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): get_auth_request con 0x7f93300a68e0 auth_method 0 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932ed9d640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f93301396a0 0x7f9330139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932ed9d640 10 monclient(hunting): get_auth_request con 0x7f93301396a0 auth_method 0 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932dd9b640 1 --2- 172.21.15.105:0/1467512506 >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 0x7f93300a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient(hunting): handle_auth_done global_id 35334 payload 274 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 -- 172.21.15.105:0/1467512506 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f93301396a0 msgr2=0x7f9330139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f93301396a0 0x7f9330139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 -- 172.21.15.105:0/1467512506 >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 msgr2=0x7f93300a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 --2- 172.21.15.105:0/1467512506 >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 0x7f93300a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 monclient: found mon.noname-a 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 -- 172.21.15.105:0/1467512506 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933006ebc0 con 0x7f93300a68e0 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932dd9b640 10 monclient: get_auth_request con 0x7f93300a4910 auth_method 0 2024-09-17T16:35:10.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932dd9b640 1 --2- 172.21.15.105:0/1467512506 >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 0x7f93300a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:10.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932e59c640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300a68e0 0x7f93300a6cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9330074500 tx=0x7f932801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf020b01e25defc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 1 -- 172.21.15.105:0/1467512506 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93280b0070 con 0x7f93300a68e0 2024-09-17T16:35:10.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:10.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: dump: 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.386 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 1 -- 172.21.15.105:0/1467512506 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93280aa810 con 0x7f93300a68e0 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 1 -- 172.21.15.105:0/1467512506 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93280a9300 con 0x7f93300a68e0 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 5 monclient: authenticate success, global_id 35334 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 4 set_mon_vals no callback set 2024-09-17T16:35:10.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f932d59a640 10 monclient: dump: 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.388 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 10 monclient: shutdown 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 -- 172.21.15.105:0/1467512506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300a68e0 msgr2=0x7f93300a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300a68e0 0x7f93300a6cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9330074500 tx=0x7f932801c3b0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 -- 172.21.15.105:0/1467512506 shutdown_connections 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f93301396a0 0x7f9330139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- 172.21.15.105:0/1467512506 >> v2:172.21.15.160:3300/0 conn(0x7f93300a4910 0x7f93300a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.370+0000 7f9334a37640 1 --2- 172.21.15.105:0/1467512506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93300a68e0 0x7f93300a6cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/1467512506 >> 172.21.15.105:0/1467512506 conn(0x7f9330068ba0 msgr2=0x7f9330068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/1467512506 shutdown_connections 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/1467512506 wait complete. 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: build_initial_monmap 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: monmap: 2024-09-17T16:35:10.390 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:10.393 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 librados: starting msgr at 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 librados: starting objecter 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 Processor -- start 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- start start 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 librados: setting wanted keys 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 librados: calling monclient init 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: init 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _add_conns ranks=[6,1,3,0,2,8,7,5,4] 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-d con 0x7f93300a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:10.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 0x7f93301d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-f con 0x7f93300a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 0x7f933013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): picked mon.noname-g con 0x7f93301396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f933006e630 con 0x7f93300a68e0 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f93300a7af0 con 0x7f93301396a0 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9330143320 con 0x7f93300a4910 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient(hunting): authenticate will time out at 1393.641508s 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42842/0 (socket says 172.21.15.105:42842) 2024-09-17T16:35:10.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 -- 172.21.15.105:0/3205819399 learned_addr learned my addr 172.21.15.105:0/3205819399 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): get_auth_request con 0x7f93300a4910 auth_method 0 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932dd9b640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 0x7f93301d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932ed9d640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 0x7f933013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient(hunting): handle_auth_done global_id 24261 payload 824 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 -- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 msgr2=0x7f933013d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 0x7f933013d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 -- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 msgr2=0x7f93301d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 0x7f93301d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 monclient: found mon.noname-d 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 -- 172.21.15.105:0/3205819399 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933013f3b0 con 0x7f93300a4910 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932dd9b640 10 monclient: get_auth_request con 0x7f93300a68e0 auth_method 0 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932ed9d640 10 monclient: get_auth_request con 0x7f93301396a0 auth_method 0 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932dd9b640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 0x7f93301d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932ed9d640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 0x7f933013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932e59c640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f93280097c0 tx=0x7f9328019150 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 client.?.objecter ms_handle_connect 0x7f93300a4910 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93280b9070 con 0x7f93300a4910 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: dump: 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.398 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: _finish_auth 0 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93280aae20 con 0x7f93300a4910 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 5 monclient: authenticate success, global_id 24261 2024-09-17T16:35:10.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _renew_subs 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933013fbb0 con 0x7f93300a4910 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93280b0040 con 0x7f93300a4910 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 monclient: dump: 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:10.400 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 client.24261.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _renew_subs 2024-09-17T16:35:10.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f932cd99640 4 set_mon_vals no callback set 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f933013ef10 con 0x7f93300a4910 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 librados: init done 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330139b10 con 0x7f93300a4910 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9328004040 con 0x7f93300a4910 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 0x7f9310040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(326..326 src has 1..326) ==== 5801+0+0 (secure 0 0 0) 0x7f93280e9020 con 0x7f93300a4910 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 10 client.24261.objecter ms_dispatch 0x7f9330000ba0 osd_map(326..326 src has 1..326) 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 3 client.24261.objecter handle_osd_map got epochs [326,326] > 0 2024-09-17T16:35:10.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 3 client.24261.objecter handle_osd_map decoding full epoch 326 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.374+0000 7f931effd640 20 client.24261.objecter dump_active .. 0 homeless 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.378+0000 7f932dd9b640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 0x7f9310040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.378+0000 7f932dd9b640 10 monclient: get_auth_request con 0x7f931003e540 auth_method 0 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.378+0000 7f931effd640 10 client.24261.objecter ms_handle_connect 0x7f931003e540 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.378+0000 7f932dd9b640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 0x7f9310040a00 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7f9324006fd0 tx=0x7f9324008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.382+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93280160f0 con 0x7f93300a4910 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.382+0000 7f931effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:10.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.382+0000 7f931effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:10.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.626+0000 7f9334a37640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:35:10.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.626+0000 7f9334a37640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:35:10.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.626+0000 7f9334a37640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:10.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.626+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f93300abac0 con 0x7f93300a4910 2024-09-17T16:35:10.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f931effd640 1 -- 172.21.15.105:0/3205819399 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f9328003b40 con 0x7f93300a4910 2024-09-17T16:35:10.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f931effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:35:10.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f931effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=12,kv=10,logm=64} crc {auth=465565380,config=692619927,health=2941319762,kv=2863082231,logm=266114117}) 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.642 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=12,config=2,health=12,kv=10,logm=64} crc {auth=465565380,config=692619927,health=2941319762,kv=2863082231,logm=3851104039}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=12,kv=10,logm=66} crc {auth=2268629661,config=692619927,health=2941319762,kv=2863082231,logm=293012257}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=12,kv=10,logm=65} crc {auth=294342712,config=692619927,health=2941319762,kv=2863082231,logm=2579710658}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=12,config=2,health=12,kv=10,logm=64} crc {auth=465565380,config=692619927,health=3455919192,kv=2863082231,logm=3335902428}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=11,kv=9,logm=66} crc {auth=465565380,config=692619927,health=3778872965,kv=732018829,logm=2033661555}) 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.643 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.644 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=11,config=1,health=12,kv=10,logm=66} crc {auth=2671702106,config=3678701417,health=2941319762,kv=2863082231,logm=1978718619}) 2024-09-17T16:35:10.644 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.644 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=8,logm=67} crc {auth=34331945,config=692619927,health=2941319762,kv=1989568803,logm=1402769979}) 2024-09-17T16:35:10.644 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.636+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=465565380,config=692619927,health=2941319762,kv=2863082231,logm=2134974979}) 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f9334a37640 10 librados: watch_flush enter 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f9334a37640 10 librados: watch_flush exit 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f9334a37640 20 client.24261.objecter shutdown clearing up homeless session... 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.634+0000 7f9334a37640 10 client.24261.objecter successfully canceled tick 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 msgr2=0x7f9310040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 0x7f9310040a00 secure :-1 s=READY pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7f9324006fd0 tx=0x7f9324008040 comp rx=0 tx=0).stop 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 10 monclient: shutdown 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 msgr2=0x7f93301d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:10.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f93280097c0 tx=0x7f9328019150 comp rx=0 tx=0).stop 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 shutdown_connections 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f931003e540 0x7f9310040a00 secure :-1 s=CLOSED pgs=3494 cs=0 l=1 rev1=1 crypto rx=0x7f9324006fd0 tx=0x7f9324008040 comp rx=0 tx=0).stop 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93301396a0 0x7f933013d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93300a68e0 0x7f93301d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 --2- 172.21.15.105:0/3205819399 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f93300a4910 0x7f93301d14d0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 >> 172.21.15.105:0/3205819399 conn(0x7f9330068ba0 msgr2=0x7f93300a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 shutdown_connections 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 -- 172.21.15.105:0/3205819399 wait complete. 2024-09-17T16:35:10.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:10.638+0000 7f9334a37640 1 librados: shutdown 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095295483}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=505108578}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2666226478}) 2024-09-17T16:35:10.649 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2653931223}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=367973108}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.640+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2725699154}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=436933954}) 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.650 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853676386}) 2024-09-17T16:35:10.651 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.644+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2916284198}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1823856350}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2668352398}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2510196820}) 2024-09-17T16:35:10.653 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3563007706}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4254086806}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=710817262}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=16404762}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2738301549}) 2024-09-17T16:35:10.654 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.648+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3403769419}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3151179528}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2108560043}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=673863930}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.657 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=50835598}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1455621363}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438886368}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1931183872}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71217104}) 2024-09-17T16:35:10.658 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.652+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2173765515}) 2024-09-17T16:35:10.660 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421951683}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4252597441}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2136259698}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=873665817}) 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.661 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2349103985}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2153470752}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3255008471}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1169033761}) 2024-09-17T16:35:10.662 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.656+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3545793580}) 2024-09-17T16:35:10.663 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=855029448}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=915822482}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1631430683}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1222976942}) 2024-09-17T16:35:10.665 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=671683414}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2106237254}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2879225863}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920937967}) 2024-09-17T16:35:10.666 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.660+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1508105042}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436764691}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1200493055}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.668 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1526276490}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1909410815}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1810923749}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3228776888}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2205895911}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4226665234}) 2024-09-17T16:35:10.669 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.664+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=805933607}) 2024-09-17T16:35:10.671 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.671 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.671 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4205021452}) 2024-09-17T16:35:10.671 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.671 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1731616998}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3048809178}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1348358506}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=136965383}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.672 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2245339931}) 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3856872175}) 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2081468543}) 2024-09-17T16:35:10.673 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.668+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=331624956}) 2024-09-17T16:35:10.675 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.675 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=148418750}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1093307045}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=654209847}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2702312932}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2403040694}) 2024-09-17T16:35:10.676 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3065117033}) 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2925254262}) 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=29617679}) 2024-09-17T16:35:10.677 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.672+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1387470546}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2121201242}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3428255538}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.679 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4282718597}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2021881466}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=139591506}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352278016}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=733741773}) 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.680 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3900070840}) 2024-09-17T16:35:10.681 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2709908947}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568638376}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693857706}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.682 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3390581242}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.676+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3930566239}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2405589040}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=38822551}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3285436742}) 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.683 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2646396151}) 2024-09-17T16:35:10.684 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2068991883}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2815292918}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3320483387}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.686 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=95143667}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3680658571}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2834506675}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1322772900}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=709487772}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=246445041}) 2024-09-17T16:35:10.687 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.680+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3475306149}) 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4073540267}) 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.690 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=128989924}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2827593433}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1193396305}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=544280292}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4268807434}) 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.691 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.692 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2421406694}) 2024-09-17T16:35:10.692 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.692 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309494767}) 2024-09-17T16:35:10.692 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.684+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3229188081}) 2024-09-17T16:35:10.696 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1747689383}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1290076467}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=39641709}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3472378513}) 2024-09-17T16:35:10.697 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4092717577}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1484513434}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2732824703}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=262833}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.688+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2428786461}) 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.698 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3371231590}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633394852}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1195976639}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.699 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2741169699}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3233159369}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=643984802}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3957374656}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301301851}) 2024-09-17T16:35:10.700 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.692+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511583480}) 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3375064977}) 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943786876}) 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.703 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2976517133}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1055901569}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1226307988}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2579995099}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.704 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3672098791}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855154451}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.696+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=704627995}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3678984491}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3272292742}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3530146365}) 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.705 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1626128947}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3811226609}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216733122}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2876843744}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095373445}) 2024-09-17T16:35:10.706 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1936164092}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799680127}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1399277919}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3875861261}) 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.712 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=110143831}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3602944286}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2569613314}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3832967736}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2992715293}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.700+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=860044251}) 2024-09-17T16:35:10.713 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1352995758}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2995027691}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2542184265}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3567327628}) 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.714 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=680518707}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2475327071}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1419155900}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202548702}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.704+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1934373531}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3908874029}) 2024-09-17T16:35:10.715 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3900432006}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=304071576}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4150426236}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3672118682}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2357020988}) 2024-09-17T16:35:10.716 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.717 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.717 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2476451585}) 2024-09-17T16:35:10.717 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.717 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1785569349}) 2024-09-17T16:35:10.717 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3396814415}) 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2361205042}) 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.728 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2847730653}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730799016}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=835321010}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3884732197}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=594181118}) 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.729 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.708+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2649052413}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417753741}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=334899066}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1334509990}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2348760978}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=749345448}) 2024-09-17T16:35:10.730 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2380101079}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3397329549}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1898299263}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2894693727}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265929389}) 2024-09-17T16:35:10.731 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.712+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1136789343}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=897406747}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=670356294}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251620976}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3536394698}) 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.732 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=504316409}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2048820156}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1351969132}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=465207103}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2092363038}) 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.733 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2478478364}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2600157250}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2157077012}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1794189141}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=630227747}) 2024-09-17T16:35:10.734 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3155403632}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.716+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2335056021}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460360386}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3628489472}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379032618}) 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.735 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4121675440}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3715585608}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2905866898}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2552186549}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1127188492}) 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.736 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554711583}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413070075}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.720+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3138217447}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430076412}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919232324}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=325762699}) 2024-09-17T16:35:10.737 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2967574695}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=101271786}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1547554899}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1793504327}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169187585}) 2024-09-17T16:35:10.738 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=339700685}) 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=519636041}) 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1611435593}) 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.758 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=499305584}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2309572050}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1860417245}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4016694258}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.724+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1911670265}) 2024-09-17T16:35:10.759 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389130877}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1329288675}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183991393}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3896104859}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2921165835}) 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.760 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2879160841}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2902340873}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=594288306}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2728538588}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2600559172}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.728+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2885268722}) 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.761 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2521178936}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=228308375}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1927003259}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1985725286}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2891770073}) 2024-09-17T16:35:10.762 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3383667382}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2702570226}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677206649}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2108522191}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1809647491}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.763 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1830282098}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2177248502}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2044665790}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.764 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3633643986}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=138951525}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3062556447}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2616187072}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.732+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3014895732}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3819457696}) 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.765 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2817243049}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2362240030}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1723090985}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2681593465}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=384041340}) 2024-09-17T16:35:10.766 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1765499207}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2932779411}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=139947869}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592490362}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3722488354}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.767 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4257507952}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3780656928}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=302781460}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1176171621}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=632576538}) 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.768 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3482328512}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.736+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1680847876}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2620937609}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3799041564}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054720535}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137245476}) 2024-09-17T16:35:10.769 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3586039019}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1800643555}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2648546354}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269307343}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.740+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361983572}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.770 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3343039376}) 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2473578721}) 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=494116326}) 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.771 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=882510059}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1849469169}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852488885}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4005713228}) 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.772 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=749361055}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.744+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1590715000}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=264484322}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=628972615}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321711157}) 2024-09-17T16:35:10.773 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2429302437}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=21879382}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2858730884}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.774 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1571686642}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2890735849}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.748+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2720423849}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1833647831}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.775 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=6944751}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=651204475}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1384819316}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=932275250}) 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.776 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2279913976}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2794749885}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3269698359}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3151198395}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3729318916}) 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.777 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1664895499}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=93436149}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=499170531}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=150218571}) 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.778 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008375829}) 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=389238116}) 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273260217}) 2024-09-17T16:35:10.779 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.752+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1779331943}) 2024-09-17T16:35:10.817 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.817 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3169860130}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1617682063}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170996947}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2989327500}) 2024-09-17T16:35:10.818 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=17584469}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=13313000}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2546306493}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036440132}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.756+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2670497588}) 2024-09-17T16:35:10.819 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3544008417}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1555036392}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=940365370}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2279833151}) 2024-09-17T16:35:10.820 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2581505177}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3041670724}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2918525387}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068219929}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1440328464}) 2024-09-17T16:35:10.821 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1333375055}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2682655633}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1736320299}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3081797612}) 2024-09-17T16:35:10.822 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3167461605}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4070469950}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3501512610}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3353091542}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.760+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1843892708}) 2024-09-17T16:35:10.823 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=468795440}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2479469927}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=950670509}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.824 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2743278467}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3691746003}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=960213970}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4101966902}) 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.825 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241008177}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.764+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=669829773}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1636918752}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1327663981}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3010976110}) 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.826 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3308438295}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2113174336}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2836101612}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2972515074}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1980549439}) 2024-09-17T16:35:10.827 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.768+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1255619214}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1694286289}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3379692218}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1137051929}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.828 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=376067998}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=628844765}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=974645019}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4257087677}) 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.829 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681403626}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3049810922}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3566664695}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3194908970}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1039108901}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.830 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=457973372}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2257397791}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077132905}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=16052931}) 2024-09-17T16:35:10.831 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028146953}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.772+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917790229}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2653005815}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3498062109}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=116012197}) 2024-09-17T16:35:10.832 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4260504831}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=921388366}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351278422}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3865121430}) 2024-09-17T16:35:10.833 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4189079931}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4027024316}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=516173056}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1152140215}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2822115739}) 2024-09-17T16:35:10.834 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.776+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2239819595}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3264930721}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2950169411}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2658775391}) 2024-09-17T16:35:10.835 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246599322}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753116550}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684319224}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3920211378}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2778556465}) 2024-09-17T16:35:10.836 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=910032524}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1844548342}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3408169099}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1092994347}) 2024-09-17T16:35:10.837 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1718463797}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.780+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=22990254}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1094880081}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3121362591}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3019308044}) 2024-09-17T16:35:10.838 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1036723000}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=132975486}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445835169}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1669631012}) 2024-09-17T16:35:10.839 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330465323}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2611529492}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1573830308}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1477127406}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2770837879}) 2024-09-17T16:35:10.840 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3086831608}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1079220245}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1798955136}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.784+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.841 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3261115094}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3560955749}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=709333622}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=824733589}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1264149974}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.842 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547375922}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2280511351}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1419191006}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744069185}) 2024-09-17T16:35:10.843 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4117521983}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857599610}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.788+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212362572}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5} crc {logm=3553353875,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=2893357816}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=76,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=3} crc {logm=3160456446,mdsmap=1248375647,mgr=4234951994,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3560280948}) 2024-09-17T16:35:10.844 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5} crc {logm=493384039,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2208355494,mon_config_key=1855541877}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4} crc {logm=1549716,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=607353073,mon_config_key=1079238812}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5} crc {logm=363958897,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=2893357816}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.845 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4} crc {logm=1042098333,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=1079238812}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=3} crc {logm=216789756,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3560280948}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1929875777,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=559725414,mon_config_key=3563937658}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5} crc {logm=3748323099,mdsmap=1248375647,mgr=725307606,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3545282200,mon_config_key=2893357816}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28,osdmap=54} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=875478172,osdmap=2948419177}) 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.846 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25,osdmap=57} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=3805676163,osdmap=3754135869}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=56} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=3140514009,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=3329469417,osdmap=676632456}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=25,osdmap=57} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=132183166,osdmap=377135942}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=24,osdmap=59} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=352991311,osd_pg_creating=2339993056,osd_snap=3953006757,osdmap=2424998821}) 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.847 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=1,osd_metadata=6,osd_pg_creating=1,osd_snap=24,osdmap=60} crc {mon_config_key=506037468,monmap=1360384041,nvmeofgw=1228700967,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=923555200,osdmap=4059102278}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=26,osdmap=57} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=2339993056,osd_snap=1351421092,osdmap=2609106506}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=3,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=57} crc {mon_config_key=2541542488,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2339993056,osd_snap=2602378374,osdmap=3628687694}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.792+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=5,osd_snap=26,osdmap=58} crc {mon_config_key=1150695490,monmap=1360384041,nvmeofgw=598337686,osd_metadata=2719151891,osd_snap=1076048138,osdmap=1258301739}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.848 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2947954205}) 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1011473779}) 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2056919434}) 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.849 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1285830741}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1884222083}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=507889231}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3438828325}) 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.850 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3040643697}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3362319964}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=199331321}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3192840834}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3263955968}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.851 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4261862349}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=135243490}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1004396260}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=409105053}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=578144580}) 2024-09-17T16:35:10.852 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.796+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2514758047}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=770862249}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3543545190}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=193052298}) 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.853 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3141314925}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2791448516}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4176034520}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1204155735}) 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.854 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=215572519}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.800+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3960741984}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4052132891}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4106679288}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2705149914}) 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.855 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2631003550}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1946951769}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=415681867}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1467978618}) 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.856 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2569627738}) 2024-09-17T16:35:10.857 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.804+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1865898528}) 2024-09-17T16:35:10.858 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=22} crc {osdmap=507244775}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=23} crc {osdmap=3917202886}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=24} crc {osdmap=500299372}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.859 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=23} crc {osdmap=1192966048}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=25} crc {osdmap=1957732125}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=24} crc {osdmap=4250396796}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=23} crc {osdmap=3349750484}) 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:35:10.860 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=23} crc {osdmap=3783335211}) 2024-09-17T16:35:10.861 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:35:10.816+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=25} crc {osdmap=1957732125}) 2024-09-17T16:35:11.663 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:35:11.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:35:11.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: get_monmap_and_config 2024-09-17T16:35:11.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: build_initial_monmap 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: monmap: 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:11.804166+0000 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:11.804166+0000 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:11.808 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 Processor -- start 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 -- start start 2024-09-17T16:35:11.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: init 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient: _add_conns ranks=[10,3,2,6,0,4,7,9,1,8,5] 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb438139b20 0x7fb43809ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-f con 0x7fb438139b20 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb438139f00 0x7fb4380a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-g con 0x7fb438139f00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-b con 0x7fb4380a4c90 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb43806c310 con 0x7fb438139b20 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb43806c4e0 con 0x7fb4380a4c90 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fb43809e070 con 0x7fb438139f00 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43e610640 10 monclient(hunting): authenticate will time out at 1395.069961s 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57266/0 (socket says 172.21.15.105:57266) 2024-09-17T16:35:11.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 -- 172.21.15.105:0/2479462665 learned_addr learned my addr 172.21.15.105:0/2479462665 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): get_auth_request con 0x7fb4380a4c90 auth_method 0 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb438139f00 0x7fb4380a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): get_auth_request con 0x7fb438139f00 auth_method 0 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 1 --2- 172.21.15.105:0/2479462665 >> v2:172.21.15.160:3300/0 conn(0x7fb438139b20 0x7fb43809ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): get_auth_request con 0x7fb438139b20 auth_method 0 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient(hunting): handle_auth_done global_id 44561 payload 274 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 -- 172.21.15.105:0/2479462665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb438139f00 msgr2=0x7fb4380a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:11.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb438139f00 0x7fb4380a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 -- 172.21.15.105:0/2479462665 >> v2:172.21.15.160:3300/0 conn(0x7fb438139b20 msgr2=0x7fb43809ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 --2- 172.21.15.105:0/2479462665 >> v2:172.21.15.160:3300/0 conn(0x7fb438139b20 0x7fb43809ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient: _finish_hunting 0 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 monclient: found mon.noname-b 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.802+0000 7fb43cbad640 1 -- 172.21.15.105:0/2479462665 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43809dd90 con 0x7fb4380a4c90 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e50 tx=0x7fb43001c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a9a662b530e1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 1 -- 172.21.15.105:0/2479462665 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4300090d0 con 0x7fb4380a4c90 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: dump: 2024-09-17T16:35:11.825 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:11.826 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 5 monclient: authenticate success, global_id 44561 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 1 -- 172.21.15.105:0/2479462665 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4300a5ab0 con 0x7fb4380a4c90 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 4 set_mon_vals no callback set 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 1 -- 172.21.15.105:0/2479462665 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4300a5ca0 con 0x7fb4380a4c90 2024-09-17T16:35:11.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb436ffd640 10 monclient: dump: 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:11.828 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: shutdown 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/2479462665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 msgr2=0x7fb4380a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e50 tx=0x7fb43001c8f0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/2479462665 shutdown_connections 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4380a4c90 0x7fb4380a5140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- 172.21.15.105:0/2479462665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb438139f00 0x7fb4380a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- 172.21.15.105:0/2479462665 >> v2:172.21.15.160:3300/0 conn(0x7fb438139b20 0x7fb43809ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/2479462665 >> 172.21.15.105:0/2479462665 conn(0x7fb4380a93f0 msgr2=0x7fb4380a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/2479462665 shutdown_connections 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/2479462665 wait complete. 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: build_initial_monmap 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: monmap: 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 librados: starting msgr at 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 librados: starting objecter 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 Processor -- start 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- start start 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 librados: setting wanted keys 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 librados: calling monclient init 2024-09-17T16:35:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: init 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: _add_conns ranks=[0,7,4,1,8,6,5,2,3] 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-a con 0x7fb4380a4c90 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb438139b20 0x7fb4381d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-i con 0x7fb438139b20 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb438139f00 0x7fb43813a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): picked mon.noname-c con 0x7fb438139f00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb438143490 con 0x7fb4380a4c90 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb43806cde0 con 0x7fb438139f00 2024-09-17T16:35:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb438143610 con 0x7fb438139b20 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient(hunting): authenticate will time out at 1395.075008s 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52842/0 (socket says 172.21.15.105:52842) 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 -- 172.21.15.105:0/500643573 learned_addr learned my addr 172.21.15.105:0/500643573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): get_auth_request con 0x7fb4380a4c90 auth_method 0 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb438139f00 0x7fb43813a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): get_auth_request con 0x7fb438139f00 auth_method 0 2024-09-17T16:35:11.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb438139b20 0x7fb4381d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43cbad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): get_auth_request con 0x7fb438139b20 auth_method 0 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:11.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient(hunting): handle_auth_done global_id 35343 payload 824 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 -- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb438139b20 msgr2=0x7fb4381d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb438139b20 0x7fb4381d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 -- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb438139f00 msgr2=0x7fb43813a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb438139f00 0x7fb43813a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 monclient: found mon.noname-a 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 -- 172.21.15.105:0/500643573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43813e010 con 0x7fb4380a4c90 2024-09-17T16:35:11.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb437fff640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e8e0 tx=0x7fb42000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 client.?.objecter ms_handle_connect 0x7fb4380a4c90 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb420017070 con 0x7fb4380a4c90 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: dump: 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:11.836 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:11.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42000ce70 con 0x7fb4380a4c90 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4200138e0 con 0x7fb4380a4c90 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 5 monclient: authenticate success, global_id 35343 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: _renew_subs 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb438138730 con 0x7fb4380a4c90 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb4367fc640 4 set_mon_vals no callback set 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.806+0000 7fb434ff9640 10 monclient: dump: 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:11.839 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 client.35343.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 monclient: _renew_subs 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb438138bf0 con 0x7fb4380a4c90 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 1 librados: init done 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43813fe40 con 0x7fb4380a4c90 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb42001a070 con 0x7fb4380a4c90 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 0x7fb418040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(328..328 src has 1..328) ==== 6186+0+0 (secure 0 0 0) 0x7fb42000f040 con 0x7fb4380a4c90 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 10 client.35343.objecter ms_dispatch 0x7fb438000ba0 osd_map(328..328 src has 1..328) 2024-09-17T16:35:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 3 client.35343.objecter handle_osd_map got epochs [328,328] > 0 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 3 client.35343.objecter handle_osd_map decoding full epoch 328 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 20 client.35343.objecter dump_active .. 0 homeless 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb4377fe640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 0x7fb418040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb4377fe640 10 monclient: get_auth_request con 0x7fb41803e580 auth_method 0 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb434ff9640 10 client.35343.objecter ms_handle_connect 0x7fb41803e580 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.810+0000 7fb4377fe640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 0x7fb418040a40 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7fb4380a6800 tx=0x7fb428009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.814+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb420021070 con 0x7fb4380a4c90 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.814+0000 7fb434ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:11.814+0000 7fb434ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb43e610640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:35:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb43e610640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb43e610640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb43813c810 con 0x7fb4380a4c90 2024-09-17T16:35:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb434ff9640 1 -- 172.21.15.105:0/500643573 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7fb420013e10 con 0x7fb4380a4c90 2024-09-17T16:35:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb434ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.050+0000 7fb434ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.057 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 10 librados: watch_flush enter 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 10 librados: watch_flush exit 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 20 client.35343.objecter shutdown clearing up homeless session... 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 10 client.35343.objecter successfully canceled tick 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 msgr2=0x7fb418040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 0x7fb418040a40 secure :-1 s=READY pgs=3495 cs=0 l=1 rev1=1 crypto rx=0x7fb4380a6800 tx=0x7fb428009190 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 10 monclient: shutdown 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 msgr2=0x7fb4381d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e8e0 tx=0x7fb42000eda0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 shutdown_connections 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb41803e580 0x7fb418040a40 unknown :-1 s=CLOSED pgs=3495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb438139f00 0x7fb43813a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb438139b20 0x7fb4381d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 --2- 172.21.15.105:0/500643573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4380a4c90 0x7fb4381d17f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 >> 172.21.15.105:0/500643573 conn(0x7fb4380a93f0 msgr2=0x7fb438132cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 shutdown_connections 2024-09-17T16:35:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 -- 172.21.15.105:0/500643573 wait complete. 2024-09-17T16:35:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.054+0000 7fb43e610640 1 librados: shutdown 2024-09-17T16:35:12.075 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:35:12.075 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:35:12.075 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:35:12.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:35:12.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: get_monmap_and_config 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: monmap: 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:12.192865+0000 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:12.192865+0000 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:12.197 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 Processor -- start 2024-09-17T16:35:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 -- start start 2024-09-17T16:35:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: init 2024-09-17T16:35:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient: _add_conns ranks=[6,0,4,2,9,7,3,5,1,10,8] 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-d con 0x7f0ba409f950 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0ba4136ae0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba40a0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-c con 0x7f0ba4136ec0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba40a1800 con 0x7f0ba4136ae0 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0ba40a1ae0 con 0x7f0ba4136ec0 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0ba40a1dc0 con 0x7f0ba409f950 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0baa5ba640 10 monclient(hunting): authenticate will time out at 1395.459312s 2024-09-17T16:35:12.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba40a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0b9b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0ba4136ae0 0x7f0ba409fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41842/0 (socket says 172.21.15.105:41842) 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42852/0 (socket says 172.21.15.105:42852) 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0b9b7fe640 1 -- 172.21.15.105:0/2611498179 learned_addr learned my addr 172.21.15.105:0/2611498179 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request con 0x7f0ba409f950 auth_method 0 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): get_auth_request con 0x7f0ba4136ec0 auth_method 0 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_done global_id 54193 payload 274 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba37fe640 1 -- 172.21.15.105:0/2611498179 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343527728 0 0) 0x7f0ba40a1800 con 0x7f0ba4136ae0 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 -- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 msgr2=0x7f0ba41365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 -- 172.21.15.105:0/2611498179 >> v1:172.21.15.105:6789/0 conn(0x7f0ba4136ae0 legacy=0x7f0ba409fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 monclient: found mon.noname-c 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.190+0000 7f0ba8b57640 1 -- 172.21.15.105:0/2611498179 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40a28f0 con 0x7f0ba4136ec0 2024-09-17T16:35:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba8b57640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba40a0440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009dc0 tx=0x7f0b9c01c8e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=cc0f0b85af4206e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 1 -- 172.21.15.105:0/2611498179 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b9c0b2070 con 0x7f0ba4136ec0 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: dump: 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 1 -- 172.21.15.105:0/2611498179 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c0a5bb0 con 0x7f0ba4136ec0 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 5 monclient: authenticate success, global_id 54193 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 4 set_mon_vals no callback set 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 1 -- 172.21.15.105:0/2611498179 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b9c0a5da0 con 0x7f0ba4136ec0 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0ba37fe640 10 monclient: dump: 2024-09-17T16:35:12.212 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.213 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 10 monclient: shutdown 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 msgr2=0x7f0ba40a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba40a0440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009dc0 tx=0x7f0b9c01c8e0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2611498179 shutdown_connections 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba40a0440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2611498179 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba409f950 0x7f0ba41365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2611498179 >> 172.21.15.105:0/2611498179 conn(0x7f0ba40a8620 msgr2=0x7f0ba40a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2611498179 shutdown_connections 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2611498179 wait complete. 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 10 monclient: monmap: 2024-09-17T16:35:12.214 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.215 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 librados: starting msgr at 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.194+0000 7f0baa5ba640 1 librados: starting objecter 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 Processor -- start 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- start start 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 librados: setting wanted keys 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 librados: calling monclient init 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: init 2024-09-17T16:35:12.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _add_conns ranks=[2,6,4,1,5,7,8,0,3] 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 0x7f0ba413baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-b con 0x7f0ba409f950 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-d con 0x7f0ba4136ae0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba413c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): picked mon.noname-c con 0x7f0ba4136ec0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba4141420 con 0x7f0ba409f950 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0ba41412a0 con 0x7f0ba4136ec0 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0ba41415a0 con 0x7f0ba4136ae0 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): authenticate will time out at 1395.464625s 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 0x7f0ba413baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba413c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba413c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36418/0 (socket says 172.21.15.105:36418) 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 0x7f0ba413baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57278/0 (socket says 172.21.15.105:57278) 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 1 -- 172.21.15.105:0/2721304420 learned_addr learned my addr 172.21.15.105:0/2721304420 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42860/0 (socket says 172.21.15.105:42860) 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): get_auth_request con 0x7f0ba4136ae0 auth_method 0 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): get_auth_request con 0x7f0ba4136ec0 auth_method 0 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request con 0x7f0ba409f950 auth_method 0 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient(hunting): handle_auth_done global_id 24279 payload 824 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 msgr2=0x7f0ba413c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba413c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 msgr2=0x7f0ba413baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 0x7f0ba413baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 monclient: found mon.noname-d 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:12.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba413d040 con 0x7f0ba4136ae0 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ba409f950 0x7f0ba413baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0ba4136ec0 0x7f0ba413c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0b9b7fe640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c410 tx=0x7f0b8c00c8d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 client.?.objecter ms_handle_connect 0x7f0ba4136ae0 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b8c0047f0 con 0x7f0ba4136ae0 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: dump: 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c004990 con 0x7f0ba4136ae0 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 5 monclient: authenticate success, global_id 24279 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _renew_subs 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba413df60 con 0x7f0ba4136ae0 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b8c0133c0 con 0x7f0ba4136ae0 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: dump: 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 client.24279.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba2ffd640 4 set_mon_vals no callback set 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _renew_subs 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba412d8e0 con 0x7f0ba4136ae0 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 librados: init done 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 0x7f0ba4063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4051e80 con 0x7f0ba402e5d0 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 0x7f0ba4063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient: get_auth_request con 0x7f0ba402e5d0 auth_method 0 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0b8c011070 con 0x7f0ba4136ae0 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_done global_id 35361 payload 824 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ba402e9b0 con 0x7f0ba402e5d0 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b8403e550 0x7f0b84040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(328..328 src has 1..328) ==== 6186+0+0 (secure 0 0 0) 0x7f0b8c04f900 con 0x7f0ba4136ae0 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 client.24279.objecter ms_dispatch 0x7f0ba4000ba0 osd_map(328..328 src has 1..328) 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 3 client.24279.objecter handle_osd_map got epochs [328,328] > 0 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 3 client.24279.objecter handle_osd_map decoding full epoch 328 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 0x7f0ba4063e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b94011cf0 tx=0x7f0b9400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 20 client.24279.objecter dump_active .. 0 homeless 2024-09-17T16:35:12.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 client.24279.objecter ms_handle_connect 0x7f0ba402e5d0 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 client.24279.objecter resend_mon_ops 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b94019070 con 0x7f0ba402e5d0 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: dump: 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.227 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b8403e550 0x7f0b84040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba8b57640 10 monclient: get_auth_request con 0x7f0b8403e550 auth_method 0 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0b94004030 con 0x7f0ba402e5d0 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 msgr2=0x7f0ba4063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 0x7f0ba4063e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b94011cf0 tx=0x7f0b9400f610 comp rx=0 tx=0).stop 2024-09-17T16:35:12.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 msgr2=0x7f0ba4063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.198+0000 7f0ba17fa640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba402e5d0 0x7f0ba4063e00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.202+0000 7f0ba17fa640 10 client.24279.objecter ms_handle_connect 0x7f0b8403e550 2024-09-17T16:35:12.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.202+0000 7f0ba8b57640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b8403e550 0x7f0b84040a10 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c01c710 tx=0x7f0b9c0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 0x7f0ba41d9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba406a1c0 con 0x7f0ba4064340 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 0x7f0ba41d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient: get_auth_request con 0x7f0ba4064340 auth_method 0 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 10 monclient(hunting): handle_auth_done global_id 35370 payload 824 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 1 -- 172.21.15.105:0/2721304420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0ba40a8b90 con 0x7f0ba4064340 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 client.24279.objecter ms_handle_connect 0x7f0ba4064340 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 client.24279.objecter resend_mon_ops 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba3fff640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 0x7f0ba41d9510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b94011cf0 tx=0x7f0b940046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b94019040 con 0x7f0ba4064340 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 monclient: dump: 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f0b9401e450 con 0x7f0ba4064340 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 msgr2=0x7f0ba41d9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 0x7f0ba41d9510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b94011cf0 tx=0x7f0b940046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 msgr2=0x7f0ba41d9510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba17fa640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ba4064340 0x7f0ba41d9510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:12.234 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 303009, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:35:12.235 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.236 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.237 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.238 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.239 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:12.240 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.241 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:12.244 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:12.247 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.249 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.251 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.253 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.255 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.256 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:12.257 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.257 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.257 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:12.257 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 librados: watch_flush enter 2024-09-17T16:35:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 librados: watch_flush exit 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 20 client.24279.objecter shutdown clearing up homeless session... 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 client.24279.objecter successfully canceled tick 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b8403e550 msgr2=0x7f0b84040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b8403e550 0x7f0b84040a10 secure :-1 s=READY pgs=3496 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c01c710 tx=0x7f0b9c0a7070 comp rx=0 tx=0).stop 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 10 monclient: shutdown 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 msgr2=0x7f0ba413c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c410 tx=0x7f0b8c00c8d0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0ba8b57640 1 -- 172.21.15.105:0/2721304420 reap_dead start 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 shutdown_connections 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 --2- 172.21.15.105:0/2721304420 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0ba4136ae0 0x7f0ba413c030 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 >> 172.21.15.105:0/2721304420 conn(0x7f0ba40a8620 msgr2=0x7f0ba4074290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 shutdown_connections 2024-09-17T16:35:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 -- 172.21.15.105:0/2721304420 wait complete. 2024-09-17T16:35:12.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.206+0000 7f0baa5ba640 1 librados: shutdown 2024-09-17T16:35:12.260 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:12.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:35:12.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: get_monmap_and_config 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: monmap: 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:12.372309+0000 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:12.372309+0000 2024-09-17T16:35:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:12.376 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 Processor -- start 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 -- start start 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: init 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient: _add_conns ranks=[10,3,2,8,7,5,9,1,6,4,0] 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 0x7f04440a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): picked mon.noname-f con 0x7f04440a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 0x7f04440a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): picked mon.noname-g con 0x7f04440a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): picked mon.noname-b con 0x7f04441396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f04440a3fc0 con 0x7f04440a68e0 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f04440a63c0 con 0x7f04441396a0 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f04440a7740 con 0x7f04440a4910 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044b570640 10 monclient(hunting): authenticate will time out at 1395.638044s 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57286/0 (socket says 172.21.15.105:57286) 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 -- 172.21.15.105:0/3249719582 learned_addr learned my addr 172.21.15.105:0/3249719582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): get_auth_request con 0x7f04441396a0 auth_method 0 2024-09-17T16:35:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044930c640 1 --2- 172.21.15.105:0/3249719582 >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 0x7f04440a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient(hunting): handle_auth_done global_id 44579 payload 274 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0448b0b640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 0x7f04440a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 -- 172.21.15.105:0/3249719582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 msgr2=0x7f04440a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 0x7f04440a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 -- 172.21.15.105:0/3249719582 >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 msgr2=0x7f04440a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 --2- 172.21.15.105:0/3249719582 >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 0x7f04440a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 monclient: found mon.noname-b 2024-09-17T16:35:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0449b0d640 1 -- 172.21.15.105:0/3249719582 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044406ebc0 con 0x7f04441396a0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044930c640 10 monclient: get_auth_request con 0x7f04440a68e0 auth_method 0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f044930c640 1 --2- 172.21.15.105:0/3249719582 >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 0x7f04440a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0448b0b640 10 monclient: get_auth_request con 0x7f04440a4910 auth_method 0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.370+0000 7f0448b0b640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 0x7f04440a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad73586c29efbd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 1 -- 172.21.15.105:0/3249719582 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f043c0b2070 con 0x7f04441396a0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: dump: 2024-09-17T16:35:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 1 -- 172.21.15.105:0/3249719582 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043c0ab7b0 con 0x7f04441396a0 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 1 -- 172.21.15.105:0/3249719582 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f043c0abe50 con 0x7f04441396a0 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 5 monclient: authenticate success, global_id 44579 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 4 set_mon_vals no callback set 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0443fff640 10 monclient: dump: 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.388 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.388 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: shutdown 2024-09-17T16:35:12.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- 172.21.15.105:0/3249719582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 msgr2=0x7f0444139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- 172.21.15.105:0/3249719582 shutdown_connections 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f04441396a0 0x7f0444139b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- 172.21.15.105:0/3249719582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a4910 0x7f04440a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- 172.21.15.105:0/3249719582 >> v2:172.21.15.160:3300/0 conn(0x7f04440a68e0 0x7f04440a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- 172.21.15.105:0/3249719582 >> 172.21.15.105:0/3249719582 conn(0x7f0444068ba0 msgr2=0x7f0444068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- 172.21.15.105:0/3249719582 shutdown_connections 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- 172.21.15.105:0/3249719582 wait complete. 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: monmap: 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.397 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 librados: starting msgr at 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 librados: starting objecter 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 Processor -- start 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- start start 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 librados: setting wanted keys 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 librados: calling monclient init 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: init 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient: _add_conns ranks=[0,3,5,7,8,4,1,6,2] 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): picked mon.noname-a con 0x7f04440a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a68e0 0x7f044413dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): picked mon.noname-g con 0x7f04440a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04441396a0 0x7f044413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): picked mon.noname-h con 0x7f04441396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f044406e630 con 0x7f04440a4910 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f04440a7af0 con 0x7f04440a68e0 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0444143320 con 0x7f04441396a0 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044b570640 10 monclient(hunting): authenticate will time out at 1395.643207s 2024-09-17T16:35:12.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52872/0 (socket says 172.21.15.105:52872) 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 -- 172.21.15.105:0/525200370 learned_addr learned my addr 172.21.15.105:0/525200370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): get_auth_request con 0x7f04440a4910 auth_method 0 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a68e0 0x7f044413dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04441396a0 0x7f044413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 10 monclient(hunting): get_auth_request con 0x7f04440a68e0 auth_method 0 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 10 monclient(hunting): get_auth_request con 0x7f04441396a0 auth_method 0 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient(hunting): handle_auth_done global_id 35379 payload 824 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04441396a0 msgr2=0x7f044413e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04441396a0 0x7f044413e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a68e0 msgr2=0x7f044413dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a68e0 0x7f044413dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 monclient: found mon.noname-a 2024-09-17T16:35:12.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f044930c640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0444123570 con 0x7f04440a4910 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04440a68e0 0x7f044413dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.374+0000 7f0449b0d640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f04441396a0 0x7f044413e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044930c640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f043800c7c0 tx=0x7f043800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.?.objecter ms_handle_connect 0x7f04440a4910 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0438007ca0 con 0x7f04440a4910 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: dump: 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.402 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.403 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04380113f0 con 0x7f04440a4910 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0438011a10 con 0x7f04440a4910 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: dump: 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.405 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f04437fe640 4 set_mon_vals no callback set 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 5 monclient: authenticate success, global_id 35379 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 monclient: _renew_subs 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044413f650 con 0x7f04440a4910 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 client.35379.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 monclient: _renew_subs 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04440ab9a0 con 0x7f04440a4910 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 1 librados: init done 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 0x7f044402e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0444051e80 con 0x7f044413e730 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 0x7f044402e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient: get_auth_request con 0x7f044413e730 auth_method 0 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0438011bb0 con 0x7f04440a4910 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 10 monclient(hunting): handle_auth_done global_id 44588 payload 824 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f044413fb20 con 0x7f044413e730 2024-09-17T16:35:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f043003e550 0x7f0430040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(328..328 src has 1..328) ==== 6186+0+0 (secure 0 0 0) 0x7f0438050150 con 0x7f04440a4910 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.35379.objecter ms_dispatch 0x7f0444000ba0 osd_map(328..328 src has 1..328) 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 3 client.35379.objecter handle_osd_map got epochs [328,328] > 0 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 0x7f044402e5d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f044413f3e0 tx=0x7f043400ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 3 client.35379.objecter handle_osd_map decoding full epoch 328 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 20 client.35379.objecter dump_active .. 0 homeless 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.35379.objecter ms_handle_connect 0x7f044413e730 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.35379.objecter resend_mon_ops 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f043400a990 con 0x7f044413e730 2024-09-17T16:35:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: dump: 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0449b0d640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f043003e550 0x7f0430040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0449b0d640 10 monclient: get_auth_request con 0x7f043003e550 auth_method 0 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f043400a520 con 0x7f044413e730 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 msgr2=0x7f044402e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 0x7f044402e5d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f044413f3e0 tx=0x7f043400ca80 comp rx=0 tx=0).stop 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 msgr2=0x7f044402e5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f044413e730 0x7f044402e5d0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f044413f3e0 tx=0x7f043400ca80 comp rx=0 tx=0).stop 2024-09-17T16:35:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0441ffb640 10 client.35379.objecter ms_handle_connect 0x7f043003e550 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.378+0000 7f0449b0d640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f043003e550 0x7f0430040a10 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f043c0af000 tx=0x7f043c0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 0x7f0444064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f04440a9ee0 con 0x7f0444063e00 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 0x7f0444064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient: get_auth_request con 0x7f0444063e00 auth_method 0 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 10 monclient(hunting): handle_auth_done global_id 44597 payload 824 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 1 -- 172.21.15.105:0/525200370 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0444139b50 con 0x7f0444063e00 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 client.35379.objecter ms_handle_connect 0x7f0444063e00 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 client.35379.objecter resend_mon_ops 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0448b0b640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 0x7f0444064210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f043400cfd0 tx=0x7f0434004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0434025070 con 0x7f0444063e00 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 monclient: dump: 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.412 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f0434029450 con 0x7f0444063e00 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 msgr2=0x7f0444064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 0x7f0444064210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f043400cfd0 tx=0x7f0434004a50 comp rx=0 tx=0).stop 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 msgr2=0x7f0444064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0441ffb640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0444063e00 0x7f0444064210 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f043400cfd0 tx=0x7f0434004a50 comp rx=0 tx=0).stop 2024-09-17T16:35:12.414 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 163038, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:12.415 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.416 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.417 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:12.418 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.419 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:12.421 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.424 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:12.425 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:12.427 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:12.428 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.431 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:12.433 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.433 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:12.437 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.438 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.441 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:12.442 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.443 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.444 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.444 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.444 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:12.444 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:12.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 librados: watch_flush enter 2024-09-17T16:35:12.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 librados: watch_flush exit 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 20 client.35379.objecter shutdown clearing up homeless session... 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 client.35379.objecter successfully canceled tick 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f043003e550 msgr2=0x7f0430040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f043003e550 0x7f0430040a10 secure :-1 s=READY pgs=3497 cs=0 l=1 rev1=1 crypto rx=0x7f043c0af000 tx=0x7f043c0a7070 comp rx=0 tx=0).stop 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 10 monclient: shutdown 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 msgr2=0x7f044413d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f0449b0d640 1 -- 172.21.15.105:0/525200370 reap_dead start 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f043800c7c0 tx=0x7f043800cc80 comp rx=0 tx=0).stop 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 shutdown_connections 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 --2- 172.21.15.105:0/525200370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04440a4910 0x7f044413d770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 >> 172.21.15.105:0/525200370 conn(0x7f0444068ba0 msgr2=0x7f04440a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 shutdown_connections 2024-09-17T16:35:12.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 -- 172.21.15.105:0/525200370 wait complete. 2024-09-17T16:35:12.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.386+0000 7f044b570640 1 librados: shutdown 2024-09-17T16:35:12.447 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:12.447 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:35:12.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: get_monmap_and_config 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: monmap: 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:12.567967+0000 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:12.567967+0000 2024-09-17T16:35:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:12.573 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 Processor -- start 2024-09-17T16:35:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 -- start start 2024-09-17T16:35:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: init 2024-09-17T16:35:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient: _add_conns ranks=[10,5,0,6,4,8,1,3,2,9,7] 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f774c0a6360 0x7f774c0a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-f con 0x7f774c0a6360 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f774c0a4c90 0x7f774c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-h con 0x7f774c0a4c90 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f774c13dd00 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c06cbe0 con 0x7f774c13dd00 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f774c09e740 con 0x7f774c0a6360 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f774c0a6740 con 0x7f774c0a4c90 2024-09-17T16:35:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 10 monclient(hunting): authenticate will time out at 1395.833733s 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f774c13dd00 0x7f774c13e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41848/0 (socket says 172.21.15.105:41848) 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 1 -- 172.21.15.105:0/2258609056 learned_addr learned my addr 172.21.15.105:0/2258609056 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f774c0a4c90 0x7f774c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 --2- 172.21.15.105:0/2258609056 >> v2:172.21.15.160:3300/0 conn(0x7f774c0a6360 0x7f774c0a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): get_auth_request con 0x7f774c0a4c90 auth_method 0 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): get_auth_request con 0x7f774c0a6360 auth_method 0 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363966070 0 0) 0x7f773c001120 con 0x7f774c13dd00 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): _negotiate 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): my global_id is 35388 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7738003530 con 0x7f774c13dd00 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2604965245 0 0) 0x7f7738003530 con 0x7f774c13dd00 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f774c0a4c90 msgr2=0x7f774c0a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f774c0a4c90 0x7f774c0a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 >> v2:172.21.15.160:3300/0 conn(0x7f774c0a6360 msgr2=0x7f774c0a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 --2- 172.21.15.105:0/2258609056 >> v2:172.21.15.160:3300/0 conn(0x7f774c0a6360 0x7f774c0a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c09fc20 con 0x7f774c13dd00 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 5 monclient: authenticate success, global_id 35388 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f773c001b40 con 0x7f774c13dd00 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: dump: 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.592 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 >> v1:172.21.15.105:6789/0 conn(0x7f774c13dd00 legacy=0x7f774c13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient: _add_conns ranks=[2,8,1,5,3,4,0,6,7] 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 0x7f7738004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): picked mon.b con 0x7f7738004240 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7738004d50 0x7f77380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): picked mon.e con 0x7f7738004d50 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7738009a30 0x7f773800bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): picked mon.f con 0x7f7738009a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f774c09e740 con 0x7f7738009a30 2024-09-17T16:35:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0a6740 con 0x7f7738004240 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f773800c7d0 con 0x7f7738004d50 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774affd640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f77529e2640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 0x7f7738004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): get_auth_request con 0x7f7738004240 auth_method 0 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7738004d50 0x7f77380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): get_auth_request con 0x7f7738004d50 auth_method 0 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient(hunting): handle_auth_done global_id 35388 payload 293 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 -- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7738004d50 msgr2=0x7f77380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7738009a30 0x7f773800bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7738004d50 0x7f77380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 -- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7738009a30 msgr2=0x7f773800bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7738009a30 0x7f773800bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 monclient: found mon.b 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 1 -- 172.21.15.105:0/2258609056 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7738003530 con 0x7f7738004240 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f774bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 10 monclient: get_auth_request con 0x7f7738009a30 auth_method 0 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7738009a30 0x7f773800bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.566+0000 7f7750f7f640 1 -- 172.21.15.105:0/2258609056 reap_dead start 2024-09-17T16:35:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774bfff640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 0x7f7738004810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ae0 tx=0x7f774401ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797619320b09a349 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77440b2070 con 0x7f7738004240 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: dump: 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.597 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77440a5ab0 con 0x7f7738004240 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 4 set_mon_vals no callback set 2024-09-17T16:35:12.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 1 -- 172.21.15.105:0/2258609056 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77440096b0 con 0x7f7738004240 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774affd640 10 monclient: dump: 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.599 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: shutdown 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 msgr2=0x7f7738004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 0x7f7738004810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ae0 tx=0x7f774401ca50 comp rx=0 tx=0).stop 2024-09-17T16:35:12.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- 172.21.15.105:0/2258609056 shutdown_connections 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 --2- 172.21.15.105:0/2258609056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7738004240 0x7f7738004810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- 172.21.15.105:0/2258609056 >> 172.21.15.105:0/2258609056 conn(0x7f774c0a5840 msgr2=0x7f774c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- 172.21.15.105:0/2258609056 shutdown_connections 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- 172.21.15.105:0/2258609056 wait complete. 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: monmap: 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:12.601 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 librados: starting msgr at 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 librados: starting objecter 2024-09-17T16:35:12.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 Processor -- start 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- start start 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 librados: setting wanted keys 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 librados: calling monclient init 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: init 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient: _add_conns ranks=[3,8,4,0,7,2,5,6,1] 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7738004240 0x7f774c072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-g con 0x7f7738004240 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-e con 0x7f774c13ce50 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 0x7f774c1d2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): picked mon.noname-c con 0x7f774c1d24d0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:12.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f774c06ce20 con 0x7f7738004240 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f774c06c1a0 con 0x7f774c1d24d0 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f774c128060 con 0x7f774c13ce50 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f77529e2640 10 monclient(hunting): authenticate will time out at 1395.839749s 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 0x7f774c1d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:35414/0 (socket says 172.21.15.105:35414) 2024-09-17T16:35:12.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 0x7f774c1d2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36426/0 (socket says 172.21.15.105:36426) 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 1 -- 172.21.15.105:0/2014056015 learned_addr learned my addr 172.21.15.105:0/2014056015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 10 monclient(hunting): get_auth_request con 0x7f774c13ce50 auth_method 0 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f774b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 10 monclient(hunting): get_auth_request con 0x7f774c1d24d0 auth_method 0 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.570+0000 7f7750f7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7738004240 0x7f774c072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): get_auth_request con 0x7f7738004240 auth_method 0 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient(hunting): handle_auth_done global_id 54161 payload 824 2024-09-17T16:35:12.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 msgr2=0x7f774c1d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 0x7f774c1d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7738004240 msgr2=0x7f774c072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7738004240 0x7f774c072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 monclient: found mon.noname-e 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c13d7f0 con 0x7f774c13ce50 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c1d24d0 0x7f774c1d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7738004240 0x7f774c072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774b7fe640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fc0 tx=0x7f773400f580 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.?.objecter ms_handle_connect 0x7f774c13ce50 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f773400fea0 con 0x7f774c13ce50 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: dump: 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.608 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340158a0 con 0x7f774c13ce50 2024-09-17T16:35:12.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f773401d400 con 0x7f774c13ce50 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: dump: 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.610 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774a7fc640 4 set_mon_vals no callback set 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 5 monclient: authenticate success, global_id 54161 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 monclient: _renew_subs 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:12.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c0a28d0 con 0x7f774c13ce50 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 client.54161.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 monclient: _renew_subs 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c129f70 con 0x7f774c13ce50 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 1 librados: init done 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 0x7f774c1d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f774c06cfa0 con 0x7f774c02e5d0 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7734015430 con 0x7f774c13ce50 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 0x7f774c1d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient: get_auth_request con 0x7f774c02e5d0 auth_method 0 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f772c03e550 0x7f772c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 10 monclient(hunting): handle_auth_done global_id 54220 payload 824 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f774c02e9b0 con 0x7f774c02e5d0 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(328..328 src has 1..328) ==== 6186+0+0 (secure 0 0 0) 0x7f7734051220 con 0x7f774c13ce50 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.54161.objecter ms_dispatch 0x7f774c000ba0 osd_map(328..328 src has 1..328) 2024-09-17T16:35:12.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 3 client.54161.objecter handle_osd_map got epochs [328,328] > 0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 3 client.54161.objecter handle_osd_map decoding full epoch 328 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 20 client.54161.objecter dump_active .. 0 homeless 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 0x7f774c1d2e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ab0 tx=0x7f77440197f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.54161.objecter ms_handle_connect 0x7f774c02e5d0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.54161.objecter resend_mon_ops 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f772c03e550 0x7f772c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 10 monclient: get_auth_request con 0x7f772c03e550 auth_method 0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77440b2050 con 0x7f774c02e5d0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: dump: 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.614 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.615 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f77440b3040 con 0x7f774c02e5d0 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 msgr2=0x7f774c1d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 0x7f774c1d2e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ab0 tx=0x7f77440197f0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 msgr2=0x7f774c1d2e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c02e5d0 0x7f774c1d2e70 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ab0 tx=0x7f77440197f0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7748ff9640 10 client.54161.objecter ms_handle_connect 0x7f772c03e550 2024-09-17T16:35:12.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.574+0000 7f7750f7f640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f772c03e550 0x7f772c040a10 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f774c06c6b0 tx=0x7f773c009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 0x7f774c064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f774c051e80 con 0x7f774c063e00 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 0x7f774c064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient: get_auth_request con 0x7f774c063e00 auth_method 0 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 10 monclient(hunting): handle_auth_done global_id 54229 payload 824 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 1 -- 172.21.15.105:0/2014056015 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f774c0a4750 con 0x7f774c063e00 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 client.54161.objecter ms_handle_connect 0x7f774c063e00 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 client.54161.objecter resend_mon_ops 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f774bfff640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 0x7f774c064210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ae0 tx=0x7f7738005840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77440bf2b0 con 0x7f774c063e00 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 monclient: dump: 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.620 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f77440bf2b0 con 0x7f774c063e00 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 msgr2=0x7f774c064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 0x7f774c064210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ae0 tx=0x7f7738005840 comp rx=0 tx=0).stop 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 msgr2=0x7f774c064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7748ff9640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f774c063e00 0x7f774c064210 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7744009ae0 tx=0x7f7738005840 comp rx=0 tx=0).stop 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19579, 2024-09-17T16:35:12.622 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 6, 2024-09-17T16:35:12.623 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.624 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.625 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:12.626 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:12.627 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:12.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:12.632 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.633 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.635 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.636 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.638 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.640 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.643 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:12.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.645 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:12.646 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:12.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 librados: watch_flush enter 2024-09-17T16:35:12.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 librados: watch_flush exit 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 20 client.54161.objecter shutdown clearing up homeless session... 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 client.54161.objecter successfully canceled tick 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f772c03e550 msgr2=0x7f772c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f772c03e550 0x7f772c040a10 secure :-1 s=READY pgs=3498 cs=0 l=1 rev1=1 crypto rx=0x7f774c06c6b0 tx=0x7f773c009040 comp rx=0 tx=0).stop 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 10 monclient: shutdown 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 msgr2=0x7f774c13d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fc0 tx=0x7f773400f580 comp rx=0 tx=0).stop 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f7750f7f640 1 -- 172.21.15.105:0/2014056015 reap_dead start 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 shutdown_connections 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 --2- 172.21.15.105:0/2014056015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f774c13ce50 0x7f774c13d2b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 >> 172.21.15.105:0/2014056015 conn(0x7f774c0a5840 msgr2=0x7f774c0a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 shutdown_connections 2024-09-17T16:35:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 -- 172.21.15.105:0/2014056015 wait complete. 2024-09-17T16:35:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.582+0000 7f77529e2640 1 librados: shutdown 2024-09-17T16:35:12.649 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:12.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:35:12.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:35:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: get_monmap_and_config 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: monmap: 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:12.764560+0000 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:12.764560+0000 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:12.769 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 Processor -- start 2024-09-17T16:35:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 -- start start 2024-09-17T16:35:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: init 2024-09-17T16:35:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient: _add_conns ranks=[9,1,8,0,6,3,4,10,5,7,2] 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-a con 0x7f48200a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f48200a4ab0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 0x7f4820139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-e con 0x7f48200a7350 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48200a4080 con 0x7f48200a4ab0 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f48200a7c40 con 0x7f48200a6a30 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f48200a37e0 con 0x7f48200a7350 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4826d64640 10 monclient(hunting): authenticate will time out at 1396.030565s 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 0x7f4820139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52878/0 (socket says 172.21.15.105:52878) 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 1 -- 172.21.15.105:0/3003051891 learned_addr learned my addr 172.21.15.105:0/3003051891 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): get_auth_request con 0x7f48200a6a30 auth_method 0 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): get_auth_request con 0x7f48200a7350 auth_method 0 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient(hunting): handle_auth_done global_id 54170 payload 274 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 -- 172.21.15.105:0/3003051891 >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 msgr2=0x7f48200a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 --2- 172.21.15.105:0/3003051891 >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 -- 172.21.15.105:0/3003051891 >> v1:172.21.15.160:6789/0 conn(0x7f48200a4ab0 legacy=0x7f48201397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 monclient: found mon.noname-e 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4825301640 1 -- 172.21.15.105:0/3003051891 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200a3ac0 con 0x7f48200a7350 2024-09-17T16:35:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.762+0000 7f4824b00640 1 --2- 172.21.15.105:0/3003051891 >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4825301640 1 --2- 172.21.15.105:0/3003051891 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 0x7f4820139f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4818009d90 tx=0x7f481801c7f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f5c9213b7828dbf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 1 -- 172.21.15.105:0/3003051891 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48180b2070 con 0x7f48200a7350 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: dump: 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.784 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 1 -- 172.21.15.105:0/3003051891 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48180a5b00 con 0x7f48200a7350 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 5 monclient: authenticate success, global_id 54170 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 1 -- 172.21.15.105:0/3003051891 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48180a5d30 con 0x7f48200a7350 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 4 set_mon_vals no callback set 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481f7fe640 10 monclient: dump: 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: shutdown 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- 172.21.15.105:0/3003051891 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 msgr2=0x7f4820139f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- 172.21.15.105:0/3003051891 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 0x7f4820139f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4818009d90 tx=0x7f481801c7f0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- 172.21.15.105:0/3003051891 shutdown_connections 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- 172.21.15.105:0/3003051891 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f48200a7350 0x7f4820139f00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- 172.21.15.105:0/3003051891 >> v2:172.21.15.105:3300/0 conn(0x7f48200a6a30 0x7f48200a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- 172.21.15.105:0/3003051891 >> 172.21.15.105:0/3003051891 conn(0x7f48200a8d00 msgr2=0x7f48200a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- 172.21.15.105:0/3003051891 shutdown_connections 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- 172.21.15.105:0/3003051891 wait complete. 2024-09-17T16:35:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: monmap: 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 librados: starting msgr at 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 librados: starting objecter 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 Processor -- start 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- start start 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 librados: setting wanted keys 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 librados: calling monclient init 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: init 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient: _add_conns ranks=[3,2,5,8,6,4,0,1,7] 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48200a4ab0 0x7f48201d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-g con 0x7f48200a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-b con 0x7f48200a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f48200a7350 0x7f48201d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): picked mon.noname-h con 0x7f48200a7350 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f482013a440 con 0x7f48200a6a30 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f482006e770 con 0x7f48200a4ab0 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f482013a5c0 con 0x7f48200a7350 2024-09-17T16:35:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f4826d64640 10 monclient(hunting): authenticate will time out at 1396.035720s 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57318/0 (socket says 172.21.15.105:57318) 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 1 -- 172.21.15.105:0/3667743698 learned_addr learned my addr 172.21.15.105:0/3667743698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 10 monclient(hunting): get_auth_request con 0x7f48200a6a30 auth_method 0 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.766+0000 7f481ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f48200a7350 0x7f48201d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48200a4ab0 0x7f48201d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): get_auth_request con 0x7f48200a4ab0 auth_method 0 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 10 monclient(hunting): get_auth_request con 0x7f48200a7350 auth_method 0 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient(hunting): handle_auth_done global_id 44606 payload 824 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f48200a7350 msgr2=0x7f48201d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f48200a7350 0x7f48201d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48200a4ab0 msgr2=0x7f48201d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48200a4ab0 0x7f48201d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 monclient: found mon.noname-b 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4820123880 con 0x7f48200a6a30 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481ffff640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f481400c410 tx=0x7f481400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.?.objecter ms_handle_connect 0x7f48200a6a30 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4814011a00 con 0x7f48200a6a30 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: dump: 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4814017070 con 0x7f48200a6a30 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4814011c60 con 0x7f48200a6a30 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: dump: 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481effd640 4 set_mon_vals no callback set 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 5 monclient: authenticate success, global_id 44606 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 monclient: _renew_subs 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482013a1e0 con 0x7f48200a6a30 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 client.44606.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 monclient: _renew_subs 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48200a1f00 con 0x7f48200a6a30 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 1 librados: init done 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 0x7f482013e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f4820051e80 con 0x7f482013dfa0 2024-09-17T16:35:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 0x7f482013e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient: get_auth_request con 0x7f482013dfa0 auth_method 0 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4814011590 con 0x7f48200a6a30 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 10 monclient(hunting): handle_auth_done global_id 24288 payload 824 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f480c03e550 0x7f480c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f482006df10 con 0x7f482013dfa0 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(329..329 src has 1..329) ==== 5801+0+0 (secure 0 0 0) 0x7f481404f920 con 0x7f48200a6a30 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.44606.objecter ms_dispatch 0x7f4820000ba0 osd_map(329..329 src has 1..329) 2024-09-17T16:35:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 3 client.44606.objecter handle_osd_map got epochs [329,329] > 0 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 3 client.44606.objecter handle_osd_map decoding full epoch 329 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4824b00640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 0x7f482013e380 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f4810011ac0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 20 client.44606.objecter dump_active .. 0 homeless 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.44606.objecter ms_handle_connect 0x7f482013dfa0 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.44606.objecter resend_mon_ops 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f481000fba0 con 0x7f482013dfa0 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: dump: 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f480c03e550 0x7f480c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 10 monclient: get_auth_request con 0x7f480c03e550 auth_method 0 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f481000f730 con 0x7f482013dfa0 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 msgr2=0x7f482013e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 0x7f482013e380 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f4810011ac0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 msgr2=0x7f482013e380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013dfa0 0x7f482013e380 secure :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f4810011ac0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f4825301640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f480c03e550 0x7f480c040a10 secure :-1 s=READY pgs=3499 cs=0 l=1 rev1=1 crypto rx=0x7f48180af000 tx=0x7f48180a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.770+0000 7f481d7fa640 10 client.44606.objecter ms_handle_connect 0x7f480c03e550 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 0x7f482013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f482006a200 con 0x7f482013e8c0 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 0x7f482013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient: get_auth_request con 0x7f482013e8c0 auth_method 0 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 10 monclient(hunting): handle_auth_done global_id 24297 payload 824 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 1 -- 172.21.15.105:0/3667743698 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48200a6e10 con 0x7f482013e8c0 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 client.44606.objecter ms_handle_connect 0x7f482013e8c0 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4824b00640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 0x7f482013a800 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f48100046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 client.44606.objecter resend_mon_ops 2024-09-17T16:35:12.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48100222c0 con 0x7f482013e8c0 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 monclient: dump: 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.805 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f48100222c0 con 0x7f482013e8c0 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 msgr2=0x7f482013a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 0x7f482013a800 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f48100046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 msgr2=0x7f482013a800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f481d7fa640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f482013e8c0 0x7f482013a800 secure :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f4810011600 tx=0x7f48100046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 460417, 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:12.807 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:12.828 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:12.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:12.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:12.831 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:12.831 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:12.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 librados: watch_flush enter 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 librados: watch_flush exit 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 20 client.44606.objecter shutdown clearing up homeless session... 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 client.44606.objecter successfully canceled tick 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f480c03e550 msgr2=0x7f480c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f480c03e550 0x7f480c040a10 secure :-1 s=READY pgs=3499 cs=0 l=1 rev1=1 crypto rx=0x7f48180af000 tx=0x7f48180a7070 comp rx=0 tx=0).stop 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 10 monclient: shutdown 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 msgr2=0x7f48201d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f481400c410 tx=0x7f481400c8d0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4825301640 1 -- 172.21.15.105:0/3667743698 reap_dead start 2024-09-17T16:35:12.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 shutdown_connections 2024-09-17T16:35:12.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 --2- 172.21.15.105:0/3667743698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f48200a6a30 0x7f48201d1540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 >> 172.21.15.105:0/3667743698 conn(0x7f48200a8d00 msgr2=0x7f48200aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 shutdown_connections 2024-09-17T16:35:12.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 -- 172.21.15.105:0/3667743698 wait complete. 2024-09-17T16:35:12.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.778+0000 7f4826d64640 1 librados: shutdown 2024-09-17T16:35:12.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:12.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:35:12.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: get_monmap_and_config 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: monmap: 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:12.955888+0000 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:12.955888+0000 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:12.961 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 Processor -- start 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 -- start start 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: init 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient: _add_conns ranks=[0,5,4,7,8,1,10,3,2,9,6] 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe62c0a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c0a5240 0x7fe62c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-h con 0x7fe62c0a5240 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 0x7fe62c13f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-c con 0x7fe62c13cd70 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62c0a0b90 con 0x7fe62c0a4750 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe62c0a08b0 con 0x7fe62c13cd70 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe62c0a0e70 con 0x7fe62c0a5240 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe6326ec640 10 monclient(hunting): authenticate will time out at 1396.221702s 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62bfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe62c0a4750 0x7fe62c0a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41852/0 (socket says 172.21.15.105:41852) 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62bfff640 1 -- 172.21.15.105:0/652087786 learned_addr learned my addr 172.21.15.105:0/652087786 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 0x7fe62c13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): get_auth_request con 0x7fe62c13cd70 auth_method 0 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c0a5240 0x7fe62c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): get_auth_request con 0x7fe62c0a5240 auth_method 0 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 1 -- 172.21.15.105:0/652087786 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166764000 0 0) 0x7fe61c001120 con 0x7fe62c0a4750 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 10 monclient(hunting): _negotiate 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 10 monclient(hunting): my global_id is 35406 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62affd640 1 -- 172.21.15.105:0/652087786 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe618003530 con 0x7fe62c0a4750 2024-09-17T16:35:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient(hunting): handle_auth_done global_id 54238 payload 274 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 -- 172.21.15.105:0/652087786 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c0a5240 msgr2=0x7fe62c13c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c0a5240 0x7fe62c13c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 -- 172.21.15.105:0/652087786 >> v1:172.21.15.105:6789/0 conn(0x7fe62c0a4750 legacy=0x7fe62c0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 monclient: found mon.noname-c 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.954+0000 7fe630c89640 1 -- 172.21.15.105:0/652087786 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c09fca0 con 0x7fe62c13cd70 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe630c89640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 0x7fe62c13f160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6240097f0 tx=0x7fe62401c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=af9b820d568341b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 1 -- 172.21.15.105:0/652087786 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6240b0070 con 0x7fe62c13cd70 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: dump: 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 5 monclient: authenticate success, global_id 54238 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 1 -- 172.21.15.105:0/652087786 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6240aa3a0 con 0x7fe62c13cd70 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 1 -- 172.21.15.105:0/652087786 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6240aa9c0 con 0x7fe62c13cd70 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 4 set_mon_vals no callback set 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62affd640 10 monclient: dump: 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: shutdown 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/652087786 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 msgr2=0x7fe62c13f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 0x7fe62c13f160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6240097f0 tx=0x7fe62401c3a0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/652087786 shutdown_connections 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c13cd70 0x7fe62c13f160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- 172.21.15.105:0/652087786 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c0a5240 0x7fe62c13c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/652087786 >> 172.21.15.105:0/652087786 conn(0x7fe62c0a5840 msgr2=0x7fe62c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/652087786 shutdown_connections 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/652087786 wait complete. 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: build_initial_monmap 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: monmap: 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 librados: starting msgr at 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 librados: starting objecter 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 Processor -- start 2024-09-17T16:35:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- start start 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 librados: setting wanted keys 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 librados: calling monclient init 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: init 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _add_conns ranks=[3,4,5,7,2,0,8,1,6] 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe62c0a4750 0x7fe62c136060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-g con 0x7fe62c0a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-c con 0x7fe62c0a5240 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c13cd70 0x7fe62c136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): picked mon.noname-h con 0x7fe62c13cd70 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c142040 con 0x7fe62c0a4750 2024-09-17T16:35:12.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe62c141ec0 con 0x7fe62c0a5240 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe62c1421c0 con 0x7fe62c13cd70 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient(hunting): authenticate will time out at 1396.227103s 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36446/0 (socket says 172.21.15.105:36446) 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 -- 172.21.15.105:0/3508132749 learned_addr learned my addr 172.21.15.105:0/3508132749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): get_auth_request con 0x7fe62c0a5240 auth_method 0 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62bfff640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe62c0a4750 0x7fe62c136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe630c89640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c13cd70 0x7fe62c136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient(hunting): handle_auth_done global_id 54247 payload 824 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c13cd70 msgr2=0x7fe62c136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe62c13cd70 0x7fe62c136ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe62c0a4750 msgr2=0x7fe62c136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe62c0a4750 0x7fe62c136060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 monclient: found mon.noname-c 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:12.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c139d40 con 0x7fe62c0a5240 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62b7fe640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe614009db0 tx=0x7fe61400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 client.?.objecter ms_handle_connect 0x7fe62c0a5240 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe614018020 con 0x7fe62c0a5240 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: dump: 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.984 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:12.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61400cdb0 con 0x7fe62c0a5240 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe614014d60 con 0x7fe62c0a5240 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 5 monclient: authenticate success, global_id 54247 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe62a7fc640 4 set_mon_vals no callback set 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe628ff9640 10 monclient: dump: 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.986 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _renew_subs 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c13a240 con 0x7fe62c0a5240 2024-09-17T16:35:12.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 client.54247.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _renew_subs 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.958+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1379e0 con 0x7fe62c0a5240 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe6326ec640 1 librados: init done 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe6326ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe6326ec640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 0x7fe62c1336e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe62c051e80 con 0x7fe62c13c5b0 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe61401b020 con 0x7fe62c0a5240 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 0x7fe62c1336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient: get_auth_request con 0x7fe62c13c5b0 auth_method 0 2024-09-17T16:35:12.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe60c03e550 0x7fe60c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 10 monclient(hunting): handle_auth_done global_id 54179 payload 824 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(329..329 src has 1..329) ==== 5801+0+0 (secure 0 0 0) 0x7fe614050770 con 0x7fe62c0a5240 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 client.54247.objecter ms_dispatch 0x7fe62c000ba0 osd_map(329..329 src has 1..329) 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 3 client.54247.objecter handle_osd_map got epochs [329,329] > 0 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 3 client.54247.objecter handle_osd_map decoding full epoch 329 2024-09-17T16:35:12.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe62c139560 con 0x7fe62c13c5b0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 20 client.54247.objecter dump_active .. 0 homeless 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 client.54247.objecter ms_handle_connect 0x7fe62c13c5b0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 client.54247.objecter resend_mon_ops 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe62bfff640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 0x7fe62c1336e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d6f0 tx=0x7fe61c00dbb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe630c89640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe60c03e550 0x7fe60c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe630c89640 10 monclient: get_auth_request con 0x7fe60c03e550 auth_method 0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe61c005d30 con 0x7fe62c13c5b0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 monclient: dump: 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.990 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.991 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe61c017070 con 0x7fe62c13c5b0 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 msgr2=0x7fe62c1336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 0x7fe62c1336e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d6f0 tx=0x7fe61c00dbb0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 msgr2=0x7fe62c1336e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c13c5b0 0x7fe62c1336e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe630c89640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe60c03e550 0x7fe60c040a10 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7fe6240ad000 tx=0x7fe6240a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.962+0000 7fe628ff9640 10 client.54247.objecter ms_handle_connect 0x7fe60c03e550 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.966+0000 7fe6326ec640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.966+0000 7fe6326ec640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 0x7fe62c134030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.966+0000 7fe6326ec640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.966+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe62c1399c0 con 0x7fe62c133c20 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 0x7fe62c134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient: get_auth_request con 0x7fe62c133c20 auth_method 0 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:12.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 10 monclient(hunting): handle_auth_done global_id 54188 payload 824 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 1 -- 172.21.15.105:0/3508132749 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe62c0a4b30 con 0x7fe62c133c20 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 client.54247.objecter ms_handle_connect 0x7fe62c133c20 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 client.54247.objecter resend_mon_ops 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe62bfff640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 0x7fe62c134030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe61c016000 tx=0x7fe61c005800 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe61c01ce10 con 0x7fe62c133c20 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 monclient: dump: 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:12.995 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:12.996 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7fe61c01ce10 con 0x7fe62c133c20 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 msgr2=0x7fe62c134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 0x7fe62c134030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe61c016000 tx=0x7fe61c005800 comp rx=0 tx=0).stop 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 msgr2=0x7fe62c134030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe628ff9640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe62c133c20 0x7fe62c134030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19967, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:13.002 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.003 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:13.004 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:13.006 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:13.007 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.009 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.010 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.011 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.012 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.014 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.015 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:13.016 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.018 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:13.019 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.020 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:13.021 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 10 librados: watch_flush enter 2024-09-17T16:35:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 10 librados: watch_flush exit 2024-09-17T16:35:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 20 client.54247.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 10 client.54247.objecter successfully canceled tick 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe60c03e550 msgr2=0x7fe60c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe60c03e550 0x7fe60c040a10 secure :-1 s=READY pgs=3500 cs=0 l=1 rev1=1 crypto rx=0x7fe6240ad000 tx=0x7fe6240a6000 comp rx=0 tx=0).stop 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 10 monclient: shutdown 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 msgr2=0x7fe62c1365a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe614009db0 tx=0x7fe61400c490 comp rx=0 tx=0).stop 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe630c89640 1 -- 172.21.15.105:0/3508132749 reap_dead start 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 shutdown_connections 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 --2- 172.21.15.105:0/3508132749 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe62c0a5240 0x7fe62c1365a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 >> 172.21.15.105:0/3508132749 conn(0x7fe62c0a5840 msgr2=0x7fe62c13d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 shutdown_connections 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 -- 172.21.15.105:0/3508132749 wait complete. 2024-09-17T16:35:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:12.970+0000 7fe6326ec640 1 librados: shutdown 2024-09-17T16:35:13.024 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:13.024 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:35:13.024 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:35:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 10 monclient: get_monmap_and_config 2024-09-17T16:35:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 10 monclient: monmap: 2024-09-17T16:35:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:13.138404+0000 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:13.138404+0000 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:13.143 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 1 Processor -- start 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 1 -- start start 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.134+0000 7f3c4ed18640 10 monclient: init 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient: _add_conns ranks=[1,0,7,5,6,2,8,9,10,3,4] 2024-09-17T16:35:13.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3c480a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c480a73d0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c48142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-i con 0x7f3c480a48a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c480a4d30 con 0x7f3c480a73d0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c480a7b80 con 0x7f3c480a68e0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3c480a2ea0 con 0x7f3c480a48a0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 10 monclient(hunting): authenticate will time out at 1396.404230s 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3c480a73d0 0x7f3c4813eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41860/0 (socket says 172.21.15.105:41860) 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 1 -- 172.21.15.105:0/4197989787 learned_addr learned my addr 172.21.15.105:0/4197989787 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c48142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request con 0x7f3c480a48a0 auth_method 0 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807852285 0 0) 0x7f3c480a4d30 con 0x7f3c480a73d0 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): _negotiate 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): my global_id is 35415 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c34003530 con 0x7f3c480a73d0 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1065408561 0 0) 0x7f3c34003530 con 0x7f3c480a73d0 2024-09-17T16:35:13.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 msgr2=0x7f3c48142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c48142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 >> v1:172.21.15.160:6789/0 conn(0x7f3c480a68e0 legacy=0x7f3c480a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c480a3620 con 0x7f3c480a73d0 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f3c30001970 con 0x7f3c480a73d0 2024-09-17T16:35:13.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 5 monclient: authenticate success, global_id 35415 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: dump: 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 >> v1:172.21.15.105:6789/0 conn(0x7f3c480a73d0 legacy=0x7f3c4813eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: _add_conns ranks=[8,7,5,2,1,0,4,6,3] 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c34004240 0x7f3c34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): picked mon.e con 0x7f3c34004240 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c34004e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): picked mon.i con 0x7f3c480a48a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c340099c0 0x7f3c3400bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): picked mon.h con 0x7f3c340099c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f3c3400c3f0 con 0x7f3c340099c0 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3c3400c570 con 0x7f3c480a48a0 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3c3400c720 con 0x7f3c34004240 2024-09-17T16:35:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4ed18640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c34004240 0x7f3c34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): get_auth_request con 0x7f3c34004240 auth_method 0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c34004e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c340099c0 0x7f3c3400bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 10 monclient(hunting): get_auth_request con 0x7f3c480a48a0 auth_method 0 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request con 0x7f3c340099c0 auth_method 0 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_done global_id 35415 payload 293 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 -- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 msgr2=0x7f3c34004e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c34004e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 -- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c340099c0 msgr2=0x7f3c3400bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c340099c0 0x7f3c3400bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 monclient: found mon.e 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 -- 172.21.15.105:0/4197989787 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c34003530 con 0x7f3c34004240 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c47fff640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c480a48a0 0x7f3c34004e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c34004240 0x7f3c34004810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c40001730 tx=0x7f3c4001c300 comp rx=0 tx=0).ready entity=mon.8 client_cookie=eeb093ad5aa03fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c40001380 con 0x7f3c34004240 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.138+0000 7f3c477fe640 10 monclient: dump: 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.165 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.166 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40003600 con 0x7f3c34004240 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 1 -- 172.21.15.105:0/4197989787 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c40003c20 con 0x7f3c34004240 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 4 set_mon_vals no callback set 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c477fe640 10 monclient: dump: 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.167 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.168 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: shutdown 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- 172.21.15.105:0/4197989787 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c34004240 msgr2=0x7f3c34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 --2- 172.21.15.105:0/4197989787 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c34004240 0x7f3c34004810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c40001730 tx=0x7f3c4001c300 comp rx=0 tx=0).stop 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 1 -- 172.21.15.105:0/4197989787 reap_dead start 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- 172.21.15.105:0/4197989787 shutdown_connections 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- 172.21.15.105:0/4197989787 >> 172.21.15.105:0/4197989787 conn(0x7f3c480a96e0 msgr2=0x7f3c48068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- 172.21.15.105:0/4197989787 shutdown_connections 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- 172.21.15.105:0/4197989787 wait complete. 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: monmap: 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 librados: starting msgr at 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 librados: starting objecter 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 Processor -- start 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- start start 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 librados: setting wanted keys 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 librados: calling monclient init 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: init 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: _add_conns ranks=[7,5,1,0,6,4,3,8,2] 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c4813ef50 0x7f3c4813f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-i con 0x7f3c4813ef50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c481d0f30 0x7f3c48139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-h con 0x7f3c481d0f30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813a070 0x7f3c4813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): picked mon.noname-f con 0x7f3c4813a070 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480a87d0 con 0x7f3c4813a070 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f3c4806b5b0 con 0x7f3c481d0f30 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3c48142c40 con 0x7f3c4813ef50 2024-09-17T16:35:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient(hunting): authenticate will time out at 1396.410319s 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c481d0f30 0x7f3c48139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c481d0f30 0x7f3c48139b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:38832/0 (socket says 172.21.15.105:38832) 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 1 -- 172.21.15.105:0/522942512 learned_addr learned my addr 172.21.15.105:0/522942512 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): get_auth_request con 0x7f3c481d0f30 auth_method 0 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c4813ef50 0x7f3c4813f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813a070 0x7f3c4813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request con 0x7f3c4813a070 auth_method 0 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): get_auth_request con 0x7f3c4813ef50 auth_method 0 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient(hunting): handle_auth_done global_id 34855 payload 824 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c481d0f30 msgr2=0x7f3c48139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3c481d0f30 0x7f3c48139b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813a070 msgr2=0x7f3c4813a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813a070 0x7f3c4813a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 monclient: found mon.noname-i 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:35:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4813d7f0 con 0x7f3c4813ef50 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813a070 0x7f3c4813a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4cab4640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c4813ef50 0x7f3c4813f330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400c8f0 tx=0x7f3c34005840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 client.?.objecter ms_handle_connect 0x7f3c4813ef50 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c40006c50 con 0x7f3c4813ef50 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 monclient: dump: 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.177 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 5 monclient: authenticate success, global_id 34855 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c4ed18640 10 monclient: _renew_subs 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.142+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40007030 con 0x7f3c4813ef50 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4813fed0 con 0x7f3c4813ef50 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c40007650 con 0x7f3c4813ef50 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c46ffd640 4 set_mon_vals no callback set 2024-09-17T16:35:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 client.34855.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: dump: 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 monclient: _renew_subs 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4813cf50 con 0x7f3c4813ef50 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 1 librados: init done 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 0x7f3c4809e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48051e80 con 0x7f3c480a96e0 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 0x7f3c4809e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient: get_auth_request con 0x7f3c480a96e0 auth_method 0 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3c400071e0 con 0x7f3c4813ef50 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 10 monclient(hunting): handle_auth_done global_id 54199 payload 824 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c480008e0 con 0x7f3c480a96e0 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c2803e580 0x7f3c28040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(329..329 src has 1..329) ==== 5801+0+0 (secure 0 0 0) 0x7f3c400e9b40 con 0x7f3c4813ef50 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 client.34855.objecter ms_dispatch 0x7f3c48000ba0 osd_map(329..329 src has 1..329) 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 3 client.34855.objecter handle_osd_map got epochs [329,329] > 0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 3 client.34855.objecter handle_osd_map decoding full epoch 329 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c47fff640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 0x7f3c4809e300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c30011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 20 client.34855.objecter dump_active .. 0 homeless 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 client.34855.objecter ms_handle_connect 0x7f3c480a96e0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 client.34855.objecter resend_mon_ops 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c2803e580 0x7f3c28040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4d2b5640 10 monclient: get_auth_request con 0x7f3c2803e580 auth_method 0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c3000fd80 con 0x7f3c480a96e0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: dump: 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 client.34855.objecter ms_handle_connect 0x7f3c2803e580 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c4d2b5640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c2803e580 0x7f3c28040a40 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7f3c48069410 tx=0x7f3c3800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3c3001a070 con 0x7f3c480a96e0 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 msgr2=0x7f3c4809e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 0x7f3c4809e300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c30011cf0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 msgr2=0x7f3c4809e300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.146+0000 7f3c457fa640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c480a96e0 0x7f3c4809e300 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c30011cf0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 0x7f3c4809e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4813bff0 con 0x7f3c4813d190 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 0x7f3c4809e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient: get_auth_request con 0x7f3c4813d190 auth_method 0 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 10 monclient(hunting): handle_auth_done global_id 54208 payload 824 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 1 -- 172.21.15.105:0/522942512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3c480a6cc0 con 0x7f3c4813d190 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c47fff640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 0x7f3c4809e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c480a37f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 client.34855.objecter ms_handle_connect 0x7f3c4813d190 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 client.34855.objecter resend_mon_ops 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c30015af0 con 0x7f3c4813d190 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 monclient: dump: 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f3c30015680 con 0x7f3c4813d190 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 msgr2=0x7f3c4809e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 0x7f3c4809e840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c480a37f0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 msgr2=0x7f3c4809e840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c457fa640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c4813d190 0x7f3c4809e840 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c4813bb00 tx=0x7f3c480a37f0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 20159, 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:13.190 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.191 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.192 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:13.193 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.194 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:13.195 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.196 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.197 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.198 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.199 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.200 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.201 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.203 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.206 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.207 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.208 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.209 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.210 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:13.211 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 librados: watch_flush enter 2024-09-17T16:35:13.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 librados: watch_flush exit 2024-09-17T16:35:13.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 20 client.34855.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 client.34855.objecter successfully canceled tick 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c2803e580 msgr2=0x7f3c28040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c2803e580 0x7f3c28040a40 secure :-1 s=READY pgs=3501 cs=0 l=1 rev1=1 crypto rx=0x7f3c48069410 tx=0x7f3c3800b040 comp rx=0 tx=0).stop 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 10 monclient: shutdown 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c4813ef50 msgr2=0x7f3c4813f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 --2- 172.21.15.105:0/522942512 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c4813ef50 0x7f3c4813f330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400c8f0 tx=0x7f3c34005840 comp rx=0 tx=0).stop 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4d2b5640 1 -- 172.21.15.105:0/522942512 reap_dead start 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4d2b5640 1 -- 172.21.15.105:0/522942512 reap_dead start 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 shutdown_connections 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 >> 172.21.15.105:0/522942512 conn(0x7f3c34004240 msgr2=0x7f3c480a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 shutdown_connections 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 -- 172.21.15.105:0/522942512 wait complete. 2024-09-17T16:35:13.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.154+0000 7f3c4ed18640 1 librados: shutdown 2024-09-17T16:35:13.215 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:13.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:35:13.215 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:35:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.330+0000 7f177e715640 10 monclient: get_monmap_and_config 2024-09-17T16:35:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.330+0000 7f177e715640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.330+0000 7f177e715640 10 monclient: monmap: 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:13.335677+0000 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:13.335677+0000 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:13.340 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 Processor -- start 2024-09-17T16:35:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 -- start start 2024-09-17T16:35:13.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient: init 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient: _add_conns ranks=[6,0,5,3,2,1,10,4,7,9,8] 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 0x7f17780a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): picked mon.noname-d con 0x7f17780a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f17780a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f17780a7200 0x7f1778139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): picked mon.noname-h con 0x7f17780a7200 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177806ebc0 con 0x7f17780a4910 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f17780a3fc0 con 0x7f17780a7200 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f17780a63c0 con 0x7f17780a68e0 2024-09-17T16:35:13.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177e715640 10 monclient(hunting): authenticate will time out at 1396.601293s 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f17777fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f17780a4910 0x7f17781396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41870/0 (socket says 172.21.15.105:41870) 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 0x7f17780a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f17777fe640 1 -- 172.21.15.105:0/4077740958 learned_addr learned my addr 172.21.15.105:0/4077740958 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): get_auth_request con 0x7f17780a68e0 auth_method 0 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177ccb2640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f17780a7200 0x7f1778139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177ccb2640 10 monclient(hunting): get_auth_request con 0x7f17780a7200 auth_method 0 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177ccb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177ccb2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f177ccb2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient(hunting): handle_auth_done global_id 24306 payload 274 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 -- 172.21.15.105:0/4077740958 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f17780a7200 msgr2=0x7f1778139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f17780a7200 0x7f1778139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 -- 172.21.15.105:0/4077740958 >> v1:172.21.15.105:6789/0 conn(0x7f17780a4910 legacy=0x7f17781396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 monclient: found mon.noname-d 2024-09-17T16:35:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1776ffd640 1 -- 172.21.15.105:0/4077740958 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515912424 0 0) 0x7f177806ebc0 con 0x7f17780a4910 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 -- 172.21.15.105:0/4077740958 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780a3690 con 0x7f17780a68e0 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1776ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.334+0000 7f1777fff640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 0x7f17780a6cc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f17680097f0 tx=0x7f176801c370 comp rx=0 tx=0).ready entity=mon.6 client_cookie=db46b341102d8533 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 1 -- 172.21.15.105:0/4077740958 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680b0070 con 0x7f17780a68e0 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: dump: 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 5 monclient: authenticate success, global_id 24306 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 1 -- 172.21.15.105:0/4077740958 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680aa810 con 0x7f17780a68e0 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 4 set_mon_vals no callback set 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 1 -- 172.21.15.105:0/4077740958 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680a9300 con 0x7f17780a68e0 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1776ffd640 10 monclient: dump: 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: shutdown 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/4077740958 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 msgr2=0x7f17780a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 0x7f17780a6cc0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f17680097f0 tx=0x7f176801c370 comp rx=0 tx=0).stop 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/4077740958 shutdown_connections 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f17780a7200 0x7f1778139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- 172.21.15.105:0/4077740958 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f17780a68e0 0x7f17780a6cc0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/4077740958 >> 172.21.15.105:0/4077740958 conn(0x7f1778068ba0 msgr2=0x7f1778068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/4077740958 shutdown_connections 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/4077740958 wait complete. 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: monmap: 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 librados: starting msgr at 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 librados: starting objecter 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 Processor -- start 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- start start 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 librados: setting wanted keys 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 librados: calling monclient init 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: init 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _add_conns ranks=[2,7,3,6,8,4,0,1,5] 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 0x7f17781d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): picked mon.noname-b con 0x7f17780a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f17780a68e0 0x7f17781d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): picked mon.noname-i con 0x7f17780a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a7200 0x7f17781d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): picked mon.noname-g con 0x7f17780a7200 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f177813a2f0 con 0x7f17780a4910 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f17780a7610 con 0x7f17780a7200 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f177813a470 con 0x7f17780a68e0 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient(hunting): authenticate will time out at 1396.606405s 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 0x7f17781d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 0x7f17781d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:57320/0 (socket says 172.21.15.105:57320) 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 -- 172.21.15.105:0/660970349 learned_addr learned my addr 172.21.15.105:0/660970349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): get_auth_request con 0x7f17780a4910 auth_method 0 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17777fe640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f17780a68e0 0x7f17781d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17777fe640 10 monclient(hunting): get_auth_request con 0x7f17780a68e0 auth_method 0 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient(hunting): handle_auth_done global_id 44615 payload 824 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177ccb2640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a7200 0x7f17781d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f17780a68e0 msgr2=0x7f17781d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f17780a68e0 0x7f17781d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a7200 msgr2=0x7f17781d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a7200 0x7f17781d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 monclient: found mon.noname-b 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780a2370 con 0x7f17780a4910 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1777fff640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 0x7f17781d10c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17680ae040 tx=0x7f1768019110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 client.?.objecter ms_handle_connect 0x7f17780a4910 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680b0040 con 0x7f17780a4910 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: dump: 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680044b0 con 0x7f17780a4910 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 5 monclient: authenticate success, global_id 44615 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _renew_subs 2024-09-17T16:35:13.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17780a1e20 con 0x7f17780a4910 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17680aac90 con 0x7f17780a4910 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f1774ff9640 10 monclient: dump: 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.367 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 client.44615.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _renew_subs 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f177813fb30 con 0x7f17780a4910 2024-09-17T16:35:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f177e715640 1 librados: init done 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.338+0000 7f17767fc640 4 set_mon_vals no callback set 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177e715640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177e715640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 0x7f17780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1778123550 con 0x7f17780a0a20 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1768004040 con 0x7f17780a4910 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 0x7f17780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176403e550 0x7f1764040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient: get_auth_request con 0x7f17780a0a20 auth_method 0 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(329..329 src has 1..329) ==== 5801+0+0 (secure 0 0 0) 0x7f17680e9970 con 0x7f17780a4910 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 client.44615.objecter ms_dispatch 0x7f1778000ba0 osd_map(329..329 src has 1..329) 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 3 client.44615.objecter handle_osd_map got epochs [329,329] > 0 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 3 client.44615.objecter handle_osd_map decoding full epoch 329 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 20 client.44615.objecter dump_active .. 0 homeless 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177ccb2640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176403e550 0x7f1764040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177ccb2640 10 monclient: get_auth_request con 0x7f176403e550 auth_method 0 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 10 monclient(hunting): handle_auth_done global_id 54174 payload 824 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17780a6cc0 con 0x7f17780a0a20 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 client.44615.objecter ms_handle_connect 0x7f17780a0a20 2024-09-17T16:35:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 client.44615.objecter resend_mon_ops 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f17777fe640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 0x7f17780a0e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17780a2190 tx=0x7f176c0019c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 client.44615.objecter ms_handle_connect 0x7f176403e550 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f177ccb2640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176403e550 0x7f1764040a10 secure :-1 s=READY pgs=3502 cs=0 l=1 rev1=1 crypto rx=0x7f1770009e10 tx=0x7f177000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f176c019070 con 0x7f17780a0a20 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 monclient: dump: 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f176c010510 con 0x7f17780a0a20 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 msgr2=0x7f17780a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 0x7f17780a0e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17780a2190 tx=0x7f176c0019c0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 msgr2=0x7f17780a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.342+0000 7f1774ff9640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a0a20 0x7f17780a0e00 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17780a2190 tx=0x7f176c0019c0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 0x7f177813e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1778051e80 con 0x7f17780a1340 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 0x7f177813e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient: get_auth_request con 0x7f17780a1340 auth_method 0 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 10 monclient(hunting): handle_auth_done global_id 54183 payload 824 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 1 -- 172.21.15.105:0/660970349 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f177813a000 con 0x7f17780a1340 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 client.44615.objecter ms_handle_connect 0x7f17780a1340 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 client.44615.objecter resend_mon_ops 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f17777fe640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 0x7f177813e230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f176c007fd0 tx=0x7f176c000ee0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f176c019040 con 0x7f17780a1340 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 monclient: dump: 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f176c01b620 con 0x7f17780a1340 2024-09-17T16:35:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 msgr2=0x7f177813e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 0x7f177813e230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f176c007fd0 tx=0x7f176c000ee0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 msgr2=0x7f177813e230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f1774ff9640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17780a1340 0x7f177813e230 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f176c007fd0 tx=0x7f176c000ee0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:13.378 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 20355, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:35:13.379 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.380 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.381 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:13.382 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.383 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.384 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.385 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.386 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.387 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.389 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.390 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.391 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.392 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.393 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.394 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.395 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.397 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.399 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.400 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:13.401 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 librados: watch_flush enter 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 librados: watch_flush exit 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 20 client.44615.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 client.44615.objecter successfully canceled tick 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176403e550 msgr2=0x7f1764040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f176403e550 0x7f1764040a10 secure :-1 s=READY pgs=3502 cs=0 l=1 rev1=1 crypto rx=0x7f1770009e10 tx=0x7f177000b040 comp rx=0 tx=0).stop 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 10 monclient: shutdown 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 msgr2=0x7f17781d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 --2- 172.21.15.105:0/660970349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17780a4910 0x7f17781d10c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17680ae040 tx=0x7f1768019110 comp rx=0 tx=0).stop 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177ccb2640 1 -- 172.21.15.105:0/660970349 reap_dead start 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177ccb2640 1 -- 172.21.15.105:0/660970349 reap_dead start 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 shutdown_connections 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 >> 172.21.15.105:0/660970349 conn(0x7f1778068ba0 msgr2=0x7f1778141320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 shutdown_connections 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 -- 172.21.15.105:0/660970349 wait complete. 2024-09-17T16:35:13.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.350+0000 7f177e715640 1 librados: shutdown 2024-09-17T16:35:13.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:13.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:35:13.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: get_monmap_and_config 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: monmap: 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:13.520002+0000 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:13.520002+0000 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:13.523 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:13.524 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 Processor -- start 2024-09-17T16:35:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 -- start start 2024-09-17T16:35:13.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: init 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient: _add_conns ranks=[10,6,5,9,2,0,1,7,4,8,3] 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f01d00a6a30 0x7f01d00a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-f con 0x7f01d00a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-d con 0x7f01d00a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d0139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-h con 0x7f01d01397f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00a7c40 con 0x7f01d00a6a30 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f01d00a7890 con 0x7f01d01397f0 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f01d00a4080 con 0x7f01d00a4ab0 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01d5f8c640 10 monclient(hunting): authenticate will time out at 1396.785830s 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42884/0 (socket says 172.21.15.105:42884) 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 -- 172.21.15.105:0/1802874847 learned_addr learned my addr 172.21.15.105:0/1802874847 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): get_auth_request con 0x7f01d00a4ab0 auth_method 0 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/1802874847 >> v2:172.21.15.160:3300/0 conn(0x7f01d00a6a30 0x7f01d00a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cffff640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d0139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request con 0x7f01d00a6a30 auth_method 0 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cffff640 10 monclient(hunting): get_auth_request con 0x7f01d01397f0 auth_method 0 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient(hunting): handle_auth_done global_id 24315 payload 274 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 -- 172.21.15.105:0/1802874847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 msgr2=0x7f01d0139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d0139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 -- 172.21.15.105:0/1802874847 >> v2:172.21.15.160:3300/0 conn(0x7f01d00a6a30 msgr2=0x7f01d00a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 --2- 172.21.15.105:0/1802874847 >> v2:172.21.15.160:3300/0 conn(0x7f01d00a6a30 0x7f01d00a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 monclient: found mon.noname-d 2024-09-17T16:35:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:13.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.518+0000 7f01ceffd640 1 -- 172.21.15.105:0/1802874847 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00a37e0 con 0x7f01d00a4ab0 2024-09-17T16:35:13.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009870 tx=0x7f01c401c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d62b662f1dd902c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 1 -- 172.21.15.105:0/1802874847 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c40b2070 con 0x7f01d00a4ab0 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: dump: 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.537 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 1 -- 172.21.15.105:0/1802874847 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c40ab7b0 con 0x7f01d00a4ab0 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 1 -- 172.21.15.105:0/1802874847 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c40abe10 con 0x7f01d00a4ab0 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 5 monclient: authenticate success, global_id 24315 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 4 set_mon_vals no callback set 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ce7fc640 10 monclient: dump: 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.539 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: shutdown 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- 172.21.15.105:0/1802874847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 msgr2=0x7f01d00a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009870 tx=0x7f01c401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- 172.21.15.105:0/1802874847 shutdown_connections 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d0139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/1802874847 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a4ab0 0x7f01d00a7350 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/1802874847 >> v2:172.21.15.160:3300/0 conn(0x7f01d00a6a30 0x7f01d00a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- 172.21.15.105:0/1802874847 >> 172.21.15.105:0/1802874847 conn(0x7f01d00a8d00 msgr2=0x7f01d00a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- 172.21.15.105:0/1802874847 shutdown_connections 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- 172.21.15.105:0/1802874847 wait complete. 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: monmap: 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.541 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 librados: starting msgr at 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 librados: starting objecter 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 Processor -- start 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- start start 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 librados: setting wanted keys 2024-09-17T16:35:13.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 librados: calling monclient init 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: init 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient: _add_conns ranks=[7,6,5,1,3,8,4,0,2] 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01d00a4ab0 0x7f01d0138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-i con 0x7f01d00a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 0x7f01d0138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-d con 0x7f01d00a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d013d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): picked mon.noname-h con 0x7f01d01397f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f01d006e770 con 0x7f01d01397f0 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f01d00a2fc0 con 0x7f01d00a6a30 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f01d0143260 con 0x7f01d00a4ab0 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01d5f8c640 10 monclient(hunting): authenticate will time out at 1396.790908s 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 0x7f01d0138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 0x7f01d0138840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:42896/0 (socket says 172.21.15.105:42896) 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 -- 172.21.15.105:0/2959104193 learned_addr learned my addr 172.21.15.105:0/2959104193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): get_auth_request con 0x7f01d00a6a30 auth_method 0 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01d00a4ab0 0x7f01d0138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request con 0x7f01d00a4ab0 auth_method 0 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient(hunting): handle_auth_done global_id 24324 payload 824 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cffff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d013d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01d00a4ab0 msgr2=0x7f01d0138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01d00a4ab0 0x7f01d0138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 msgr2=0x7f01d013d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d01397f0 0x7f01d013d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 monclient: found mon.noname-d 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d013faf0 con 0x7f01d00a6a30 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01ceffd640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 0x7f01d0138840 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f01c40b0040 tx=0x7f01c4019190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 client.?.objecter ms_handle_connect 0x7f01d00a6a30 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c40b2070 con 0x7f01d00a6a30 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: dump: 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.546 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4003620 con 0x7f01d00a6a30 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c4003c40 con 0x7f01d00a6a30 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01b7fff640 10 monclient: dump: 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.522+0000 7f01cdffb640 4 set_mon_vals no callback set 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 5 monclient: authenticate success, global_id 24324 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 monclient: _renew_subs 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d0124010 con 0x7f01d00a6a30 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 client.24324.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 monclient: _renew_subs 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00abd50 con 0x7f01d00a6a30 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 1 librados: init done 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 0x7f01d013e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f01d0051e80 con 0x7f01d013dce0 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 0x7f01d013e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f01c40bf060 con 0x7f01d00a6a30 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient: get_auth_request con 0x7f01d013dce0 auth_method 0 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01bc03e580 0x7f01bc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(329..329 src has 1..329) ==== 5801+0+0 (secure 0 0 0) 0x7f01c40e99e0 con 0x7f01d00a6a30 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 client.24324.objecter ms_dispatch 0x7f01d0000ba0 osd_map(329..329 src has 1..329) 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 3 client.24324.objecter handle_osd_map got epochs [329,329] > 0 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 3 client.24324.objecter handle_osd_map decoding full epoch 329 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 20 client.24324.objecter dump_active .. 0 homeless 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cffff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01bc03e580 0x7f01bc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cffff640 10 monclient: get_auth_request con 0x7f01bc03e580 auth_method 0 2024-09-17T16:35:13.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_done global_id 19958 payload 824 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01d0139ca0 con 0x7f01d013dce0 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 client.24324.objecter ms_handle_connect 0x7f01d013dce0 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 client.24324.objecter resend_mon_ops 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 0x7f01d013e0c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c000f970 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c0019020 con 0x7f01d013dce0 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 monclient: dump: 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01cffff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01bc03e580 0x7f01bc040a40 secure :-1 s=READY pgs=3503 cs=0 l=1 rev1=1 crypto rx=0x7f01d0122750 tx=0x7f01c800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 client.24324.objecter ms_handle_connect 0x7f01bc03e580 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01c0014670 con 0x7f01d013dce0 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 msgr2=0x7f01d013e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 0x7f01d013e0c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c000f970 comp rx=0 tx=0).stop 2024-09-17T16:35:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 msgr2=0x7f01d013e0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.526+0000 7f01b7fff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013dce0 0x7f01d013e0c0 secure :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c000f970 comp rx=0 tx=0).stop 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01d5f8c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 0x7f01d00a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01d5f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f01d006a200 con 0x7f01d013e600 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 0x7f01d00a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient: get_auth_request con 0x7f01d013e600 auth_method 0 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 10 monclient(hunting): handle_auth_done global_id 19967 payload 824 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 1 -- 172.21.15.105:0/2959104193 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01d0074850 con 0x7f01d013e600 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01cf7fe640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 0x7f01d00a0a20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c00046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 10 client.24324.objecter ms_handle_connect 0x7f01d013e600 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 10 client.24324.objecter resend_mon_ops 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01c0019040 con 0x7f01d013e600 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.534+0000 7f01b7fff640 10 monclient: dump: 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.557 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9289 (secure 0 0 0) 0x7f01c0004050 con 0x7f01d013e600 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:13.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 msgr2=0x7f01d00a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 0x7f01d00a0a20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c00046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 msgr2=0x7f01d00a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01b7fff640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f01d013e600 0x7f01d00a0a20 secure :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f01c000f4b0 tx=0x7f01c00046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 461141, 2024-09-17T16:35:13.559 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 7, 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:13.560 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:13.561 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:13.563 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.564 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.566 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:13.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.569 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:13.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.572 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:13.575 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.576 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.577 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.578 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.579 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.580 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:13.581 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 10 librados: watch_flush enter 2024-09-17T16:35:13.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 10 librados: watch_flush exit 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 20 client.24324.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 10 client.24324.objecter successfully canceled tick 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01bc03e580 msgr2=0x7f01bc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f01bc03e580 0x7f01bc040a40 secure :-1 s=READY pgs=3503 cs=0 l=1 rev1=1 crypto rx=0x7f01d0122750 tx=0x7f01c800b040 comp rx=0 tx=0).stop 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 10 monclient: shutdown 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 msgr2=0x7f01d0138840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 --2- 172.21.15.105:0/2959104193 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01d00a6a30 0x7f01d0138840 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f01c40b0040 tx=0x7f01c4019190 comp rx=0 tx=0).stop 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01cffff640 1 -- 172.21.15.105:0/2959104193 reap_dead start 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01cffff640 1 -- 172.21.15.105:0/2959104193 reap_dead start 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 shutdown_connections 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 >> 172.21.15.105:0/2959104193 conn(0x7f01d00a8d00 msgr2=0x7f01d00a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 shutdown_connections 2024-09-17T16:35:13.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 -- 172.21.15.105:0/2959104193 wait complete. 2024-09-17T16:35:13.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.538+0000 7f01d5f8c640 1 librados: shutdown 2024-09-17T16:35:13.585 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:13.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:35:13.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: get_monmap_and_config 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: monmap: 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:13.705685+0000 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:13.705685+0000 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 Processor -- start 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 -- start start 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: init 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient: _add_conns ranks=[4,10,5,8,9,0,7,1,6,2,3] 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-c con 0x7eff4809b510 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7eff480a25a0 0x7eff480a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-f con 0x7eff480a25a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7eff480a42c0 0x7eff480a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-h con 0x7eff480a42c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7eff4809f3b0 con 0x7eff480a25a0 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7eff480a18e0 con 0x7eff4809b510 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7eff480a1bc0 con 0x7eff480a42c0 2024-09-17T16:35:13.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff4db9d640 10 monclient(hunting): authenticate will time out at 1396.971481s 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:36450/0 (socket says 172.21.15.105:36450) 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 1 -- 172.21.15.105:0/2080255903 learned_addr learned my addr 172.21.15.105:0/2080255903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): get_auth_request con 0x7eff4809b510 auth_method 0 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff47fff640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7eff480a42c0 0x7eff480a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff47fff640 10 monclient(hunting): get_auth_request con 0x7eff480a42c0 auth_method 0 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.702+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2080255903 >> v2:172.21.15.160:3300/0 conn(0x7eff480a25a0 0x7eff480a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff46ffd640 10 monclient(hunting): get_auth_request con 0x7eff480a25a0 auth_method 0 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 10 monclient(hunting): handle_auth_done global_id 54256 payload 274 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 -- 172.21.15.105:0/2080255903 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7eff480a42c0 msgr2=0x7eff480a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7eff480a42c0 0x7eff480a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 -- 172.21.15.105:0/2080255903 >> v2:172.21.15.160:3300/0 conn(0x7eff480a25a0 msgr2=0x7eff480a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 --2- 172.21.15.105:0/2080255903 >> v2:172.21.15.160:3300/0 conn(0x7eff480a25a0 0x7eff480a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 monclient: found mon.noname-c 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:35:13.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 -- 172.21.15.105:0/2080255903 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff480a5c20 con 0x7eff4809b510 2024-09-17T16:35:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff477fe640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff380098a0 tx=0x7eff3801c400 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1dcc7d8c415b9055 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 1 -- 172.21.15.105:0/2080255903 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff380b0070 con 0x7eff4809b510 2024-09-17T16:35:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:35:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: dump: 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 1 -- 172.21.15.105:0/2080255903 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff380aa810 con 0x7eff4809b510 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 1 -- 172.21.15.105:0/2080255903 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff380a9300 con 0x7eff4809b510 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 5 monclient: authenticate success, global_id 54256 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 4 set_mon_vals no callback set 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff467fc640 10 monclient: dump: 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 10 monclient: shutdown 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 -- 172.21.15.105:0/2080255903 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 msgr2=0x7eff4809b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff380098a0 tx=0x7eff3801c400 comp rx=0 tx=0).stop 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 -- 172.21.15.105:0/2080255903 shutdown_connections 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7eff480a42c0 0x7eff480a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2080255903 >> v2:172.21.15.160:3300/0 conn(0x7eff480a25a0 0x7eff480a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2080255903 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7eff4809b510 0x7eff4809b8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 -- 172.21.15.105:0/2080255903 >> 172.21.15.105:0/2080255903 conn(0x7eff480a4e70 msgr2=0x7eff480a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 -- 172.21.15.105:0/2080255903 shutdown_connections 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 1 -- 172.21.15.105:0/2080255903 wait complete. 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.706+0000 7eff4db9d640 10 monclient: monmap: 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 librados: starting msgr at 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 librados: starting objecter 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 Processor -- start 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- start start 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 librados: setting wanted keys 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 librados: calling monclient init 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: init 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _add_conns ranks=[8,7,0,4,6,3,5,1,2] 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 0x7eff48093cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-e con 0x7eff4809b510 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff480a25a0 0x7eff48094200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-i con 0x7eff480a25a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff480a42c0 0x7eff48094740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): picked mon.noname-a con 0x7eff480a42c0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff480a0450 con 0x7eff480a42c0 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7eff480a02d0 con 0x7eff480a25a0 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7eff480a05d0 con 0x7eff4809b510 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): authenticate will time out at 1396.976779s 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 0x7eff48093cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 0x7eff48093cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:35444/0 (socket says 172.21.15.105:35444) 2024-09-17T16:35:13.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 -- 172.21.15.105:0/2392528965 learned_addr learned my addr 172.21.15.105:0/2392528965 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): get_auth_request con 0x7eff4809b510 auth_method 0 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff480a42c0 0x7eff48094740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): get_auth_request con 0x7eff480a42c0 auth_method 0 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff480a25a0 0x7eff48094200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): get_auth_request con 0x7eff480a25a0 auth_method 0 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient(hunting): handle_auth_done global_id 54197 payload 824 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff480a25a0 msgr2=0x7eff48094200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff480a25a0 0x7eff48094200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff480a42c0 msgr2=0x7eff48094740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff480a42c0 0x7eff48094740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 monclient: found mon.noname-e 2024-09-17T16:35:13.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff48096040 con 0x7eff4809b510 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff477fe640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 0x7eff48093cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff380097c0 tx=0x7eff380191a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 client.?.objecter ms_handle_connect 0x7eff4809b510 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff380b9070 con 0x7eff4809b510 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: dump: 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.734 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff380044b0 con 0x7eff4809b510 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 5 monclient: authenticate success, global_id 54197 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _renew_subs 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff480a5250 con 0x7eff4809b510 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff380b0040 con 0x7eff4809b510 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff457fa640 4 set_mon_vals no callback set 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 client.54197.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 monclient: dump: 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.736 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _renew_subs 2024-09-17T16:35:13.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7eff48097840 con 0x7eff4809b510 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 librados: init done 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 0x7eff48071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7eff48051e80 con 0x7eff48097ff0 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7eff380a4d10 con 0x7eff4809b510 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 0x7eff48071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient: get_auth_request con 0x7eff48097ff0 auth_method 0 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7eff3403e570 0x7eff34040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(330..330 src has 1..330) ==== 6173+0+0 (secure 0 0 0) 0x7eff380e8cf0 con 0x7eff4809b510 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 client.54197.objecter ms_dispatch 0x7eff48000ba0 osd_map(330..330 src has 1..330) 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 3 client.54197.objecter handle_osd_map got epochs [330,330] > 0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 3 client.54197.objecter handle_osd_map decoding full epoch 330 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 20 client.54197.objecter dump_active .. 0 homeless 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7eff3403e570 0x7eff34040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 10 monclient(hunting): handle_auth_done global_id 34891 payload 824 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff480a49a0 con 0x7eff48097ff0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff47fff640 10 monclient: get_auth_request con 0x7eff3403e570 auth_method 0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 client.54197.objecter ms_handle_connect 0x7eff48097ff0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 10 client.54197.objecter resend_mon_ops 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 0x7eff48071ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff4806d390 tx=0x7eff3c00f910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.710+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff3c019020 con 0x7eff48097ff0 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 monclient: dump: 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.740 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 client.54197.objecter ms_handle_connect 0x7eff3403e570 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff47fff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7eff3403e570 0x7eff34040a30 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7eff40003ab0 tx=0x7eff40009820 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7eff3c014670 con 0x7eff48097ff0 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 msgr2=0x7eff48071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 0x7eff48071ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff4806d390 tx=0x7eff3c00f910 comp rx=0 tx=0).stop 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 msgr2=0x7eff48071ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.714+0000 7eff2ffff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48097ff0 0x7eff48071ff0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff4806d390 tx=0x7eff3c00f910 comp rx=0 tx=0).stop 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff4db9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 0x7eff48072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff4db9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7eff4806a1c0 con 0x7eff48072530 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 0x7eff48072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient: get_auth_request con 0x7eff48072530 auth_method 0 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 10 monclient(hunting): handle_auth_done global_id 34900 payload 824 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.718+0000 7eff46ffd640 1 -- 172.21.15.105:0/2392528965 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7eff480a3d80 con 0x7eff48072530 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 client.54197.objecter ms_handle_connect 0x7eff48072530 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 client.54197.objecter resend_mon_ops 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff46ffd640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 0x7eff48072910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c10 tx=0x7eff3c0046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff3c019040 con 0x7eff48072530 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 monclient: dump: 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7eff3c01e420 con 0x7eff48072530 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 msgr2=0x7eff48072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 0x7eff48072910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c10 tx=0x7eff3c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 msgr2=0x7eff48072910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff2ffff640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7eff48072530 0x7eff48072910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:13.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 52, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 304524, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:35:13.747 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 8, 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.748 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.749 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:13.750 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:13.751 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:13.752 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.754 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.756 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.764 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.765 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.766 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:13.767 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:13.768 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:13.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:13.768 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:13.768 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 10 librados: watch_flush enter 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 10 librados: watch_flush exit 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 20 client.54197.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 10 client.54197.objecter successfully canceled tick 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7eff3403e570 msgr2=0x7eff34040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7eff3403e570 0x7eff34040a30 secure :-1 s=READY pgs=3504 cs=0 l=1 rev1=1 crypto rx=0x7eff40003ab0 tx=0x7eff40009820 comp rx=0 tx=0).stop 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 10 monclient: shutdown 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 msgr2=0x7eff48093cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 --2- 172.21.15.105:0/2392528965 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7eff4809b510 0x7eff48093cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff380097c0 tx=0x7eff380191a0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff47fff640 1 -- 172.21.15.105:0/2392528965 reap_dead start 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff47fff640 1 -- 172.21.15.105:0/2392528965 reap_dead start 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 shutdown_connections 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 >> 172.21.15.105:0/2392528965 conn(0x7eff480a4e70 msgr2=0x7eff480a7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 shutdown_connections 2024-09-17T16:35:13.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 -- 172.21.15.105:0/2392528965 wait complete. 2024-09-17T16:35:13.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.722+0000 7eff4db9d640 1 librados: shutdown 2024-09-17T16:35:13.771 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:35:13.771 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i'] 2024-09-17T16:35:13.771 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['d'] 2024-09-17T16:35:13.771 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-17T16:35:13.771 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-09-17T16:35:13.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: get_monmap_and_config 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: monmap: 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:13.881165+0000 2024-09-17T16:35:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:13.881165+0000 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:13.886 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 Processor -- start 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 -- start start 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: init 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient: _add_conns ranks=[10,9,0,5,3,7,6,2,1,8,4] 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe2500a68e0 0x7fe2500a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-f con 0x7fe2500a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2500a4910 0x7fe2500a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-a con 0x7fe2500a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe2501396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25006ebc0 con 0x7fe2501396a0 2024-09-17T16:35:13.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe2500a3fc0 con 0x7fe2500a4910 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe2500a63c0 con 0x7fe2500a68e0 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe25609a640 10 monclient(hunting): authenticate will time out at 1397.146853s 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe2501396a0 0x7fe250139b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41872/0 (socket says 172.21.15.105:41872) 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24ffff640 1 -- 172.21.15.105:0/1633928641 learned_addr learned my addr 172.21.15.105:0/1633928641 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 --2- 172.21.15.105:0/1633928641 >> v2:172.21.15.105:3300/0 conn(0x7fe2500a4910 0x7fe2500a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): get_auth_request con 0x7fe2500a4910 auth_method 0 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24f7fe640 1 --2- 172.21.15.105:0/1633928641 >> v2:172.21.15.160:3300/0 conn(0x7fe2500a68e0 0x7fe2500a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request con 0x7fe2500a68e0 auth_method 0 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient(hunting): handle_auth_done global_id 35442 payload 274 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 -- 172.21.15.105:0/1633928641 >> v2:172.21.15.160:3300/0 conn(0x7fe2500a68e0 msgr2=0x7fe2500a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 --2- 172.21.15.105:0/1633928641 >> v2:172.21.15.160:3300/0 conn(0x7fe2500a68e0 0x7fe2500a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 -- 172.21.15.105:0/1633928641 >> v1:172.21.15.105:6789/0 conn(0x7fe2501396a0 legacy=0x7fe250139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 monclient: found mon.noname-a 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24e7fc640 1 -- 172.21.15.105:0/1633928641 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856706613 0 0) 0x7fe25006ebc0 con 0x7fe2501396a0 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24effd640 1 -- 172.21.15.105:0/1633928641 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2500a3690 con 0x7fe2500a4910 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.878+0000 7fe24e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24effd640 1 --2- 172.21.15.105:0/1633928641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2500a4910 0x7fe2500a7200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe250074a10 tx=0x7fe24401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2e5c2c8a943687 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 1 -- 172.21.15.105:0/1633928641 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe2440b0070 con 0x7fe2500a4910 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: dump: 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.893 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 1 -- 172.21.15.105:0/1633928641 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2440a4e50 con 0x7fe2500a4910 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 5 monclient: authenticate success, global_id 35442 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 1 -- 172.21.15.105:0/1633928641 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe2440aac80 con 0x7fe2500a4910 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 4 set_mon_vals no callback set 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe24e7fc640 10 monclient: dump: 2024-09-17T16:35:13.895 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.896 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: shutdown 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- 172.21.15.105:0/1633928641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2500a4910 msgr2=0x7fe2500a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- 172.21.15.105:0/1633928641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2500a4910 0x7fe2500a7200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe250074a10 tx=0x7fe24401c340 comp rx=0 tx=0).stop 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- 172.21.15.105:0/1633928641 shutdown_connections 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- 172.21.15.105:0/1633928641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2500a4910 0x7fe2500a7200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- 172.21.15.105:0/1633928641 >> v2:172.21.15.160:3300/0 conn(0x7fe2500a68e0 0x7fe2500a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- 172.21.15.105:0/1633928641 >> 172.21.15.105:0/1633928641 conn(0x7fe250068ba0 msgr2=0x7fe250068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- 172.21.15.105:0/1633928641 shutdown_connections 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- 172.21.15.105:0/1633928641 wait complete. 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: build_initial_monmap 2024-09-17T16:35:13.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: monmap: 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:13.898 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 librados: starting msgr at 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 librados: starting objecter 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 Processor -- start 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- start start 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 librados: setting wanted keys 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 librados: calling monclient init 2024-09-17T16:35:13.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: init 2024-09-17T16:35:13.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:13.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient: _add_conns ranks=[3,5,1,8,0,4,7,2,6] 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe2500a4910 0x7fe25013d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-g con 0x7fe2500a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 0x7fe25013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-h con 0x7fe2500a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2501396a0 0x7fe25013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): picked mon.noname-f con 0x7fe2501396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501444b0 con 0x7fe2501396a0 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe250144330 con 0x7fe2500a4910 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe250144630 con 0x7fe2500a68e0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.882+0000 7fe25609a640 10 monclient(hunting): authenticate will time out at 1397.151979s 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 0x7fe25013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe2500a4910 0x7fe25013d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2501396a0 0x7fe25013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 0x7fe25013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:38886/0 (socket says 172.21.15.105:38886) 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 -- 172.21.15.105:0/1916452953 learned_addr learned my addr 172.21.15.105:0/1916452953 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): get_auth_request con 0x7fe2500a68e0 auth_method 0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request con 0x7fe2500a4910 auth_method 0 2024-09-17T16:35:13.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): get_auth_request con 0x7fe2501396a0 auth_method 0 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient(hunting): handle_auth_done global_id 19985 payload 824 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe2500a4910 msgr2=0x7fe25013d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe2500a4910 0x7fe25013d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2501396a0 msgr2=0x7fe25013dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2501396a0 0x7fe25013dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 monclient: found mon.noname-h 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:35:13.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2500abc60 con 0x7fe2500a68e0 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24effd640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 0x7fe25013d720 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe244009710 tx=0x7fe2440190e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 client.?.objecter ms_handle_connect 0x7fe2500a68e0 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe2440b0040 con 0x7fe2500a68e0 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: dump: 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.911 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: _finish_auth 0 2024-09-17T16:35:13.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 5 monclient: authenticate success, global_id 19985 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 monclient: _renew_subs 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe244003cd0 con 0x7fe2500a68e0 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe250122970 con 0x7fe2500a68e0 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe2440ab870 con 0x7fe2500a68e0 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 10 monclient: dump: 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.913 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.914 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24dffb640 4 set_mon_vals no callback set 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 client.19985.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 monclient: _renew_subs 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2500ab730 con 0x7fe2500a68e0 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 1 librados: init done 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 0x7fe250063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe250051e80 con 0x7fe25002e5d0 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 0x7fe250063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient: get_auth_request con 0x7fe25002e5d0 auth_method 0 2024-09-17T16:35:13.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_done global_id 34909 payload 824 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe25002e9b0 con 0x7fe25002e5d0 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe2440ab400 con 0x7fe2500a68e0 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe24f7fe640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 0x7fe250063e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe240010820 tx=0x7fe240010ce0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.886+0000 7fe237fff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe23c03e480 0x7fe23c040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 client.19985.objecter ms_handle_connect 0x7fe25002e5d0 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 client.19985.objecter resend_mon_ops 2024-09-17T16:35:13.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(330..330 src has 1..330) ==== 6173+0+0 (secure 0 0 0) 0x7fe2440e9a40 con 0x7fe2500a68e0 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 client.19985.objecter ms_dispatch 0x7fe250000ba0 osd_map(330..330 src has 1..330) 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 3 client.19985.objecter handle_osd_map got epochs [330,330] > 0 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 3 client.19985.objecter handle_osd_map decoding full epoch 330 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 20 client.19985.objecter dump_active .. 0 homeless 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe240018020 con 0x7fe25002e5d0 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 monclient: dump: 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.917 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe24ffff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe23c03e480 0x7fe23c040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe24ffff640 10 monclient: get_auth_request con 0x7fe23c03e480 auth_method 0 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe240004240 con 0x7fe25002e5d0 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 msgr2=0x7fe250063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 0x7fe250063e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe240010820 tx=0x7fe240010ce0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 msgr2=0x7fe250063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe25002e5d0 0x7fe250063e00 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe240010820 tx=0x7fe240010ce0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe24ffff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe23c03e480 0x7fe23c040940 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7fe25013f2c0 tx=0x7fe248004d50 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.890+0000 7fe237fff640 10 client.19985.objecter ms_handle_connect 0x7fe23c03e480 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.894+0000 7fe25609a640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.894+0000 7fe25609a640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 0x7fe2501db590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.894+0000 7fe25609a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:13.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.894+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe2500a9e70 con 0x7fe250064340 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 0x7fe2501db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient: get_auth_request con 0x7fe250064340 auth_method 0 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 10 monclient(hunting): handle_auth_done global_id 34918 payload 824 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 1 -- 172.21.15.105:0/1916452953 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fe250139b10 con 0x7fe250064340 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24f7fe640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 0x7fe2501db590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe240000c10 tx=0x7fe2400046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 client.19985.objecter ms_handle_connect 0x7fe250064340 2024-09-17T16:35:13.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 client.19985.objecter resend_mon_ops 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe240018040 con 0x7fe250064340 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 monclient: dump: 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:13.921 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fe24000ed80 con 0x7fe250064340 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 msgr2=0x7fe2501db590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 0x7fe2501db590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe240000c10 tx=0x7fe2400046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 msgr2=0x7fe2501db590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:13.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe237fff640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe250064340 0x7fe2501db590 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe240000c10 tx=0x7fe2400046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.923 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:13.923 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:35:13.923 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:35:13.923 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 10 librados: watch_flush enter 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 10 librados: watch_flush exit 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 20 client.19985.objecter shutdown clearing up homeless session... 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 10 client.19985.objecter successfully canceled tick 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe23c03e480 msgr2=0x7fe23c040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe23c03e480 0x7fe23c040940 secure :-1 s=READY pgs=3505 cs=0 l=1 rev1=1 crypto rx=0x7fe25013f2c0 tx=0x7fe248004d50 comp rx=0 tx=0).stop 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 10 monclient: shutdown 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 msgr2=0x7fe25013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:13.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 --2- 172.21.15.105:0/1916452953 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe2500a68e0 0x7fe25013d720 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe244009710 tx=0x7fe2440190e0 comp rx=0 tx=0).stop 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24ffff640 1 -- 172.21.15.105:0/1916452953 reap_dead start 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe24ffff640 1 -- 172.21.15.105:0/1916452953 reap_dead start 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 shutdown_connections 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 >> 172.21.15.105:0/1916452953 conn(0x7fe250068ba0 msgr2=0x7fe250141300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 shutdown_connections 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 -- 172.21.15.105:0/1916452953 wait complete. 2024-09-17T16:35:13.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:13.898+0000 7fe25609a640 1 librados: shutdown 2024-09-17T16:35:13.926 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-17T16:35:13.926 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-17T16:35:13.927 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:35:13.956 INFO:tasks.ceph.mon.i:Stopped 2024-09-17T16:35:13.956 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-17T16:35:13.956 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-17T16:35:13.957 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:35:28.957 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-17T16:35:28.958 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-17T16:35:28.958 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-17T16:35:28.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: get_monmap_and_config 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: build_initial_monmap 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: monmap: 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:29.088 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:29.085043+0000 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:29.085043+0000 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:29.089 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:29.090 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:29.090 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 Processor -- start 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 -- start start 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: init 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient: _add_conns ranks=[5,2,8,9,7,6,0,1,3,4,10] 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): picked mon.noname-h con 0x7fce740a3720 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:29.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): picked mon.noname-b con 0x7fce740a3b00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): picked mon.noname-e con 0x7fce740a3f00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fce7406db50 con 0x7fce740a3b00 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fce740a0e80 con 0x7fce740a3720 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fce740a1490 con 0x7fce740a3f00 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:29.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce79737640 10 monclient(hunting): authenticate will time out at 1412.350819s 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:39960/0 (socket says 172.21.15.105:39960) 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 learned_addr learned my addr 172.21.15.105:0/498165232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce740a3b00 auth_method 0 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce740a3f00 auth_method 0 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:29.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce740a3720 auth_method 0 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:29.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce740a3f00 auth_method 0 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce740a3720 auth_method 0 2024-09-17T16:35:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce740a3b00 auth_method 0 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:35:29.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:29.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce740a3b00 auth_method 0 2024-09-17T16:35:29.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce740a3f00 auth_method 0 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce740a3720 auth_method 0 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:29.690+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:30.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce740a3f00 auth_method 0 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce740a3b00 auth_method 0 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:30.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce740a3720 auth_method 0 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:30.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:30.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:30.494+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): tick 2024-09-17T16:35:32.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 msgr2=0x7fce740a5c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce740a3f00 0x7fce740a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 msgr2=0x7fce740a1e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fce740a3720 0x7fce740a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 msgr2=0x7fce740a56c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce740a3b00 0x7fce740a56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient: _add_conns ranks=[4,9,7,6,1,10,0,3] 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-c con 0x7fce5c003830 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:32.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-a con 0x7fce5c006220 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-i con 0x7fce5c008d10 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fce7406db50 con 0x7fce5c006220 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fce740a0e80 con 0x7fce5c003830 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fce740a1490 con 0x7fce5c008d10 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce70ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 msgr2=0x7fce5c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c003830 auth_method 0 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c006220 auth_method 0 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:32.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c006220 auth_method 0 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:32.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 msgr2=0x7fce5c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:32.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.286+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c003830 auth_method 0 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.290+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c006220 auth_method 0 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 msgr2=0x7fce5c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.690+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:32.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c003830 auth_method 0 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:32.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:32.694+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:35:33.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 msgr2=0x7fce5c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c006220 auth_method 0 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:33.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.494+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:33.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:33.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c003830 auth_method 0 2024-09-17T16:35:33.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:35:33.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:33.498+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:35:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): tick 2024-09-17T16:35:35.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 msgr2=0x7fce5c00b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fce5c008d10 0x7fce5c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 msgr2=0x7fce5c005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fce5c003830 0x7fce5c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 msgr2=0x7fce5c0086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.105:3300/0 conn(0x7fce5c006220 0x7fce5c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 reap_dead start 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient: _add_conns ranks=[6,3,10,1,0] 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce737fe640 1 -- 172.21.15.105:0/498165232 reap_dead start 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 0x7fce5c00bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-d con 0x7fce5c0091c0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce5c003c30 0x7fce5c0066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-g con 0x7fce5c003c30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.160:3300/0 conn(0x7fce5c006be0 0x7fce5c006fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): picked mon.noname-f con 0x7fce5c006be0 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fce7406db50 con 0x7fce5c006be0 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fce740a0e80 con 0x7fce5c003c30 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 1 -- 172.21.15.105:0/498165232 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fce740a1490 con 0x7fce5c0091c0 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce70ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 0x7fce5c00bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c0091c0 auth_method 0 2024-09-17T16:35:35.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.082+0000 7fce72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.160:3300/0 conn(0x7fce5c006be0 0x7fce5c006fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce5c003c30 0x7fce5c0066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c003c30 auth_method 0 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce5c006be0 auth_method 0 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient(hunting): handle_auth_done global_id 24333 payload 274 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce5c003c30 msgr2=0x7fce5c0066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce5c003c30 0x7fce5c0066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 >> v2:172.21.15.160:3300/0 conn(0x7fce5c006be0 msgr2=0x7fce5c006fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.160:3300/0 conn(0x7fce5c006be0 0x7fce5c006fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 monclient: found mon.noname-d 2024-09-17T16:35:35.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:35.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.086+0000 7fce72ffd640 1 -- 172.21.15.105:0/498165232 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce740a7240 con 0x7fce5c0091c0 2024-09-17T16:35:35.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce72ffd640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 0x7fce5c00bd90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0036e0 tx=0x7fce6401dc00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7b400b20e636e135 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 1 -- 172.21.15.105:0/498165232 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce640ac070 con 0x7fce5c0091c0 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: dump: 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.098 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 1 -- 172.21.15.105:0/498165232 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce640a6cc0 con 0x7fce5c0091c0 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce79737640 5 monclient: authenticate success, global_id 24333 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce79737640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce79737640 4 set_mon_vals no callback set 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 1 -- 172.21.15.105:0/498165232 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce6400f040 con 0x7fce5c0091c0 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.090+0000 7fce71ffb640 10 monclient: dump: 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.100 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.101 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: shutdown 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 msgr2=0x7fce5c00bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 0x7fce5c00bd90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0036e0 tx=0x7fce6401dc00 comp rx=0 tx=0).stop 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- 172.21.15.105:0/498165232 shutdown_connections 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- 172.21.15.105:0/498165232 >> v2:172.21.15.160:3300/0 conn(0x7fce5c006be0 0x7fce5c006fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce5c003c30 0x7fce5c0066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- 172.21.15.105:0/498165232 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fce5c0091c0 0x7fce5c00bd90 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- 172.21.15.105:0/498165232 >> 172.21.15.105:0/498165232 conn(0x7fce740aa390 msgr2=0x7fce740a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- 172.21.15.105:0/498165232 shutdown_connections 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- 172.21.15.105:0/498165232 wait complete. 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: monmap: 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.108 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 librados: starting msgr at 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 librados: starting objecter 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 Processor -- start 2024-09-17T16:35:35.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- start start 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 librados: setting wanted keys 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 librados: calling monclient init 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: init 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient: _add_conns ranks=[2,0,8,3,7,5,1,6,4] 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 0x7fce7412a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): picked mon.noname-b con 0x7fce5c003c30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 0x7fce74127e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): picked mon.noname-a con 0x7fce5c006be0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 0x7fce74128360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): picked mon.noname-e con 0x7fce5c0091c0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:35.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce740a6260 con 0x7fce5c006be0 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fce740a60e0 con 0x7fce5c003c30 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fce740a63e0 con 0x7fce5c0091c0 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce79737640 10 monclient(hunting): authenticate will time out at 1418.363387s 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 0x7fce74128360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 0x7fce7412a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 0x7fce74128360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:39272/0 (socket says 172.21.15.105:39272) 2024-09-17T16:35:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 0x7fce74127e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 1 -- 172.21.15.105:0/3884485015 learned_addr learned my addr 172.21.15.105:0/3884485015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): get_auth_request con 0x7fce5c003c30 auth_method 0 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): get_auth_request con 0x7fce5c0091c0 auth_method 0 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): get_auth_request con 0x7fce5c006be0 auth_method 0 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.094+0000 7fce737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 10 monclient(hunting): handle_auth_done global_id 44624 payload 824 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 -- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 msgr2=0x7fce74128360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 0x7fce74128360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 -- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 msgr2=0x7fce74127e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 0x7fce74127e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 monclient: found mon.noname-b 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 -- 172.21.15.105:0/3884485015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7412bba0 con 0x7fce5c003c30 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce727fc640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 0x7fce74127e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce72ffd640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 0x7fce7412a3a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007750 tx=0x7fce6401b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 client.?.objecter ms_handle_connect 0x7fce5c003c30 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce640be070 con 0x7fce5c003c30 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: dump: 2024-09-17T16:35:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce64003a90 con 0x7fce5c003c30 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 5 monclient: authenticate success, global_id 44624 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce640ac030 con 0x7fce5c003c30 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 monclient: dump: 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.116 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.117 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce70ff9640 4 set_mon_vals no callback set 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _renew_subs 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce740a04e0 con 0x7fce5c003c30 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 client.44624.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _renew_subs 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7412ad30 con 0x7fce5c003c30 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 1 librados: init done 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fce64003620 con 0x7fce5c003c30 2024-09-17T16:35:35.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7413c880 con 0x7fce5c003c30 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 0x7fce5c042a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(333..333 src has 1..333) ==== 6202+0+0 (secure 0 0 0) 0x7fce640b5040 con 0x7fce5c003c30 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 10 client.44624.objecter ms_dispatch 0x7fce74000ba0 osd_map(333..333 src has 1..333) 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 3 client.44624.objecter handle_osd_map got epochs [333,333] > 0 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 3 client.44624.objecter handle_osd_map decoding full epoch 333 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce5b7fe640 20 client.44624.objecter dump_active .. 0 homeless 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce727fc640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 0x7fce5c042a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.098+0000 7fce727fc640 10 monclient: get_auth_request con 0x7fce5c040570 auth_method 0 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.102+0000 7fce5b7fe640 10 client.44624.objecter ms_handle_connect 0x7fce5c040570 2024-09-17T16:35:35.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.102+0000 7fce727fc640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 0x7fce5c042a30 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7fce74129eb0 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.102+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce64009720 con 0x7fce5c003c30 2024-09-17T16:35:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.102+0000 7fce5b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.102+0000 7fce5b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce79737640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:35:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce79737640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce79737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fce74129840 con 0x7fce5c003c30 2024-09-17T16:35:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce5b7fe640 1 -- 172.21.15.105:0/3884485015 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3206 (secure 0 0 0) 0x7fce640a6c90 con 0x7fce5c003c30 2024-09-17T16:35:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce5b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:35:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.350+0000 7fce5b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:35.358 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"h","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 10 librados: watch_flush enter 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 10 librados: watch_flush exit 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 20 client.44624.objecter shutdown clearing up homeless session... 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 10 client.44624.objecter successfully canceled tick 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 msgr2=0x7fce5c042a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 0x7fce5c042a30 secure :-1 s=READY pgs=3506 cs=0 l=1 rev1=1 crypto rx=0x7fce74129eb0 tx=0x7fce68008040 comp rx=0 tx=0).stop 2024-09-17T16:35:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 10 monclient: shutdown 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 msgr2=0x7fce7412a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 0x7fce7412a3a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce5c007750 tx=0x7fce6401b840 comp rx=0 tx=0).stop 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 shutdown_connections 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fce5c040570 0x7fce5c042a30 unknown :-1 s=CLOSED pgs=3506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fce5c0091c0 0x7fce74128360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c006be0 0x7fce74127e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 --2- 172.21.15.105:0/3884485015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c003c30 0x7fce7412a3a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 >> 172.21.15.105:0/3884485015 conn(0x7fce740aa390 msgr2=0x7fce7409fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 shutdown_connections 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 -- 172.21.15.105:0/3884485015 wait complete. 2024-09-17T16:35:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.354+0000 7fce79737640 1 librados: shutdown 2024-09-17T16:35:35.376 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-17T16:35:35.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-17T16:35:35.377 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: get_monmap_and_config 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: monmap: 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:35.495939+0000 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:35.495939+0000 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.499 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:35.500 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 Processor -- start 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 -- start start 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: init 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient: _add_conns ranks=[9,10,5,6,1,7,3,0,2,8,4] 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7500a2de0 0x7fe7500a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): picked mon.noname-a con 0x7fe7500a2de0 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 0x7fe75009b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): picked mon.noname-f con 0x7fe7500a3700 addr v2:172.21.15.160:3300/0 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3ae0 0x7fe75009b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): picked mon.noname-h con 0x7fe7500a3ae0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe75009c9a0 con 0x7fe7500a2de0 2024-09-17T16:35:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe7500a20b0 con 0x7fe7500a3700 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe7500a2770 con 0x7fe7500a3ae0 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe755056640 10 monclient(hunting): authenticate will time out at 1418.761591s 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7500a2de0 0x7fe7500a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7500a2de0 0x7fe7500a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35874/0 (socket says 172.21.15.105:35874) 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 -- 172.21.15.105:0/25779897 learned_addr learned my addr 172.21.15.105:0/25779897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request con 0x7fe7500a2de0 auth_method 0 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74f59e640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3ae0 0x7fe75009b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74e59c640 1 --2- 172.21.15.105:0/25779897 >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 0x7fe75009b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_done global_id 35469 payload 274 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 -- 172.21.15.105:0/25779897 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3ae0 msgr2=0x7fe75009b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3ae0 0x7fe75009b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 -- 172.21.15.105:0/25779897 >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 msgr2=0x7fe75009b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/25779897 >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 0x7fe75009b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 monclient: found mon.noname-a 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74ed9d640 1 -- 172.21.15.105:0/25779897 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500a24f0 con 0x7fe7500a2de0 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74e59c640 10 monclient: get_auth_request con 0x7fe7500a3700 auth_method 0 2024-09-17T16:35:35.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.494+0000 7fe74e59c640 1 --2- 172.21.15.105:0/25779897 >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 0x7fe75009b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7500a2de0 0x7fe7500a31c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a93d0 tx=0x7fe74801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a82881cf5628c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 1 -- 172.21.15.105:0/25779897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7480090d0 con 0x7fe7500a2de0 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: dump: 2024-09-17T16:35:35.513 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.514 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 1 -- 172.21.15.105:0/25779897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7480a59a0 con 0x7fe7500a2de0 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 1 -- 172.21.15.105:0/25779897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7480ad3f0 con 0x7fe7500a2de0 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 5 monclient: authenticate success, global_id 35469 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 4 set_mon_vals no callback set 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74dd9b640 10 monclient: dump: 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: shutdown 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- 172.21.15.105:0/25779897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7500a2de0 msgr2=0x7fe7500a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7500a2de0 0x7fe7500a31c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe7500a93d0 tx=0x7fe74801c8b0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- 172.21.15.105:0/25779897 shutdown_connections 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3ae0 0x7fe75009b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- 172.21.15.105:0/25779897 >> v2:172.21.15.160:3300/0 conn(0x7fe7500a3700 0x7fe75009b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- 172.21.15.105:0/25779897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7500a2de0 0x7fe7500a31c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- 172.21.15.105:0/25779897 >> 172.21.15.105:0/25779897 conn(0x7fe7500a83e0 msgr2=0x7fe7500a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- 172.21.15.105:0/25779897 shutdown_connections 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- 172.21.15.105:0/25779897 wait complete. 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: monmap: 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 librados: starting msgr at 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 librados: starting objecter 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 Processor -- start 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- start start 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 librados: setting wanted keys 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 librados: calling monclient init 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: init 2024-09-17T16:35:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient: _add_conns ranks=[1,5,2,4,6,7,0,3,8] 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7500a2de0 0x7fe7501d1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): picked mon.noname-f con 0x7fe7500a2de0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3700 0x7fe7501d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): picked mon.noname-h con 0x7fe7500a3700 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 0x7fe75013c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): picked mon.noname-b con 0x7fe7500a3ae0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501413d0 con 0x7fe7500a2de0 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe75009bee0 con 0x7fe7500a3ae0 2024-09-17T16:35:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe750141550 con 0x7fe7500a3700 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe755056640 10 monclient(hunting): authenticate will time out at 1418.767235s 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 0x7fe75013c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 0x7fe75013c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:40002/0 (socket says 172.21.15.105:40002) 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 learned_addr learned my addr 172.21.15.105:0/4010703023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): get_auth_request con 0x7fe7500a3ae0 auth_method 0 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7500a2de0 0x7fe7501d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request con 0x7fe7500a2de0 auth_method 0 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74e59c640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3700 0x7fe7501d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient(hunting): handle_auth_done global_id 44633 payload 824 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3700 msgr2=0x7fe7501d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe7500a3700 0x7fe7501d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7500a2de0 msgr2=0x7fe7501d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7500a2de0 0x7fe7501d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 monclient: found mon.noname-b 2024-09-17T16:35:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500a12b0 con 0x7fe7500a3ae0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe74f59e640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 0x7fe75013c3f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c5c0 tx=0x7fe74000ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe7377fe640 10 client.?.objecter ms_handle_connect 0x7fe7500a3ae0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.498+0000 7fe7377fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe740015070 con 0x7fe7500a3ae0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: dump: 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.523 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe740011a70 con 0x7fe7500a3ae0 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 5 monclient: authenticate success, global_id 44633 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 monclient: _renew_subs 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7500a1590 con 0x7fe7500a3ae0 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe740011c60 con 0x7fe7500a3ae0 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: dump: 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.525 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74cd99640 4 set_mon_vals no callback set 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 client.44633.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 monclient: _renew_subs 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75009b070 con 0x7fe7500a3ae0 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 1 librados: init done 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 0x7fe75013c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe750051e80 con 0x7fe75009fed0 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 0x7fe75013c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient: get_auth_request con 0x7fe75009fed0 auth_method 0 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe740011600 con 0x7fe7500a3ae0 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_done global_id 35478 payload 824 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7500a31c0 con 0x7fe75009fed0 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe73c03e550 0x7fe73c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(334..334 src has 1..334) ==== 5817+0+0 (secure 0 0 0) 0x7fe74004f810 con 0x7fe7500a3ae0 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 client.44633.objecter ms_dispatch 0x7fe750000ba0 osd_map(334..334 src has 1..334) 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 3 client.44633.objecter handle_osd_map got epochs [334,334] > 0 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 3 client.44633.objecter handle_osd_map decoding full epoch 334 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 0x7fe75013c930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ae000 tx=0x7fe748019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 20 client.44633.objecter dump_active .. 0 homeless 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 client.44633.objecter ms_handle_connect 0x7fe75009fed0 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 client.44633.objecter resend_mon_ops 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7480bd030 con 0x7fe75009fed0 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74e59c640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe73c03e550 0x7fe73c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: dump: 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.529 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74e59c640 10 monclient: get_auth_request con 0x7fe73c03e550 auth_method 0 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe748004040 con 0x7fe75009fed0 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:35.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 msgr2=0x7fe75013c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 0x7fe75013c930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ae000 tx=0x7fe748019650 comp rx=0 tx=0).stop 2024-09-17T16:35:35.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 msgr2=0x7fe75013c930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75009fed0 0x7fe75013c930 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe7480ae000 tx=0x7fe748019650 comp rx=0 tx=0).stop 2024-09-17T16:35:35.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe74e59c640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe73c03e550 0x7fe73c040a10 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.502+0000 7fe7377fe640 10 client.44633.objecter ms_handle_connect 0x7fe73c03e550 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe755056640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe755056640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 0x7fe75013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe755056640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe75006a1c0 con 0x7fe75013ce70 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 0x7fe75013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe74ed9d640 10 monclient: get_auth_request con 0x7fe75013ce70 auth_method 0 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe74ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe74ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.506+0000 7fe74ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 10 monclient(hunting): handle_auth_done global_id 35487 payload 824 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 1 -- 172.21.15.105:0/4010703023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe75013d7c0 con 0x7fe75013ce70 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 client.44633.objecter ms_handle_connect 0x7fe75013ce70 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74ed9d640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 0x7fe75013d280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe748000f90 tx=0x7fe7480a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 client.44633.objecter resend_mon_ops 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7480bd040 con 0x7fe75013ce70 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 monclient: dump: 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7fe7480c3810 con 0x7fe75013ce70 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 msgr2=0x7fe75013d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 0x7fe75013d280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe748000f90 tx=0x7fe7480a7000 comp rx=0 tx=0).stop 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 msgr2=0x7fe75013d280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe7377fe640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75013ce70 0x7fe75013d280 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe748000f90 tx=0x7fe7480a7000 comp rx=0 tx=0).stop 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:35.536 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 326313, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.537 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:35.544 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.545 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.545 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.547 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.547 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.547 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.549 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.550 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.552 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.553 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:35.554 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:35.556 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:35.558 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:35.559 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.560 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:35.561 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 10 librados: watch_flush enter 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 10 librados: watch_flush exit 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 20 client.44633.objecter shutdown clearing up homeless session... 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 10 client.44633.objecter successfully canceled tick 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe73c03e550 msgr2=0x7fe73c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe73c03e550 0x7fe73c040a10 secure :-1 s=READY pgs=3507 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744008040 comp rx=0 tx=0).stop 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 10 monclient: shutdown 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 msgr2=0x7fe75013c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 --2- 172.21.15.105:0/4010703023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7500a3ae0 0x7fe75013c3f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c5c0 tx=0x7fe74000ca80 comp rx=0 tx=0).stop 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 reap_dead start 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe74f59e640 1 -- 172.21.15.105:0/4010703023 reap_dead start 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 shutdown_connections 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 >> 172.21.15.105:0/4010703023 conn(0x7fe7500a83e0 msgr2=0x7fe75013dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 shutdown_connections 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 -- 172.21.15.105:0/4010703023 wait complete. 2024-09-17T16:35:35.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.510+0000 7fe755056640 1 librados: shutdown 2024-09-17T16:35:35.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:35.564 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-17T16:35:35.565 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: get_monmap_and_config 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: monmap: 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.680 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:35.677242+0000 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:35.677242+0000 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:35.681 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:35.682 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 Processor -- start 2024-09-17T16:35:35.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 -- start start 2024-09-17T16:35:35.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: init 2024-09-17T16:35:35.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient: _add_conns ranks=[2,9,1,3,8,5,0,7,6,10,4] 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-b con 0x7f23e80a5860 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 0x7f23e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-a con 0x7f23e80a18f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f23e80a4070 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e80a1d70 con 0x7f23e80a4070 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f23e80a66c0 con 0x7f23e80a18f0 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23e80a33d0 con 0x7f23e80a5860 2024-09-17T16:35:35.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23f0305640 10 monclient(hunting): authenticate will time out at 1418.943665s 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 0x7f23e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:40008/0 (socket says 172.21.15.105:40008) 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 1 -- 172.21.15.105:0/1965543971 learned_addr learned my addr 172.21.15.105:0/1965543971 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): get_auth_request con 0x7f23e80a18f0 auth_method 0 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 10 monclient(hunting): get_auth_request con 0x7f23e80a5860 auth_method 0 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ee0a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.674+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_done global_id 44642 payload 274 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 -- 172.21.15.105:0/1965543971 >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 msgr2=0x7f23e80a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 --2- 172.21.15.105:0/1965543971 >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 0x7f23e80a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 -- 172.21.15.105:0/1965543971 >> v1:172.21.15.160:6789/0 conn(0x7f23e80a4070 legacy=0x7f23e8137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 monclient: found mon.noname-b 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:35.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 -- 172.21.15.105:0/1965543971 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e80a36b0 con 0x7f23e80a5860 2024-09-17T16:35:35.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/1965543971 >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 0x7f23e80a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ee0a1640 1 --2- 172.21.15.105:0/1965543971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93be0dc175fc6186 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 1 -- 172.21.15.105:0/1965543971 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23e40b0070 con 0x7f23e80a5860 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: dump: 2024-09-17T16:35:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.700 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.701 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 1 -- 172.21.15.105:0/1965543971 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e40aa810 con 0x7f23e80a5860 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 5 monclient: authenticate success, global_id 44642 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 1 -- 172.21.15.105:0/1965543971 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23e40a9300 con 0x7f23e80a5860 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 4 set_mon_vals no callback set 2024-09-17T16:35:35.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23ed09f640 10 monclient: dump: 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 10 monclient: shutdown 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 -- 172.21.15.105:0/1965543971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 msgr2=0x7f23e80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 --2- 172.21.15.105:0/1965543971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e401c4b0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 -- 172.21.15.105:0/1965543971 shutdown_connections 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 --2- 172.21.15.105:0/1965543971 >> v2:172.21.15.105:3300/0 conn(0x7f23e80a18f0 0x7f23e80a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 --2- 172.21.15.105:0/1965543971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e80a5860 0x7f23e80a5c40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 -- 172.21.15.105:0/1965543971 >> 172.21.15.105:0/1965543971 conn(0x7f23e80aa450 msgr2=0x7f23e80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 -- 172.21.15.105:0/1965543971 shutdown_connections 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 1 -- 172.21.15.105:0/1965543971 wait complete. 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.678+0000 7f23f0305640 10 monclient: monmap: 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 librados: starting msgr at 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 librados: starting objecter 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 Processor -- start 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- start start 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 librados: setting wanted keys 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 librados: calling monclient init 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: init 2024-09-17T16:35:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _add_conns ranks=[0,7,1,5,4,6,2,8,3] 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 0x7f23e81cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-a con 0x7f23e80a18f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f23e80a4070 0x7f23e81cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-i con 0x7f23e80a4070 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e80a5860 0x7f23e81cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): picked mon.noname-f con 0x7f23e80a5860 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8142490 con 0x7f23e80a18f0 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8142310 con 0x7f23e80a5860 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f23e8142610 con 0x7f23e80a4070 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): authenticate will time out at 1418.948798s 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 0x7f23e81cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 0x7f23e81cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35906/0 (socket says 172.21.15.105:35906) 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 -- 172.21.15.105:0/4165918196 learned_addr learned my addr 172.21.15.105:0/4165918196 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): get_auth_request con 0x7f23e80a18f0 auth_method 0 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f23e80a4070 msgr2=0x7f23e81cfa60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f23e80a4070 0x7f23e81cfa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient(hunting): handle_auth_done global_id 35505 payload 824 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f23e80a4070 msgr2=0x7f23e81cfa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f23e80a4070 0x7f23e81cfa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e80a5860 msgr2=0x7f23e81cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23e80a5860 0x7f23e81cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 monclient: found mon.noname-a 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81205d0 con 0x7f23e80a18f0 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee0a1640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 0x7f23e81cf520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e4019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 client.?.objecter ms_handle_connect 0x7f23e80a18f0 2024-09-17T16:35:35.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23e40b0040 con 0x7f23e80a18f0 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: dump: 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.711 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 5 monclient: authenticate success, global_id 35505 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e40aae30 con 0x7f23e80a18f0 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23e40044b0 con 0x7f23e80a18f0 2024-09-17T16:35:35.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ec89e640 4 set_mon_vals no callback set 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: dump: 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.713 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _renew_subs 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e813cc30 con 0x7f23e80a18f0 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 client.35505.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _renew_subs 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e813b360 con 0x7f23e80a18f0 2024-09-17T16:35:35.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 librados: init done 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 0x7f23e80725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23e813d5c0 con 0x7f23e8072210 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 0x7f23e80725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient: get_auth_request con 0x7f23e8072210 auth_method 0 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f23e4004040 con 0x7f23e80a18f0 2024-09-17T16:35:35.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_done global_id 44651 payload 824 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23e80a07c0 con 0x7f23e8072210 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23cc03e580 0x7f23cc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(334..334 src has 1..334) ==== 5817+0+0 (secure 0 0 0) 0x7f23e40e9fa0 con 0x7f23e80a18f0 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 client.35505.objecter ms_dispatch 0x7f23e8000ba0 osd_map(334..334 src has 1..334) 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 3 client.35505.objecter handle_osd_map got epochs [334,334] > 0 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 3 client.35505.objecter handle_osd_map decoding full epoch 334 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 0x7f23e80725f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23e813c9d0 tx=0x7f23d800ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 20 client.35505.objecter dump_active .. 0 homeless 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 client.35505.objecter ms_handle_connect 0x7f23e8072210 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 client.35505.objecter resend_mon_ops 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23d8017070 con 0x7f23e8072210 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:35.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: dump: 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.717 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee8a2640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23cc03e580 0x7f23cc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23ee8a2640 10 monclient: get_auth_request con 0x7f23cc03e580 auth_method 0 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f23d800ee50 con 0x7f23e8072210 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 msgr2=0x7f23e80725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 0x7f23e80725f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23e813c9d0 tx=0x7f23d800ec00 comp rx=0 tx=0).stop 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 msgr2=0x7f23e80725f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.682+0000 7f23deffd640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072210 0x7f23e80725f0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23e813c9d0 tx=0x7f23d800ec00 comp rx=0 tx=0).stop 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.686+0000 7f23deffd640 10 client.35505.objecter ms_handle_connect 0x7f23cc03e580 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.686+0000 7f23ee8a2640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23cc03e580 0x7f23cc040a40 secure :-1 s=READY pgs=3508 cs=0 l=1 rev1=1 crypto rx=0x7f23e0001ca0 tx=0x7f23e0007d00 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 0x7f23e813b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23e8051e80 con 0x7f23e8072b30 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 0x7f23e813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient: get_auth_request con 0x7f23e8072b30 auth_method 0 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 10 monclient(hunting): handle_auth_done global_id 44660 payload 824 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 1 -- 172.21.15.105:0/4165918196 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f23e813db60 con 0x7f23e8072b30 2024-09-17T16:35:35.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ed8a0640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 0x7f23e813b530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23d8019000 tx=0x7f23d8004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 client.35505.objecter ms_handle_connect 0x7f23e8072b30 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 client.35505.objecter resend_mon_ops 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23d8017070 con 0x7f23e8072b30 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 monclient: dump: 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.720 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9610 (secure 0 0 0) 0x7f23d800ce50 con 0x7f23e8072b30 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 msgr2=0x7f23e813b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 0x7f23e813b530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23d8019000 tx=0x7f23d8004740 comp rx=0 tx=0).stop 2024-09-17T16:35:35.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 msgr2=0x7f23e813b530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23deffd640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23e8072b30 0x7f23e813b530 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23d8019000 tx=0x7f23d8004740 comp rx=0 tx=0).stop 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 186342, 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:35.722 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.723 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.724 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:35.725 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:35.726 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.727 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.728 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.730 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:35.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:35.734 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.736 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.738 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:35.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.741 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.742 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.743 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:35.744 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 librados: watch_flush enter 2024-09-17T16:35:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 librados: watch_flush exit 2024-09-17T16:35:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 20 client.35505.objecter shutdown clearing up homeless session... 2024-09-17T16:35:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 client.35505.objecter successfully canceled tick 2024-09-17T16:35:35.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23cc03e580 msgr2=0x7f23cc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23cc03e580 0x7f23cc040a40 secure :-1 s=READY pgs=3508 cs=0 l=1 rev1=1 crypto rx=0x7f23e0001ca0 tx=0x7f23e0007d00 comp rx=0 tx=0).stop 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 10 monclient: shutdown 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 msgr2=0x7f23e81cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 --2- 172.21.15.105:0/4165918196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23e80a18f0 0x7f23e81cf520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e4019250 comp rx=0 tx=0).stop 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ee8a2640 1 -- 172.21.15.105:0/4165918196 reap_dead start 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23ee8a2640 1 -- 172.21.15.105:0/4165918196 reap_dead start 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 shutdown_connections 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 >> 172.21.15.105:0/4165918196 conn(0x7f23e80aa450 msgr2=0x7f23e80a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 shutdown_connections 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 -- 172.21.15.105:0/4165918196 wait complete. 2024-09-17T16:35:35.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.690+0000 7f23f0305640 1 librados: shutdown 2024-09-17T16:35:35.748 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:35.748 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-17T16:35:35.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:35:35.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: get_monmap_and_config 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: monmap: 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:35.857909+0000 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:35.857909+0000 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:35.862 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 Processor -- start 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 -- start start 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: init 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient: _add_conns ranks=[6,0,9,1,2,4,8,5,3,10,7] 2024-09-17T16:35:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-d con 0x7f0f2c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f2c0a6cc0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f2c13eaf0 0x7f0f2c140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-a con 0x7f0f2c13eaf0 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f2c0a7b80 con 0x7f0f2c0a6cc0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2c0a2ea0 con 0x7f0f2c13eaf0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0f2c069ab0 con 0x7f0f2c0a68e0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f32e9d640 10 monclient(hunting): authenticate will time out at 1419.123598s 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:56460/0 (socket says 172.21.15.105:56460) 2024-09-17T16:35:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 1 -- 172.21.15.105:0/4256285802 learned_addr learned my addr 172.21.15.105:0/4256285802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f23fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0f2c0a6cc0 0x7f0f2c0a70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37468/0 (socket says 172.21.15.105:37468) 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): get_auth_request con 0x7f0f2c0a68e0 auth_method 0 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 1 --2- 172.21.15.105:0/4256285802 >> v2:172.21.15.105:3300/0 conn(0x7f0f2c13eaf0 0x7f0f2c140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): get_auth_request con 0x7f0f2c13eaf0 auth_method 0 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.854+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 10 monclient(hunting): handle_auth_done global_id 24342 payload 274 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 -- 172.21.15.105:0/4256285802 >> v2:172.21.15.105:3300/0 conn(0x7f0f2c13eaf0 msgr2=0x7f0f2c140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 --2- 172.21.15.105:0/4256285802 >> v2:172.21.15.105:3300/0 conn(0x7f0f2c13eaf0 0x7f0f2c140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 -- 172.21.15.105:0/4256285802 >> v1:172.21.15.105:6789/0 conn(0x7f0f2c0a6cc0 legacy=0x7f0f2c0a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 monclient: found mon.noname-d 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 -- 172.21.15.105:0/4256285802 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c0a3620 con 0x7f0f2c0a68e0 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f30c39640 1 --2- 172.21.15.105:0/4256285802 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f2801c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ef72f14708b4db46 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 1 -- 172.21.15.105:0/4256285802 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f280b0070 con 0x7f0f2c0a68e0 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: dump: 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.876 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 1 -- 172.21.15.105:0/4256285802 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f280aa810 con 0x7f0f2c0a68e0 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 5 monclient: authenticate success, global_id 24342 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:35.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 1 -- 172.21.15.105:0/4256285802 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f280a9300 con 0x7f0f2c0a68e0 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 4 set_mon_vals no callback set 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f237fe640 10 monclient: dump: 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.878 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 10 monclient: shutdown 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- 172.21.15.105:0/4256285802 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 msgr2=0x7f0f2c0a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/4256285802 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f2801c420 comp rx=0 tx=0).stop 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- 172.21.15.105:0/4256285802 shutdown_connections 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/4256285802 >> v2:172.21.15.105:3300/0 conn(0x7f0f2c13eaf0 0x7f0f2c140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/4256285802 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0f2c0a68e0 0x7f0f2c0a48a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- 172.21.15.105:0/4256285802 >> 172.21.15.105:0/4256285802 conn(0x7f0f2c0a96e0 msgr2=0x7f0f2c068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- 172.21.15.105:0/4256285802 shutdown_connections 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- 172.21.15.105:0/4256285802 wait complete. 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 10 monclient: build_initial_monmap 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 10 monclient: monmap: 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:35.880 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 librados: starting msgr at 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 librados: starting objecter 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 Processor -- start 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 -- start start 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 librados: setting wanted keys 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 1 librados: calling monclient init 2024-09-17T16:35:35.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.858+0000 7f0f32e9d640 10 monclient: init 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _add_conns ranks=[5,3,8,7,4,2,0,1,6] 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0f2c0a6cc0 0x7f0f2c13c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-h con 0x7f0f2c0a6cc0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f2c13eaf0 0x7f0f2c136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-g con 0x7f0f2c13eaf0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 0x7f0f2c136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): picked mon.noname-e con 0x7f0f2c136bb0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c143fd0 con 0x7f0f2c13eaf0 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0f2c143e50 con 0x7f0f2c0a6cc0 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0f2c144150 con 0x7f0f2c136bb0 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): authenticate will time out at 1419.128352s 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 0x7f0f2c136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 0x7f0f2c136ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:39288/0 (socket says 172.21.15.105:39288) 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 learned_addr learned my addr 172.21.15.105:0/1518467516 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): get_auth_request con 0x7f0f2c136bb0 auth_method 0 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0f2c0a6cc0 0x7f0f2c13c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient(hunting): handle_auth_done global_id 54215 payload 824 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f23fff640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f2c13eaf0 0x7f0f2c136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0f2c0a6cc0 msgr2=0x7f0f2c13c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0f2c0a6cc0 0x7f0f2c13c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f2c13eaf0 msgr2=0x7f0f2c136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f2c13eaf0 0x7f0f2c136670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient: _finish_hunting 0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 monclient: found mon.noname-e 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c13c0f0 con 0x7f0f2c136bb0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f3143a640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 0x7f0f2c136ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400efc0 tx=0x7f0f2400c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.?.objecter ms_handle_connect 0x7f0f2c136bb0 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f24018020 con 0x7f0f2c136bb0 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: dump: 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.885 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: _finish_auth 0 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: _check_auth_tickets 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2400cd30 con 0x7f0f2c136bb0 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f24014d60 con 0x7f0f2c136bb0 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: dump: 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 5 monclient: authenticate success, global_id 54215 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _renew_subs 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f22ffd640 4 set_mon_vals no callback set 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c13ac70 con 0x7f0f2c136bb0 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 client.54215.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _renew_subs 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c13d8b0 con 0x7f0f2c136bb0 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 librados: init done 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 0x7f0f2c138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0f2c051e80 con 0x7f0f2c13bb30 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 0x7f0f2c138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0f2401b020 con 0x7f0f2c136bb0 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient: get_auth_request con 0x7f0f2c13bb30 auth_method 0 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f1803e550 0x7f0f18040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 10 monclient(hunting): handle_auth_done global_id 54265 payload 824 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f2c13a070 con 0x7f0f2c13bb30 2024-09-17T16:35:35.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(334..334 src has 1..334) ==== 5817+0+0 (secure 0 0 0) 0x7f0f24050c20 con 0x7f0f2c136bb0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.54215.objecter ms_dispatch 0x7f0f2c000ba0 osd_map(334..334 src has 1..334) 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 3 client.54215.objecter handle_osd_map got epochs [334,334] > 0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 3 client.54215.objecter handle_osd_map decoding full epoch 334 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 20 client.54215.objecter dump_active .. 0 homeless 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.54215.objecter ms_handle_connect 0x7f0f2c13bb30 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.54215.objecter resend_mon_ops 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f30c39640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 0x7f0f2c138580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c069470 tx=0x7f0f280191c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f23fff640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f1803e550 0x7f0f18040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f23fff640 10 monclient: get_auth_request con 0x7f0f1803e550 auth_method 0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f280b0040 con 0x7f0f2c13bb30 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:35.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: dump: 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.891 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0f28003620 con 0x7f0f2c13bb30 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 msgr2=0x7f0f2c138580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 0x7f0f2c138580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c069470 tx=0x7f0f280191c0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 msgr2=0x7f0f2c138580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c13bb30 0x7f0f2c138580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f217fa640 10 client.54215.objecter ms_handle_connect 0x7f0f1803e550 2024-09-17T16:35:35.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.862+0000 7f0f23fff640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f1803e550 0x7f0f18040a10 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c139ce0 tx=0x7f0f1c009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:35.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 0x7f0f2c138ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0f2c0a3c70 con 0x7f0f2c0a96e0 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 0x7f0f2c138ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient: get_auth_request con 0x7f0f2c0a96e0 auth_method 0 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 10 monclient(hunting): handle_auth_done global_id 54274 payload 824 2024-09-17T16:35:35.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 1 -- 172.21.15.105:0/1518467516 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0f2c13b800 con 0x7f0f2c0a96e0 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 client.54215.objecter ms_handle_connect 0x7f0f2c0a96e0 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f30c39640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 0x7f0f2c138ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f280a8000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 client.54215.objecter resend_mon_ops 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f280bb040 con 0x7f0f2c0a96e0 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 monclient: dump: 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:35.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9442 (secure 0 0 0) 0x7f0f280bcbe0 con 0x7f0f2c0a96e0 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:35.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 msgr2=0x7f0f2c138ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 0x7f0f2c138ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009870 tx=0x7f0f280a8000 comp rx=0 tx=0).stop 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 msgr2=0x7f0f2c138ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f217fa640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f2c0a96e0 0x7f0f2c138ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:35.899 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 42867, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:35.901 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.902 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:35.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:35.904 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:35.910 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.912 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.915 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:35.917 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:35.922 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 librados: watch_flush enter 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 librados: watch_flush exit 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 20 client.54215.objecter shutdown clearing up homeless session... 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 client.54215.objecter successfully canceled tick 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f1803e550 msgr2=0x7f0f18040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f1803e550 0x7f0f18040a10 secure :-1 s=READY pgs=3509 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c139ce0 tx=0x7f0f1c009040 comp rx=0 tx=0).stop 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 10 monclient: shutdown 2024-09-17T16:35:35.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 msgr2=0x7f0f2c136ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 --2- 172.21.15.105:0/1518467516 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f2c136bb0 0x7f0f2c136ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400efc0 tx=0x7f0f2400c490 comp rx=0 tx=0).stop 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 reap_dead start 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f3143a640 1 -- 172.21.15.105:0/1518467516 reap_dead start 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 shutdown_connections 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 >> 172.21.15.105:0/1518467516 conn(0x7f0f2c0a68e0 msgr2=0x7f0f2c0a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 shutdown_connections 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 -- 172.21.15.105:0/1518467516 wait complete. 2024-09-17T16:35:35.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:35.870+0000 7f0f32e9d640 1 librados: shutdown 2024-09-17T16:35:35.925 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:35.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-17T16:35:35.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 10 monclient: get_monmap_and_config 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 10 monclient: monmap: 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:36.038110+0000 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:36.038110+0000 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:36.042 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 1 Processor -- start 2024-09-17T16:35:36.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 1 -- start start 2024-09-17T16:35:36.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.034+0000 7fcb2fc03640 10 monclient: init 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient: _add_conns ranks=[5,6,1,9,10,2,3,7,8,0,4] 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb280a4750 0x7fcb280a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-h con 0x7fcb280a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-d con 0x7fcb2809e740 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcb2809e0b0 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb280a0a80 con 0x7fcb2809e0b0 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fcb2809ef60 con 0x7fcb280a4750 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fcb280a31c0 con 0x7fcb2809e740 2024-09-17T16:35:36.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient(hunting): authenticate will time out at 1419.304199s 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:56462/0 (socket says 172.21.15.105:56462) 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 -- 172.21.15.105:0/89934228 learned_addr learned my addr 172.21.15.105:0/89934228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): get_auth_request con 0x7fcb2809e740 auth_method 0 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb280a4750 0x7fcb280a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d99f640 10 monclient(hunting): get_auth_request con 0x7fcb280a4750 auth_method 0 2024-09-17T16:35:36.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d99f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d99f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d99f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_done global_id 24351 payload 274 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 -- 172.21.15.105:0/89934228 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb280a4750 msgr2=0x7fcb280a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb280a4750 0x7fcb280a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 -- 172.21.15.105:0/89934228 >> v1:172.21.15.160:6789/0 conn(0x7fcb2809e0b0 legacy=0x7fcb28136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 monclient: found mon.noname-d 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 -- 172.21.15.105:0/89934228 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb280a6710 con 0x7fcb2809e740 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2d19e640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00d180 tx=0x7fcb1c01eb10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=582eb826e5be9153 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 1 -- 172.21.15.105:0/89934228 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb1c0a7620 con 0x7fcb2809e740 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: dump: 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.054 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 5 monclient: authenticate success, global_id 24351 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 1 -- 172.21.15.105:0/89934228 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c0a77e0 con 0x7fcb2809e740 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 1 -- 172.21.15.105:0/89934228 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb1c0a7e40 con 0x7fcb2809e740 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 4 set_mon_vals no callback set 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2c99d640 10 monclient: dump: 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.056 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.057 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 10 monclient: shutdown 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- 172.21.15.105:0/89934228 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 msgr2=0x7fcb280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00d180 tx=0x7fcb1c01eb10 comp rx=0 tx=0).stop 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- 172.21.15.105:0/89934228 shutdown_connections 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2809e740 0x7fcb280a5070 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/89934228 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb280a4750 0x7fcb280a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- 172.21.15.105:0/89934228 >> 172.21.15.105:0/89934228 conn(0x7fcb280a5840 msgr2=0x7fcb280a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- 172.21.15.105:0/89934228 shutdown_connections 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.038+0000 7fcb2fc03640 1 -- 172.21.15.105:0/89934228 wait complete. 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: monmap: 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.058 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 librados: starting msgr at 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 librados: starting objecter 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 Processor -- start 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- start start 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 librados: setting wanted keys 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 librados: calling monclient init 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: init 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _add_conns ranks=[5,0,3,1,7,2,8,6,4] 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb2809e0b0 0x7fcb281352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-h con 0x7fcb2809e0b0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 0x7fcb281357f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-a con 0x7fcb2809e740 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:36.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb280a4750 0x7fcb2813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): picked mon.noname-g con 0x7fcb280a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2806ccb0 con 0x7fcb2809e740 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2806cb30 con 0x7fcb280a4750 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fcb2806ce30 con 0x7fcb2809e0b0 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): authenticate will time out at 1419.309262s 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 0x7fcb281357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 0x7fcb281357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35930/0 (socket says 172.21.15.105:35930) 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 -- 172.21.15.105:0/1649800785 learned_addr learned my addr 172.21.15.105:0/1649800785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): get_auth_request con 0x7fcb2809e740 auth_method 0 2024-09-17T16:35:36.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb2809e0b0 0x7fcb281352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2e1a0640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb280a4750 0x7fcb2813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient(hunting): handle_auth_done global_id 35532 payload 824 2024-09-17T16:35:36.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb2809e0b0 msgr2=0x7fcb281352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb2809e0b0 0x7fcb281352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb280a4750 msgr2=0x7fcb2813a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb280a4750 0x7fcb2813a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 monclient: found mon.noname-a 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb28123d60 con 0x7fcb2809e740 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2e1a0640 10 monclient: get_auth_request con 0x7fcb280a4750 auth_method 0 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient: get_auth_request con 0x7fcb2809e0b0 auth_method 0 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2e1a0640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb280a4750 0x7fcb2813a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fcb2809e0b0 0x7fcb281352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d19e640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 0x7fcb281357f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c000f90 tx=0x7fcb1c01b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 client.?.objecter ms_handle_connect 0x7fcb2809e740 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb1c0a7620 con 0x7fcb2809e740 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: dump: 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.064 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 5 monclient: authenticate success, global_id 35532 2024-09-17T16:35:36.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c003e10 con 0x7fcb2809e740 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _renew_subs 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb280a5e20 con 0x7fcb2809e740 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb1c0044b0 con 0x7fcb2809e740 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: dump: 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.066 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1b7fe640 4 set_mon_vals no callback set 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 client.35532.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _renew_subs 2024-09-17T16:35:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb28135d70 con 0x7fcb2809e740 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 librados: init done 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 0x7fcb2813a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fcb28051e80 con 0x7fcb2813a5b0 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 0x7fcb2813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient: get_auth_request con 0x7fcb2813a5b0 auth_method 0 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fcb1c0c0030 con 0x7fcb2809e740 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb0c03e480 0x7fcb0c040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_done global_id 24360 payload 824 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb280a89b0 con 0x7fcb2813a5b0 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(334..334 src has 1..334) ==== 5817+0+0 (secure 0 0 0) 0x7fcb1c0be020 con 0x7fcb2809e740 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 client.35532.objecter ms_dispatch 0x7fcb28000ba0 osd_map(334..334 src has 1..334) 2024-09-17T16:35:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 3 client.35532.objecter handle_osd_map got epochs [334,334] > 0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 3 client.35532.objecter handle_osd_map decoding full epoch 334 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 20 client.35532.objecter dump_active .. 0 homeless 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 client.35532.objecter ms_handle_connect 0x7fcb2813a5b0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 client.35532.objecter resend_mon_ops 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 0x7fcb2813a990 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000e740 tx=0x7fcb2000ec00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2e1a0640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb0c03e480 0x7fcb0c040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb20017020 con 0x7fcb2813a5b0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb1a7fc640 10 monclient: dump: 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.070 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.042+0000 7fcb2e1a0640 10 monclient: get_auth_request con 0x7fcb0c03e480 auth_method 0 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcb2000ed30 con 0x7fcb2813a5b0 2024-09-17T16:35:36.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 msgr2=0x7fcb2813a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 0x7fcb2813a990 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000e740 tx=0x7fcb2000ec00 comp rx=0 tx=0).stop 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 msgr2=0x7fcb2813a990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813a5b0 0x7fcb2813a990 secure :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000e740 tx=0x7fcb2000ec00 comp rx=0 tx=0).stop 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb1a7fc640 10 client.35532.objecter ms_handle_connect 0x7fcb0c03e480 2024-09-17T16:35:36.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.046+0000 7fcb2e1a0640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb0c03e480 0x7fcb0c040940 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fcb24003dc0 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 0x7fcb280720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fcb28136500 con 0x7fcb2813aed0 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 0x7fcb280720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient: get_auth_request con 0x7fcb2813aed0 auth_method 0 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 10 monclient(hunting): handle_auth_done global_id 24369 payload 824 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 1 -- 172.21.15.105:0/1649800785 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcb280a8410 con 0x7fcb2813aed0 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 client.35532.objecter ms_handle_connect 0x7fcb2813aed0 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2d99f640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 0x7fcb280720e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c10 tx=0x7fcb200046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 client.35532.objecter resend_mon_ops 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb20017040 con 0x7fcb2813aed0 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 monclient: dump: 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7fcb2000c950 con 0x7fcb2813aed0 2024-09-17T16:35:36.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 msgr2=0x7fcb280720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 0x7fcb280720e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c10 tx=0x7fcb200046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 msgr2=0x7fcb280720e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb1a7fc640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fcb2813aed0 0x7fcb280720e0 secure :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c10 tx=0x7fcb200046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:36.077 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 483689, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.078 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.079 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.080 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:36.081 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.082 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:36.083 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:36.100 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 librados: watch_flush enter 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 librados: watch_flush exit 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 20 client.35532.objecter shutdown clearing up homeless session... 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 client.35532.objecter successfully canceled tick 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb0c03e480 msgr2=0x7fcb0c040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fcb0c03e480 0x7fcb0c040940 secure :-1 s=READY pgs=3510 cs=0 l=1 rev1=1 crypto rx=0x7fcb24003dc0 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 10 monclient: shutdown 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 msgr2=0x7fcb281357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 --2- 172.21.15.105:0/1649800785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb2809e740 0x7fcb281357f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c000f90 tx=0x7fcb1c01b8b0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2e1a0640 1 -- 172.21.15.105:0/1649800785 reap_dead start 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2e1a0640 1 -- 172.21.15.105:0/1649800785 reap_dead start 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 shutdown_connections 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 >> 172.21.15.105:0/1649800785 conn(0x7fcb280a5840 msgr2=0x7fcb280a1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 shutdown_connections 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 -- 172.21.15.105:0/1649800785 wait complete. 2024-09-17T16:35:36.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.050+0000 7fcb2fc03640 1 librados: shutdown 2024-09-17T16:35:36.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:36.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-17T16:35:36.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:35:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: get_monmap_and_config 2024-09-17T16:35:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: monmap: 2024-09-17T16:35:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:36.209419+0000 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:36.209419+0000 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:36.214 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 Processor -- start 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 -- start start 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: init 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient: _add_conns ranks=[0,6,1,7,4,9,10,2,3,8,5] 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f051c0a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 0x7f051c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-d con 0x7f051c09e160 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f051c0a5240 addr v1:172.21.15.160:6789/0 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c09e5e0 con 0x7f051c0a4750 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c09fc40 con 0x7f051c0a5240 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f051c09f680 con 0x7f051c09e160 2024-09-17T16:35:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0523eb9640 10 monclient(hunting): authenticate will time out at 1419.475293s 2024-09-17T16:35:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521c55640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f051c0a4750 0x7f051c0a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37474/0 (socket says 172.21.15.105:37474) 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521c55640 1 -- 172.21.15.105:0/1194922239 learned_addr learned my addr 172.21.15.105:0/1194922239 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 1 --2- 172.21.15.105:0/1194922239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 0x7f051c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): get_auth_request con 0x7f051c09e160 auth_method 0 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient(hunting): handle_auth_done global_id 24378 payload 274 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 1 -- 172.21.15.105:0/1194922239 >> v1:172.21.15.160:6789/0 conn(0x7f051c0a5240 legacy=0x7f051c13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 1 -- 172.21.15.105:0/1194922239 >> v1:172.21.15.105:6789/0 conn(0x7f051c0a4750 legacy=0x7f051c0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 1 monclient: found mon.noname-d 2024-09-17T16:35:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.206+0000 7f0521454640 1 -- 172.21.15.105:0/1194922239 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c0a0b40 con 0x7f051c09e160 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0521454640 1 --2- 172.21.15.105:0/1194922239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 0x7f051c13b740 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f05180097f0 tx=0x7f051801c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=606a946ae8c3116d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 1 -- 172.21.15.105:0/1194922239 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05180b0070 con 0x7f051c09e160 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: dump: 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 1 -- 172.21.15.105:0/1194922239 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05180aa810 con 0x7f051c09e160 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 5 monclient: authenticate success, global_id 24378 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 4 set_mon_vals no callback set 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 1 -- 172.21.15.105:0/1194922239 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f051c09ea00 con 0x7f051c09e160 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0520c53640 10 monclient: dump: 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.223 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 10 monclient: shutdown 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- 172.21.15.105:0/1194922239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 msgr2=0x7f051c13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1194922239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 0x7f051c13b740 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f05180097f0 tx=0x7f051801c3a0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- 172.21.15.105:0/1194922239 shutdown_connections 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1194922239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f051c09e160 0x7f051c13b740 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- 172.21.15.105:0/1194922239 >> 172.21.15.105:0/1194922239 conn(0x7f051c0a9540 msgr2=0x7f051c0a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- 172.21.15.105:0/1194922239 shutdown_connections 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- 172.21.15.105:0/1194922239 wait complete. 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 10 monclient: monmap: 2024-09-17T16:35:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 librados: starting msgr at 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 librados: starting objecter 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 Processor -- start 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 -- start start 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 librados: setting wanted keys 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 1 librados: calling monclient init 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.214+0000 7f0523eb9640 10 monclient: init 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _add_conns ranks=[1,8,0,5,6,7,3,4,2] 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f051c09e160 0x7f051c123120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-f con 0x7f051c09e160 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:35:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c0a4750 0x7f051c123660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-e con 0x7f051c0a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 0x7f051c123ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): picked mon.noname-a con 0x7f051c0a5240 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f051c06ccb0 con 0x7f051c0a5240 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f051c06cb30 con 0x7f051c09e160 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f051c06ce30 con 0x7f051c0a4750 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): authenticate will time out at 1419.484425s 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 0x7f051c123ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 0x7f051c123ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35944/0 (socket says 172.21.15.105:35944) 2024-09-17T16:35:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 learned_addr learned my addr 172.21.15.105:0/1903518461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): get_auth_request con 0x7f051c0a5240 auth_method 0 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient(hunting): handle_auth_done global_id 35541 payload 824 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c0a4750 msgr2=0x7f051c123660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c0a4750 0x7f051c123660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f051c09e160 msgr2=0x7f051c123120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f051c09e160 0x7f051c123120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 monclient: found mon.noname-a 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c126ea0 con 0x7f051c0a5240 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0522456640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 0x7f051c123ba0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f051400b970 tx=0x7f051400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 client.?.objecter ms_handle_connect 0x7f051c0a5240 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0514019070 con 0x7f051c0a5240 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: dump: 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 5 monclient: authenticate success, global_id 35541 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _renew_subs 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c0a86b0 con 0x7f051c0a5240 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051400c940 con 0x7f051c0a5240 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 client.35541.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _renew_subs 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c124b40 con 0x7f051c0a5240 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 librados: init done 2024-09-17T16:35:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0513fff640 4 set_mon_vals no callback set 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f051401c360 con 0x7f051c0a5240 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 monclient: dump: 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 0x7f051c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f051c051e80 con 0x7f051c1cfb20 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 0x7f051c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient: get_auth_request con 0x7f051c1cfb20 auth_method 0 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 10 monclient(hunting): handle_auth_done global_id 54224 payload 824 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f051c126790 con 0x7f051c1cfb20 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f0521c55640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 0x7f051c02d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f050c00c340 tx=0x7f050c00c800 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 client.35541.objecter ms_handle_connect 0x7f051c1cfb20 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 10 client.35541.objecter resend_mon_ops 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.218+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0514006d10 con 0x7f051c0a5240 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f050003e550 0x7f0500040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(335..335 src has 1..335) ==== 6189+0+0 (secure 0 0 0) 0x7f051404f700 con 0x7f051c0a5240 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 client.35541.objecter ms_dispatch 0x7f051c000ba0 osd_map(335..335 src has 1..335) 2024-09-17T16:35:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 3 client.35541.objecter handle_osd_map got epochs [335,335] > 0 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 3 client.35541.objecter handle_osd_map decoding full epoch 335 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 20 client.35541.objecter dump_active .. 0 homeless 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f050c017070 con 0x7f051c1cfb20 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 monclient: dump: 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f050c0070d0 con 0x7f051c1cfb20 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 msgr2=0x7f051c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 0x7f051c02d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f050c00c340 tx=0x7f050c00c800 comp rx=0 tx=0).stop 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 msgr2=0x7f051c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cfb20 0x7f051c02d450 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f050c00c340 tx=0x7f050c00c800 comp rx=0 tx=0).stop 2024-09-17T16:35:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f0521454640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f050003e550 0x7f0500040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f0521454640 10 monclient: get_auth_request con 0x7f050003e550 auth_method 0 2024-09-17T16:35:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f05127fc640 10 client.35541.objecter ms_handle_connect 0x7f050003e550 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.222+0000 7f0521454640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f050003e550 0x7f0500040a10 secure :-1 s=READY pgs=3511 cs=0 l=1 rev1=1 crypto rx=0x7f05180ad000 tx=0x7f0518004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.226+0000 7f0523eb9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.226+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 0x7f051c02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.226+0000 7f0523eb9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.226+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f051c06a200 con 0x7f051c1cff00 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 0x7f051c02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient: get_auth_request con 0x7f051c1cff00 auth_method 0 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 10 monclient(hunting): handle_auth_done global_id 54233 payload 824 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 1 -- 172.21.15.105:0/1903518461 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f051c0a4b30 con 0x7f051c1cff00 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 client.35541.objecter ms_handle_connect 0x7f051c1cff00 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 client.35541.objecter resend_mon_ops 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0521c55640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 0x7f051c02d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f050c00c310 tx=0x7f050c000ee0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f050c017070 con 0x7f051c1cff00 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 monclient: dump: 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.244 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7f050c022440 con 0x7f051c1cff00 2024-09-17T16:35:36.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 msgr2=0x7f051c02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 0x7f051c02d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f050c00c310 tx=0x7f050c000ee0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 msgr2=0x7f051c02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f05127fc640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f051c1cff00 0x7f051c02d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 43227, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:36.268 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 10 librados: watch_flush enter 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 10 librados: watch_flush exit 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 20 client.35541.objecter shutdown clearing up homeless session... 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 10 client.35541.objecter successfully canceled tick 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f050003e550 msgr2=0x7f0500040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f050003e550 0x7f0500040a10 secure :-1 s=READY pgs=3511 cs=0 l=1 rev1=1 crypto rx=0x7f05180ad000 tx=0x7f0518004740 comp rx=0 tx=0).stop 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 10 monclient: shutdown 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 msgr2=0x7f051c123ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 --2- 172.21.15.105:0/1903518461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f051c0a5240 0x7f051c123ba0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f051400b970 tx=0x7f051400be30 comp rx=0 tx=0).stop 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 reap_dead start 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0522456640 1 -- 172.21.15.105:0/1903518461 reap_dead start 2024-09-17T16:35:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 shutdown_connections 2024-09-17T16:35:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 >> 172.21.15.105:0/1903518461 conn(0x7f051c0a9540 msgr2=0x7f051c06ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 shutdown_connections 2024-09-17T16:35:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 -- 172.21.15.105:0/1903518461 wait complete. 2024-09-17T16:35:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.230+0000 7f0523eb9640 1 librados: shutdown 2024-09-17T16:35:36.271 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:36.271 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-17T16:35:36.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:35:36.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.370+0000 7fa114e47640 10 monclient: get_monmap_and_config 2024-09-17T16:35:36.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.370+0000 7fa114e47640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.370+0000 7fa114e47640 10 monclient: monmap: 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:36.375494+0000 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:36.375494+0000 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:36.379 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:36.380 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 Processor -- start 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 -- start start 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient: init 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient: _add_conns ranks=[9,4,2,8,7,1,10,0,6,5,3] 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-a con 0x7fa1100a2620 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa1100a2f40 0x7fa110137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-c con 0x7fa1100a2f40 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-b con 0x7fa110137940 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa11006e580 con 0x7fa1100a2620 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa11006f2e0 con 0x7fa110137940 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa1100a15e0 con 0x7fa1100a2f40 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient(hunting): authenticate will time out at 1419.641131s 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35952/0 (socket says 172.21.15.105:35952) 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:40026/0 (socket says 172.21.15.105:40026) 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 1 -- 172.21.15.105:0/1733039478 learned_addr learned my addr 172.21.15.105:0/1733039478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): get_auth_request con 0x7fa1100a2620 auth_method 0 2024-09-17T16:35:36.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): get_auth_request con 0x7fa110137940 auth_method 0 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa1100a2f40 0x7fa110137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): get_auth_request con 0x7fa1100a2f40 auth_method 0 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient(hunting): handle_auth_done global_id 44669 payload 274 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 -- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa1100a2f40 msgr2=0x7fa110137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa1100a2f40 0x7fa110137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 -- 172.21.15.105:0/1733039478 >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 msgr2=0x7fa1100a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/1733039478 >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 monclient: found mon.noname-b 2024-09-17T16:35:36.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 -- 172.21.15.105:0/1733039478 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1100a3c30 con 0x7fa110137940 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10e59c640 1 --2- 172.21.15.105:0/1733039478 >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa108009d90 tx=0x7fa10801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65cbc5e4b35e921 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 1 -- 172.21.15.105:0/1733039478 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1080b2070 con 0x7fa110137940 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: dump: 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.395 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 1 -- 172.21.15.105:0/1733039478 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080a5a00 con 0x7fa110137940 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 1 -- 172.21.15.105:0/1733039478 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1080a5c30 con 0x7fa110137940 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 5 monclient: authenticate success, global_id 44669 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa114e47640 4 set_mon_vals no callback set 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.374+0000 7fa10d59a640 10 monclient: dump: 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: shutdown 2024-09-17T16:35:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 msgr2=0x7fa110137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa108009d90 tx=0x7fa10801c830 comp rx=0 tx=0).stop 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/1733039478 shutdown_connections 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa110137940 0x7fa110137df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- 172.21.15.105:0/1733039478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa1100a2f40 0x7fa110137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- 172.21.15.105:0/1733039478 >> v2:172.21.15.105:3300/0 conn(0x7fa1100a2620 0x7fa1100a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/1733039478 >> 172.21.15.105:0/1733039478 conn(0x7fa110068ba0 msgr2=0x7fa110068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/1733039478 shutdown_connections 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/1733039478 wait complete. 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: monmap: 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:36.399 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.400 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 librados: starting msgr at 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 librados: starting objecter 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 Processor -- start 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- start start 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 librados: setting wanted keys 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 librados: calling monclient init 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: init 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _add_conns ranks=[8,2,7,3,4,1,5,0,6] 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 0x7fa11013b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-e con 0x7fa1100a2620 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 0x7fa11013b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-b con 0x7fa1100a2f40 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa110137940 0x7fa11013be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): picked mon.noname-i con 0x7fa110137940 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa1100a1f50 con 0x7fa1100a2f40 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa1100a1dd0 con 0x7fa110137940 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa110141500 con 0x7fa1100a2620 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): authenticate will time out at 1419.645979s 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 0x7fa11013b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 0x7fa11013b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 0x7fa11013b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:39310/0 (socket says 172.21.15.105:39310) 2024-09-17T16:35:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 1 -- 172.21.15.105:0/906208269 learned_addr learned my addr 172.21.15.105:0/906208269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 0x7fa11013b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:40042/0 (socket says 172.21.15.105:40042) 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): get_auth_request con 0x7fa1100a2620 auth_method 0 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): get_auth_request con 0x7fa1100a2f40 auth_method 0 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10ed9d640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa110137940 msgr2=0x7fa11013be80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa110137940 0x7fa11013be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient(hunting): handle_auth_done global_id 44678 payload 824 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 msgr2=0x7fa11013b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 0x7fa11013b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa110137940 msgr2=0x7fa11013be80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa110137940 0x7fa11013be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 monclient: found mon.noname-b 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11011f470 con 0x7fa1100a2f40 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10e59c640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1100a2620 0x7fa11013b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10dd9b640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 0x7fa11013b940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e870 tx=0x7fa10400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 client.?.objecter ms_handle_connect 0x7fa1100a2f40 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa104017070 con 0x7fa1100a2f40 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: dump: 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.406 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10400cdb0 con 0x7fa1100a2f40 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1040138e0 con 0x7fa1100a2f40 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 5 monclient: authenticate success, global_id 44678 2024-09-17T16:35:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _renew_subs 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11011f750 con 0x7fa1100a2f40 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa10cd99640 4 set_mon_vals no callback set 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 10 monclient: dump: 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 client.44678.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _renew_subs 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11011f190 con 0x7fa1100a2f40 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 librados: init done 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 0x7fa110070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa11013d6d0 con 0x7fa11013c3c0 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa10401a070 con 0x7fa1100a2f40 2024-09-17T16:35:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa0f003e550 0x7fa0f0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.378+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(335..335 src has 1..335) ==== 6189+0+0 (secure 0 0 0) 0x7fa10400f040 con 0x7fa1100a2f40 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 client.44678.objecter ms_dispatch 0x7fa110000ba0 osd_map(335..335 src has 1..335) 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 0x7fa110070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 3 client.44678.objecter handle_osd_map got epochs [335,335] > 0 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 3 client.44678.objecter handle_osd_map decoding full epoch 335 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient: get_auth_request con 0x7fa11013c3c0 auth_method 0 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 20 client.44678.objecter dump_active .. 0 homeless 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa0f003e550 0x7fa0f0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10ed9d640 10 monclient: get_auth_request con 0x7fa0f003e550 auth_method 0 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 10 monclient(hunting): handle_auth_done global_id 54280 payload 824 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa11013db60 con 0x7fa11013c3c0 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 client.44678.objecter ms_handle_connect 0x7fa11013c3c0 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10e59c640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 0x7fa110070ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa100011db0 tx=0x7fa10000f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 client.44678.objecter resend_mon_ops 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 client.44678.objecter ms_handle_connect 0x7fa0f003e550 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa10ed9d640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa0f003e550 0x7fa0f0040a10 secure :-1 s=READY pgs=3512 cs=0 l=1 rev1=1 crypto rx=0x7fa1080af000 tx=0x7fa1080a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa10000fe50 con 0x7fa11013c3c0 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 monclient: dump: 2024-09-17T16:35:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa10000fe50 con 0x7fa11013c3c0 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 msgr2=0x7fa110070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 0x7fa110070ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa100011db0 tx=0x7fa10000f610 comp rx=0 tx=0).stop 2024-09-17T16:35:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 msgr2=0x7fa110070ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.382+0000 7fa0feffd640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa11013c3c0 0x7fa110070ff0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa100011db0 tx=0x7fa10000f610 comp rx=0 tx=0).stop 2024-09-17T16:35:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa114e47640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa114e47640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 0x7fa110071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa114e47640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa110051e80 con 0x7fa110071530 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa10e59c640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 0x7fa110071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.386+0000 7fa10e59c640 10 monclient: get_auth_request con 0x7fa110071530 auth_method 0 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 10 monclient(hunting): handle_auth_done global_id 54289 payload 824 2024-09-17T16:35:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 1 -- 172.21.15.105:0/906208269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa1100a7040 con 0x7fa110071530 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 client.44678.objecter ms_handle_connect 0x7fa110071530 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 client.44678.objecter resend_mon_ops 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10e59c640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 0x7fa110071940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa10001c000 tx=0x7fa1000046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa100022aa0 con 0x7fa110071530 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 monclient: dump: 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7fa10002b480 con 0x7fa110071530 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 msgr2=0x7fa110071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 0x7fa110071940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa10001c000 tx=0x7fa1000046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 msgr2=0x7fa110071940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa0feffd640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa110071530 0x7fa110071940 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa10001c000 tx=0x7fa1000046d0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:36.421 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 43395, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:36.422 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.423 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:36.425 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.426 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.428 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:36.429 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.430 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.433 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.435 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.436 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.437 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.441 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.442 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.443 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.444 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:36.445 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 10 librados: watch_flush enter 2024-09-17T16:35:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 10 librados: watch_flush exit 2024-09-17T16:35:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 20 client.44678.objecter shutdown clearing up homeless session... 2024-09-17T16:35:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 10 client.44678.objecter successfully canceled tick 2024-09-17T16:35:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa0f003e550 msgr2=0x7fa0f0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa0f003e550 0x7fa0f0040a10 secure :-1 s=READY pgs=3512 cs=0 l=1 rev1=1 crypto rx=0x7fa1080af000 tx=0x7fa1080a7070 comp rx=0 tx=0).stop 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 10 monclient: shutdown 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 msgr2=0x7fa11013b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 --2- 172.21.15.105:0/906208269 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1100a2f40 0x7fa11013b940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e870 tx=0x7fa10400ed30 comp rx=0 tx=0).stop 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10ed9d640 1 -- 172.21.15.105:0/906208269 reap_dead start 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa10ed9d640 1 -- 172.21.15.105:0/906208269 reap_dead start 2024-09-17T16:35:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 shutdown_connections 2024-09-17T16:35:36.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 >> 172.21.15.105:0/906208269 conn(0x7fa110068ba0 msgr2=0x7fa1100a5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 shutdown_connections 2024-09-17T16:35:36.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 -- 172.21.15.105:0/906208269 wait complete. 2024-09-17T16:35:36.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.390+0000 7fa114e47640 1 librados: shutdown 2024-09-17T16:35:36.449 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:36.449 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-17T16:35:36.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:35:36.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: get_monmap_and_config 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: monmap: 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:36.559911+0000 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:36.559911+0000 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:36.564 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 Processor -- start 2024-09-17T16:35:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 -- start start 2024-09-17T16:35:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: init 2024-09-17T16:35:36.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient: _add_conns ranks=[7,6,9,0,8,1,5,3,4,10,2] 2024-09-17T16:35:36.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 0x7f2d640a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-i con 0x7f2d640a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-d con 0x7f2d640a5070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d64139a70 0x7f2d64139f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-a con 0x7f2d64139a70 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2d6409def0 con 0x7f2d64139a70 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2d6406c390 con 0x7f2d640a5070 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2d6409e4b0 con 0x7f2d640a4750 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d69a19640 10 monclient(hunting): authenticate will time out at 1419.825719s 2024-09-17T16:35:36.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:56504/0 (socket says 172.21.15.105:56504) 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 -- 172.21.15.105:0/2672460189 learned_addr learned my addr 172.21.15.105:0/2672460189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): get_auth_request con 0x7f2d640a5070 auth_method 0 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d62ffd640 1 -- 172.21.15.105:0/2672460189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 msgr2=0x7f2d640a4b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d62ffd640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 0x7f2d640a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 1 --2- 172.21.15.105:0/2672460189 >> v2:172.21.15.105:3300/0 conn(0x7f2d64139a70 0x7f2d64139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): get_auth_request con 0x7f2d64139a70 auth_method 0 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient(hunting): handle_auth_done global_id 24387 payload 274 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 -- 172.21.15.105:0/2672460189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 msgr2=0x7f2d640a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 0x7f2d640a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 -- 172.21.15.105:0/2672460189 >> v2:172.21.15.105:3300/0 conn(0x7f2d64139a70 msgr2=0x7f2d64139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 --2- 172.21.15.105:0/2672460189 >> v2:172.21.15.105:3300/0 conn(0x7f2d64139a70 0x7f2d64139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 monclient: found mon.noname-d 2024-09-17T16:35:36.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.558+0000 7f2d627fc640 1 -- 172.21.15.105:0/2672460189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6409dc10 con 0x7f2d640a5070 2024-09-17T16:35:36.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098a0 tx=0x7f2d4c01c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=42ab36e9e298a6fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 1 -- 172.21.15.105:0/2672460189 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d4c0b0070 con 0x7f2d640a5070 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: dump: 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.578 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 5 monclient: authenticate success, global_id 24387 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 1 -- 172.21.15.105:0/2672460189 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c0aa810 con 0x7f2d640a5070 2024-09-17T16:35:36.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 4 set_mon_vals no callback set 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 1 -- 172.21.15.105:0/2672460189 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d4c0a9300 con 0x7f2d640a5070 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d61ffb640 10 monclient: dump: 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.580 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: shutdown 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/2672460189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 msgr2=0x7f2d64139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098a0 tx=0x7f2d4c01c420 comp rx=0 tx=0).stop 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/2672460189 shutdown_connections 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- 172.21.15.105:0/2672460189 >> v2:172.21.15.105:3300/0 conn(0x7f2d64139a70 0x7f2d64139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2d640a5070 0x7f2d64139530 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- 172.21.15.105:0/2672460189 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2d640a4750 0x7f2d640a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/2672460189 >> 172.21.15.105:0/2672460189 conn(0x7f2d640a93a0 msgr2=0x7f2d640a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/2672460189 shutdown_connections 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/2672460189 wait complete. 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: monmap: 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:36.582 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 librados: starting msgr at 2024-09-17T16:35:36.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 librados: starting objecter 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 Processor -- start 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- start start 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 librados: setting wanted keys 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 librados: calling monclient init 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: init 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _add_conns ranks=[8,2,0,1,6,7,5,3,4] 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 0x7f2d641d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-e con 0x7f2d640a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d640a5070 0x7f2d641d1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-b con 0x7f2d640a5070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d64139a70 0x7f2d641d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): picked mon.noname-a con 0x7f2d64139a70 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:36.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d640747c0 con 0x7f2d64139a70 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6409ee90 con 0x7f2d640a5070 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f2d640a2e40 con 0x7f2d640a4750 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient(hunting): authenticate will time out at 1419.830771s 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 0x7f2d641d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d64139a70 0x7f2d641d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d640a5070 0x7f2d641d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 0x7f2d641d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:39318/0 (socket says 172.21.15.105:39318) 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 -- 172.21.15.105:0/3927027254 learned_addr learned my addr 172.21.15.105:0/3927027254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): get_auth_request con 0x7f2d640a4750 auth_method 0 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): get_auth_request con 0x7f2d640a5070 auth_method 0 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): get_auth_request con 0x7f2d64139a70 auth_method 0 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient(hunting): handle_auth_done global_id 54251 payload 824 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d640a5070 msgr2=0x7f2d641d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d640a5070 0x7f2d641d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d64139a70 msgr2=0x7f2d641d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d64139a70 0x7f2d641d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 monclient: found mon.noname-e 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d64123a60 con 0x7f2d640a4750 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d627fc640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d640a5070 0x7f2d641d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d637fe640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d64139a70 0x7f2d641d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d62ffd640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 0x7f2d641d13d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b9e0 tx=0x7f2d5c00bea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 client.?.objecter ms_handle_connect 0x7f2d640a4750 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d5c00ce10 con 0x7f2d640a4750 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: dump: 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.588 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.589 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5c010070 con 0x7f2d640a4750 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d5c0158e0 con 0x7f2d640a4750 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 5 monclient: authenticate success, global_id 54251 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _renew_subs 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d640a4c50 con 0x7f2d640a4750 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d617fa640 4 set_mon_vals no callback set 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:35:36.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d5bfff640 10 monclient: dump: 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.591 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 client.54251.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _renew_subs 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641234a0 con 0x7f2d640a4750 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.562+0000 7f2d69a19640 1 librados: init done 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d69a19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d69a19640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 0x7f2d641201b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2d64051e80 con 0x7f2d6411fdd0 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2d5c015470 con 0x7f2d640a4750 2024-09-17T16:35:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 0x7f2d641201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2d4403e550 0x7f2d44040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient: get_auth_request con 0x7f2d6411fdd0 auth_method 0 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(335..335 src has 1..335) ==== 6189+0+0 (secure 0 0 0) 0x7f2d5c04ff50 con 0x7f2d640a4750 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 client.54251.objecter ms_dispatch 0x7f2d64000ba0 osd_map(335..335 src has 1..335) 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 3 client.54251.objecter handle_osd_map got epochs [335,335] > 0 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 3 client.54251.objecter handle_osd_map decoding full epoch 335 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 20 client.54251.objecter dump_active .. 0 homeless 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d637fe640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2d4403e550 0x7f2d44040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d637fe640 10 monclient: get_auth_request con 0x7f2d4403e550 auth_method 0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 10 monclient(hunting): handle_auth_done global_id 54210 payload 824 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d64074940 con 0x7f2d6411fdd0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 client.54251.objecter ms_handle_connect 0x7f2d6411fdd0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 client.54251.objecter resend_mon_ops 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d627fc640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 0x7f2d641201b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d640a5ff0 tx=0x7f2d4c0191c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 client.54251.objecter ms_handle_connect 0x7f2d4403e550 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d637fe640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2d4403e550 0x7f2d44040a10 secure :-1 s=READY pgs=3513 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009eb0 tx=0x7f2d54009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d4c0ba070 con 0x7f2d6411fdd0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 monclient: dump: 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.595 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2d4c0aa810 con 0x7f2d6411fdd0 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 msgr2=0x7f2d641201b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 0x7f2d641201b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d640a5ff0 tx=0x7f2d4c0191c0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 msgr2=0x7f2d641201b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.566+0000 7f2d5bfff640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d6411fdd0 0x7f2d641201b0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d640a5ff0 tx=0x7f2d4c0191c0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d69a19640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d69a19640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 0x7f2d6413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d69a19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2d641394e0 con 0x7f2d641206f0 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 0x7f2d6413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient: get_auth_request con 0x7f2d641206f0 auth_method 0 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 10 monclient(hunting): handle_auth_done global_id 54219 payload 824 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 1 -- 172.21.15.105:0/3927027254 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d640008e0 con 0x7f2d641206f0 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d627fc640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 0x7f2d6413c7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01c250 tx=0x7f2d4c0a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 client.54251.objecter ms_handle_connect 0x7f2d641206f0 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 client.54251.objecter resend_mon_ops 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d4c0b0040 con 0x7f2d641206f0 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 monclient: dump: 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7f2d4c0038f0 con 0x7f2d641206f0 2024-09-17T16:35:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 msgr2=0x7f2d6413c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 0x7f2d6413c7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01c250 tx=0x7f2d4c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 msgr2=0x7f2d6413c7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d5bfff640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d641206f0 0x7f2d6413c7d0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01c250 tx=0x7f2d4c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:35:36.601 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 43583, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:35:36.602 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.603 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.604 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:36.605 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.606 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.608 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:36.609 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.611 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.613 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.614 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.616 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:36.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.621 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:36.622 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:36.623 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.574+0000 7f2d69a19640 10 librados: watch_flush enter 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 10 librados: watch_flush exit 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 20 client.54251.objecter shutdown clearing up homeless session... 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 10 client.54251.objecter successfully canceled tick 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2d4403e550 msgr2=0x7f2d44040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2d4403e550 0x7f2d44040a10 secure :-1 s=READY pgs=3513 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009eb0 tx=0x7f2d54009190 comp rx=0 tx=0).stop 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 10 monclient: shutdown 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 msgr2=0x7f2d641d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 --2- 172.21.15.105:0/3927027254 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f2d640a4750 0x7f2d641d13d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b9e0 tx=0x7f2d5c00bea0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d637fe640 1 -- 172.21.15.105:0/3927027254 reap_dead start 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d637fe640 1 -- 172.21.15.105:0/3927027254 reap_dead start 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 shutdown_connections 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 >> 172.21.15.105:0/3927027254 conn(0x7f2d640a93a0 msgr2=0x7f2d640a7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 shutdown_connections 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 -- 172.21.15.105:0/3927027254 wait complete. 2024-09-17T16:35:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.578+0000 7f2d69a19640 1 librados: shutdown 2024-09-17T16:35:36.626 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:36.626 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-17T16:35:36.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:35:36.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: get_monmap_and_config 2024-09-17T16:35:36.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: monmap: 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:35:36.736779+0000 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:35:36.736779+0000 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.740 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:35:36.741 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 Processor -- start 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 -- start start 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: init 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient: _add_conns ranks=[0,3,9,8,5,2,6,10,7,4,1] 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd2980a2de0 addr v1:172.21.15.105:6789/0 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2980a38d0 0x7fd29813c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-g con 0x7fd2980a38d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd29813ced0 0x7fd29813f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-a con 0x7fd29813ced0 addr v2:172.21.15.105:3300/0 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2980a20b0 con 0x7fd2980a2de0 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd29809c9a0 con 0x7fd29813ced0 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd2980a2770 con 0x7fd2980a38d0 2024-09-17T16:35:36.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29ca80640 10 monclient(hunting): authenticate will time out at 1420.003207s 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd2980a2de0 0x7fd2980a31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37476/0 (socket says 172.21.15.105:37476) 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29659c640 1 -- 172.21.15.105:0/1190486586 learned_addr learned my addr 172.21.15.105:0/1190486586 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1190486586 >> v2:172.21.15.105:3300/0 conn(0x7fd29813ced0 0x7fd29813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): get_auth_request con 0x7fd29813ced0 auth_method 0 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd295d9b640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2980a38d0 0x7fd29813c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd295d9b640 10 monclient(hunting): get_auth_request con 0x7fd2980a38d0 auth_method 0 2024-09-17T16:35:36.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd295d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd295d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd295d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient(hunting): handle_auth_done global_id 35577 payload 274 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 -- 172.21.15.105:0/1190486586 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2980a38d0 msgr2=0x7fd29813c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2980a38d0 0x7fd29813c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 -- 172.21.15.105:0/1190486586 >> v1:172.21.15.105:6789/0 conn(0x7fd2980a2de0 legacy=0x7fd2980a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 monclient: found mon.noname-a 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29559a640 1 -- 172.21.15.105:0/1190486586 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2040856232 0 0) 0x7fd2980a20b0 con 0x7fd2980a2de0 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd296d9d640 1 -- 172.21.15.105:0/1190486586 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2980a24f0 con 0x7fd29813ced0 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.734+0000 7fd29559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:35:36.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29813ced0 0x7fd29813f2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2900097f0 tx=0x7fd29001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bc8b0dcfa8c25b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 1 -- 172.21.15.105:0/1190486586 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2900b0070 con 0x7fd29813ced0 2024-09-17T16:35:36.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:35:36.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: dump: 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.752 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 5 monclient: authenticate success, global_id 35577 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 1 -- 172.21.15.105:0/1190486586 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900aa810 con 0x7fd29813ced0 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 10 monclient: get_monmap_and_config success 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 1 -- 172.21.15.105:0/1190486586 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2900a9300 con 0x7fd29813ced0 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 4 set_mon_vals no callback set 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:35:36.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29559a640 10 monclient: dump: 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.755 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 10 monclient: shutdown 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 -- 172.21.15.105:0/1190486586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29813ced0 msgr2=0x7fd29813f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29813ced0 0x7fd29813f2c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd2900097f0 tx=0x7fd29001c360 comp rx=0 tx=0).stop 2024-09-17T16:35:36.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 -- 172.21.15.105:0/1190486586 shutdown_connections 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29813ced0 0x7fd29813f2c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1190486586 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2980a38d0 0x7fd29813c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 -- 172.21.15.105:0/1190486586 >> 172.21.15.105:0/1190486586 conn(0x7fd2980a83e0 msgr2=0x7fd2980a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 -- 172.21.15.105:0/1190486586 shutdown_connections 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 1 -- 172.21.15.105:0/1190486586 wait complete. 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 10 monclient: build_initial_monmap 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.738+0000 7fd29ca80640 10 monclient: monmap: 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:35:36.757 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:35:36.758 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:35:36.759 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:35:36.759 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:35:36.759 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:35:36.759 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:35:36.759 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 librados: starting msgr at 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 librados: starting objecter 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 Processor -- start 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- start start 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 librados: setting wanted keys 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 librados: calling monclient init 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: init 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _reopen_session rank -1 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _add_conns ranks=[7,0,6,4,5,1,8,3,2] 2024-09-17T16:35:36.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd2980a2de0 0x7fd2981cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-i con 0x7fd2980a2de0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 0x7fd2981cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-a con 0x7fd2980a38d0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd29813ced0 0x7fd298070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): picked mon.noname-d con 0x7fd29813ced0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd29809fa60 con 0x7fd2980a38d0 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd29809f8e0 con 0x7fd29813ced0 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fd29809fbe0 con 0x7fd2980a2de0 2024-09-17T16:35:36.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): _renew_subs 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): authenticate will time out at 1420.008944s 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 1 -- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd2980a2de0 msgr2=0x7fd2981cd090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3303/0 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 0x7fd2981cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd2980a2de0 0x7fd2981cd090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd29813ced0 0x7fd298070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 0x7fd2981cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35996/0 (socket says 172.21.15.105:35996) 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 1 -- 172.21.15.105:0/1873001928 learned_addr learned my addr 172.21.15.105:0/1873001928 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): get_auth_request con 0x7fd2980a38d0 auth_method 0 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): get_auth_request con 0x7fd29813ced0 auth_method 0 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient(hunting): handle_auth_done global_id 24396 payload 824 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd2980a2de0 msgr2=0x7fd2981cd090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd2980a2de0 0x7fd2981cd090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 msgr2=0x7fd2981cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 0x7fd2981cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 monclient: found mon.noname-d 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:35:36.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd298072610 con 0x7fd29813ced0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2980a38d0 0x7fd2981cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd296d9d640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd29813ced0 0x7fd298070cb0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd2980a2280 tx=0x7fd290019100 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 client.?.objecter ms_handle_connect 0x7fd29813ced0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2900b0040 con 0x7fd29813ced0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: dump: 2024-09-17T16:35:36.772 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.773 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: _finish_auth 0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: _check_auth_tickets 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900044b0 con 0x7fd29813ced0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 5 monclient: authenticate success, global_id 24396 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _renew_subs 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29813cc90 con 0x7fd29813ced0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2900aac20 con 0x7fd29813ced0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:35:36.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 client.24396.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd294d99640 4 set_mon_vals no callback set 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 monclient: dump: 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.775 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _renew_subs 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd298073170 con 0x7fd29813ced0 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 librados: init done 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 0x7fd2980711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd298051e80 con 0x7fd298074d90 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd290004040 con 0x7fd29813ced0 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 0x7fd2980711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd27803e540 0x7fd278040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient: get_auth_request con 0x7fd298074d90 auth_method 0 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(335..335 src has 1..335) ==== 6189+0+0 (secure 0 0 0) 0x7fd2900e9890 con 0x7fd29813ced0 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 client.24396.objecter ms_dispatch 0x7fd298000ba0 osd_map(335..335 src has 1..335) 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 3 client.24396.objecter handle_osd_map got epochs [335,335] > 0 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 3 client.24396.objecter handle_osd_map decoding full epoch 335 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 20 client.24396.objecter dump_active .. 0 homeless 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd27803e540 0x7fd278040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd295d9b640 10 monclient: get_auth_request con 0x7fd27803e540 auth_method 0 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 10 monclient(hunting): handle_auth_done global_id 20003 payload 824 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2980a31c0 con 0x7fd298074d90 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 client.24396.objecter ms_handle_connect 0x7fd298074d90 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd28effd640 10 client.24396.objecter resend_mon_ops 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.742+0000 7fd29659c640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 0x7fd2980711f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd288004c00 tx=0x7fd28800a430 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 client.24396.objecter ms_handle_connect 0x7fd27803e540 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd295d9b640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd27803e540 0x7fd278040a00 secure :-1 s=READY pgs=3514 cs=0 l=1 rev1=1 crypto rx=0x7fd298072410 tx=0x7fd280008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd288017070 con 0x7fd298074d90 2024-09-17T16:35:36.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 monclient: dump: 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd28800ab80 con 0x7fd298074d90 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:35:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 msgr2=0x7fd2980711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 0x7fd2980711f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd288004c00 tx=0x7fd28800a430 comp rx=0 tx=0).stop 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 msgr2=0x7fd2980711f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.746+0000 7fd28effd640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298074d90 0x7fd2980711f0 secure :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd288004c00 tx=0x7fd28800a430 comp rx=0 tx=0).stop 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29ca80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 0x7fd298071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29ca80640 10 monclient(hunting): start opening mon connection 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd29806a1c0 con 0x7fd298071730 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 0x7fd298071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient: get_auth_request con 0x7fd298071730 auth_method 0 2024-09-17T16:35:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.750+0000 7fd29659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29659c640 10 monclient(hunting): handle_auth_done global_id 20012 payload 824 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29659c640 1 -- 172.21.15.105:0/1873001928 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd298072080 con 0x7fd298071730 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29659c640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 0x7fd298071b40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd288002240 tx=0x7fd288000f50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:35:36.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 client.24396.objecter ms_handle_connect 0x7fd298071730 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 client.24396.objecter resend_mon_ops 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd288017070 con 0x7fd298071730 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 monclient: dump: 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:35:36.783 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:35:36.784 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9278 (secure 0 0 0) 0x7fd288018040 con 0x7fd298071730 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 msgr2=0x7fd298071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 0x7fd298071b40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd288002240 tx=0x7fd288000f50 comp rx=0 tx=0).stop 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 msgr2=0x7fd298071b40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd28effd640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd298071730 0x7fd298071b40 secure :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd288002240 tx=0x7fd288000f50 comp rx=0 tx=0).stop 2024-09-17T16:35:36.785 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 58, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 484357, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:35:36.786 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:35:36.787 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.788 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.789 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:35:36.790 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:35:36.791 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.792 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.794 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.796 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:35:36.797 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:35:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:35:36.802 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.805 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:35:36.807 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.808 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.809 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:35:36.810 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:35:36.811 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:35:36.811 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:35:36.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:35:36.811 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:35:36.811 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:35:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 10 librados: watch_flush enter 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 10 librados: watch_flush exit 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 20 client.24396.objecter shutdown clearing up homeless session... 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 10 client.24396.objecter successfully canceled tick 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd27803e540 msgr2=0x7fd278040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd27803e540 0x7fd278040a00 secure :-1 s=READY pgs=3514 cs=0 l=1 rev1=1 crypto rx=0x7fd298072410 tx=0x7fd280008040 comp rx=0 tx=0).stop 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 10 monclient: shutdown 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd29813ced0 msgr2=0x7fd298070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 --2- 172.21.15.105:0/1873001928 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd29813ced0 0x7fd298070cb0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd2980a2280 tx=0x7fd290019100 comp rx=0 tx=0).stop 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd296d9d640 1 -- 172.21.15.105:0/1873001928 reap_dead start 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd296d9d640 1 -- 172.21.15.105:0/1873001928 reap_dead start 2024-09-17T16:35:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 shutdown_connections 2024-09-17T16:35:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 >> 172.21.15.105:0/1873001928 conn(0x7fd2980a83e0 msgr2=0x7fd29809ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:35:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 shutdown_connections 2024-09-17T16:35:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 -- 172.21.15.105:0/1873001928 wait complete. 2024-09-17T16:35:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:35:36.754+0000 7fd29ca80640 1 librados: shutdown 2024-09-17T16:35:36.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:35:36.814 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:37:06.816 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-17T16:37:06.816 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-17T16:37:06.816 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-17T16:37:06.820 INFO:tasks.ceph.mon.i:Started 2024-09-17T16:37:06.820 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-17T16:37:06.820 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-17T16:37:06.820 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:37:06.928 INFO:tasks.ceph.mon.i.smithi160.stderr:2024-09-17T16:37:06.923+0000 7f155ab64e00 -1 mon.i@-1(???).log v0 log_external_backlog rewinding external_log_to from 327 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:37:21.821 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-17T16:37:21.822 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-17T16:37:21.822 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:37:21.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:37:21.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: get_monmap_and_config 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: build_initial_monmap 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: monmap: 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:21.948762+0000 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:21.948762+0000 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:21.952 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:21.953 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 Processor -- start 2024-09-17T16:37:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 -- start start 2024-09-17T16:37:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: init 2024-09-17T16:37:21.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient: _add_conns ranks=[8,3,7,9,4,5,1,2,6,10,0] 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): picked mon.noname-e con 0x7f6850137830 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68500a56c0 0x7f68500a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): picked mon.noname-g con 0x7f68500a56c0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a6020 0x7f685013e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): picked mon.noname-i con 0x7f68500a6020 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f68500a1d60 con 0x7f68500a56c0 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f68500a33c0 con 0x7f68500a6020 2024-09-17T16:37:21.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f68500a1660 con 0x7f6850137830 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6858218640 10 monclient(hunting): authenticate will time out at 1525.214483s 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43390/0 (socket says 172.21.15.105:43390) 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 -- 172.21.15.105:0/692919935 learned_addr learned my addr 172.21.15.105:0/692919935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): get_auth_request con 0x7f6850137830 auth_method 0 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68557b3640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68500a56c0 0x7f68500a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68567b5640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a6020 0x7f685013e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68557b3640 10 monclient(hunting): get_auth_request con 0x7f68500a56c0 auth_method 0 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68557b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68557b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68557b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68567b5640 10 monclient(hunting): get_auth_request con 0x7f68500a6020 auth_method 0 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68567b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68567b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f68567b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient(hunting): handle_auth_done global_id 54260 payload 274 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 -- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a6020 msgr2=0x7f685013e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a6020 0x7f685013e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 -- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68500a56c0 msgr2=0x7f68500a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68500a56c0 0x7f68500a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient: _finish_hunting 0 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 monclient: found mon.noname-e 2024-09-17T16:37:21.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.945+0000 7f6855fb4640 1 -- 172.21.15.105:0/692919935 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500a1ae0 con 0x7f6850137830 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6855fb4640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f684c0098d0 tx=0x7f684c01c560 comp rx=0 tx=0).ready entity=mon.8 client_cookie=60108f6b3e2ee0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 1 -- 172.21.15.105:0/692919935 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f684c0b0070 con 0x7f6850137830 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: dump: 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:21.962 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:21.963 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: _finish_auth 0 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: _check_auth_tickets 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 1 -- 172.21.15.105:0/692919935 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684c0aa810 con 0x7f6850137830 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 5 monclient: authenticate success, global_id 54260 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 1 -- 172.21.15.105:0/692919935 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f684c0a9300 con 0x7f6850137830 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:21.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 4 set_mon_vals no callback set 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6854fb2640 10 monclient: dump: 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:21.965 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 10 monclient: shutdown 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- 172.21.15.105:0/692919935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 msgr2=0x7f6850137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f684c0098d0 tx=0x7f684c01c560 comp rx=0 tx=0).stop 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- 172.21.15.105:0/692919935 shutdown_connections 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a6020 0x7f685013e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f68500a56c0 0x7f68500a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 --2- 172.21.15.105:0/692919935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6850137830 0x7f6850137c10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- 172.21.15.105:0/692919935 >> 172.21.15.105:0/692919935 conn(0x7f68500aa390 msgr2=0x7f68500a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:21.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- 172.21.15.105:0/692919935 shutdown_connections 2024-09-17T16:37:21.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- 172.21.15.105:0/692919935 wait complete. 2024-09-17T16:37:21.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 10 monclient: build_initial_monmap 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 10 monclient: monmap: 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:21.972 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 librados: starting msgr at 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 librados: starting objecter 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 Processor -- start 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 -- start start 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 librados: setting wanted keys 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 1 librados: calling monclient init 2024-09-17T16:37:21.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.949+0000 7f6858218640 10 monclient: init 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _add_conns ranks=[7,0,4,2,8,3,5,1,6] 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a56c0 0x7f685006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): picked mon.noname-i con 0x7f68500a56c0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): picked mon.noname-a con 0x7f68500a6020 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): picked mon.noname-c con 0x7f6850137830 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6850071310 con 0x7f68500a6020 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6850071190 con 0x7f6850137830 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:21.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6850138150 con 0x7f68500a56c0 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient(hunting): authenticate will time out at 1525.220427s 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46204/0 (socket says 172.21.15.105:46204) 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51796/0 (socket says 172.21.15.105:51796) 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 1 -- 172.21.15.105:0/1018380807 learned_addr learned my addr 172.21.15.105:0/1018380807 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): get_auth_request con 0x7f68500a6020 auth_method 0 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): get_auth_request con 0x7f6850137830 auth_method 0 2024-09-17T16:37:21.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a56c0 0x7f685006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:21.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): get_auth_request con 0x7f68500a56c0 auth_method 0 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient(hunting): handle_auth_done global_id 54292 payload 824 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 -- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a56c0 msgr2=0x7f685006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a56c0 0x7f685006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 -- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 msgr2=0x7f685006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:21.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient: _finish_hunting 0 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 monclient: found mon.noname-c 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 -- 172.21.15.105:0/1018380807 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685006f0c0 con 0x7f6850137830 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68557b3640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68567b5640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6848007d60 tx=0x7f684800e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 client.?.objecter ms_handle_connect 0x7f6850137830 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f684800edd0 con 0x7f6850137830 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:21.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: dump: 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:21.979 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: _finish_auth 0 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: _check_auth_tickets 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6848014890 con 0x7f6850137830 2024-09-17T16:37:21.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6848013380 con 0x7f6850137830 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 monclient: dump: 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:21.981 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6843fff640 4 set_mon_vals no callback set 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 5 monclient: authenticate success, global_id 54292 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _renew_subs 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685013d810 con 0x7f6850137830 2024-09-17T16:37:21.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 client.54292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _renew_subs 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6850137c10 con 0x7f6850137830 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 librados: init done 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f684801b030 con 0x7f6850137830 2024-09-17T16:37:21.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68500a5ae0 con 0x7f6850137830 2024-09-17T16:37:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 0x7f6834040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:21.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(376..376 src has 1..376) ==== 6202+0+0 (secure 0 0 0) 0x7f6848019020 con 0x7f6850137830 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 client.54292.objecter ms_dispatch 0x7f6850000ba0 osd_map(376..376 src has 1..376) 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 3 client.54292.objecter handle_osd_map got epochs [376,376] > 0 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 3 client.54292.objecter handle_osd_map decoding full epoch 376 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 20 client.54292.objecter dump_active .. 0 homeless 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 0x7f6834040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:21.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 10 monclient: get_auth_request con 0x7f683403e550 auth_method 0 2024-09-17T16:37:21.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f6855fb4640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 0x7f6834040a10 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ad000 tx=0x7f684c0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:21.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.953+0000 7f68427fc640 10 client.54292.objecter ms_handle_connect 0x7f683403e550 2024-09-17T16:37:21.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.957+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6848022060 con 0x7f6850137830 2024-09-17T16:37:21.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.957+0000 7f68427fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:21.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:21.957+0000 7f68427fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:22.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f6858218640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:37:22.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f6858218640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:22.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f6858218640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:22.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f685013cbe0 con 0x7f6850137830 2024-09-17T16:37:22.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f68427fc640 1 -- 172.21.15.105:0/1018380807 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f6848051d50 con 0x7f6850137830 2024-09-17T16:37:22.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f68427fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:22.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.185+0000 7f68427fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:22.195 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":62,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 10 librados: watch_flush enter 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 10 librados: watch_flush exit 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 20 client.54292.objecter shutdown clearing up homeless session... 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 10 client.54292.objecter successfully canceled tick 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 msgr2=0x7f6834040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 0x7f6834040a10 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ad000 tx=0x7f684c0a6000 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 10 monclient: shutdown 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 msgr2=0x7f685006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6848007d60 tx=0x7f684800e4b0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 shutdown_connections 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f683403e550 0x7f6834040a10 unknown :-1 s=CLOSED pgs=3516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6850137830 0x7f685006dcd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68500a6020 0x7f685006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 --2- 172.21.15.105:0/1018380807 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f68500a56c0 0x7f685006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 >> 172.21.15.105:0/1018380807 conn(0x7f68500aa390 msgr2=0x7f68500a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 shutdown_connections 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 -- 172.21.15.105:0/1018380807 wait complete. 2024-09-17T16:37:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.189+0000 7f6858218640 1 librados: shutdown 2024-09-17T16:37:22.212 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:37:22.212 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:37:22.212 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.317+0000 7fb007595640 10 monclient: get_monmap_and_config 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.317+0000 7fb007595640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.317+0000 7fb007595640 10 monclient: monmap: 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.330 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:22.323703+0000 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:22.323703+0000 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:22.331 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 Processor -- start 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 -- start start 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient: init 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient: _add_conns ranks=[4,10,8,2,6,0,5,3,7,1,9] 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0000a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): picked mon.noname-c con 0x7fb0000a4640 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb00006c620 0x7fb0000a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): picked mon.noname-f con 0x7fb00006c620 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): picked mon.noname-e con 0x7fb0001363f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb0000a54a0 con 0x7fb00006c620 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb00009e190 con 0x7fb0000a4640 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb00009da00 con 0x7fb0001363f0 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb007595640 10 monclient(hunting): authenticate will time out at 1525.589538s 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0000a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43406/0 (socket says 172.21.15.105:43406) 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 -- 172.21.15.105:0/3761868281 learned_addr learned my addr 172.21.15.105:0/3761868281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): get_auth_request con 0x7fb0001363f0 auth_method 0 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): get_auth_request con 0x7fb0000a4640 auth_method 0 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb004b30640 1 --2- 172.21.15.105:0/3761868281 >> v2:172.21.15.160:3300/0 conn(0x7fb00006c620 0x7fb0000a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb004b30640 10 monclient(hunting): get_auth_request con 0x7fb00006c620 auth_method 0 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb004b30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb004b30640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb004b30640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient(hunting): handle_auth_done global_id 54269 payload 274 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 -- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 msgr2=0x7fb0000a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0000a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 -- 172.21.15.105:0/3761868281 >> v2:172.21.15.160:3300/0 conn(0x7fb00006c620 msgr2=0x7fb0000a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 --2- 172.21.15.105:0/3761868281 >> v2:172.21.15.160:3300/0 conn(0x7fb00006c620 0x7fb0000a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 monclient: found mon.noname-e 2024-09-17T16:37:22.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005b32640 1 -- 172.21.15.105:0/3761868281 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00009deb0 con 0x7fb0001363f0 2024-09-17T16:37:22.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.321+0000 7fb005331640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0000a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faff8009e50 tx=0x7faff801c980 comp rx=0 tx=0).ready entity=mon.8 client_cookie=98934f727c3ef6a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 1 -- 172.21.15.105:0/3761868281 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff80090d0 con 0x7fb0001363f0 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: dump: 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.345 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 1 -- 172.21.15.105:0/3761868281 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff80a5b40 con 0x7fb0001363f0 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 5 monclient: authenticate success, global_id 54269 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 1 -- 172.21.15.105:0/3761868281 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff80a5d30 con 0x7fb0001363f0 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff3fff640 10 monclient: dump: 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.347 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 4 set_mon_vals no callback set 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: shutdown 2024-09-17T16:37:22.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 msgr2=0x7fb0001368a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faff8009e50 tx=0x7faff801c980 comp rx=0 tx=0).stop 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/3761868281 shutdown_connections 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- 172.21.15.105:0/3761868281 >> v2:172.21.15.160:3300/0 conn(0x7fb00006c620 0x7fb0000a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb0001368a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- 172.21.15.105:0/3761868281 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0000a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/3761868281 >> 172.21.15.105:0/3761868281 conn(0x7fb000068dd0 msgr2=0x7fb0000691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/3761868281 shutdown_connections 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/3761868281 wait complete. 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: monmap: 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.349 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.350 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 librados: starting msgr at 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 librados: starting objecter 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 Processor -- start 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- start start 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 librados: setting wanted keys 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 librados: calling monclient init 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: init 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _add_conns ranks=[7,4,8,0,3,1,2,5,6] 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb00006c620 0x7fb0001d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): picked mon.noname-i con 0x7fb00006c620 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0001d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): picked mon.noname-c con 0x7fb0000a4640 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb00013d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): picked mon.noname-e con 0x7fb0001363f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb000143360 con 0x7fb0000a4640 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb00009f340 con 0x7fb00006c620 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb0001434e0 con 0x7fb0001363f0 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): authenticate will time out at 1525.594424s 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb00013d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0001d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb00013d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43420/0 (socket says 172.21.15.105:43420) 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 learned_addr learned my addr 172.21.15.105:0/2260355638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): get_auth_request con 0x7fb0001363f0 auth_method 0 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): get_auth_request con 0x7fb0000a4640 auth_method 0 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb004b30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient(hunting): handle_auth_done global_id 54278 payload 824 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb00006c620 msgr2=0x7fb0001d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb00006c620 0x7fb0001d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 msgr2=0x7fb0001d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb0000a4640 0x7fb0001d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 monclient: found mon.noname-e 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00013eee0 con 0x7fb0001363f0 2024-09-17T16:37:22.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb005b32640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb00013d4e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faff801c7b0 tx=0x7faff8019720 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 client.?.objecter ms_handle_connect 0x7fb0001363f0 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff80bc020 con 0x7fb0001363f0 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: dump: 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.355 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff8009210 con 0x7fb0001363f0 2024-09-17T16:37:22.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faff8003bb0 con 0x7fb0001363f0 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 5 monclient: authenticate success, global_id 54278 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _renew_subs 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb000120960 con 0x7fb0001363f0 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff37fe640 4 set_mon_vals no callback set 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7faff1ffb640 10 monclient: dump: 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.357 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 client.54278.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _renew_subs 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00013fa40 con 0x7fb0001363f0 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 librados: init done 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 0x7fb00013de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.325+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0001205e0 con 0x7fb00013da20 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 0x7fb00013de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient: get_auth_request con 0x7fb00013da20 auth_method 0 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7faff80bf020 con 0x7fb0001363f0 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 10 monclient(hunting): handle_auth_done global_id 35613 payload 824 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0001356f0 con 0x7fb00013da20 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafe403e550 0x7fafe4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(376..376 src has 1..376) ==== 6202+0+0 (secure 0 0 0) 0x7faff80b3040 con 0x7fb0001363f0 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 client.54278.objecter ms_dispatch 0x7fb000000ba0 osd_map(376..376 src has 1..376) 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 3 client.54278.objecter handle_osd_map got epochs [376,376] > 0 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 3 client.54278.objecter handle_osd_map decoding full epoch 376 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb005331640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 0x7fb00013de00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e770 tx=0x7faffc00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 20 client.54278.objecter dump_active .. 0 homeless 2024-09-17T16:37:22.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 client.54278.objecter ms_handle_connect 0x7fb00013da20 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 client.54278.objecter resend_mon_ops 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faffc017070 con 0x7fb00013da20 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 monclient: dump: 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb004b30640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafe403e550 0x7fafe4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb004b30640 10 monclient: get_auth_request con 0x7fafe403e550 auth_method 0 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faffc00c720 con 0x7fb00013da20 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 msgr2=0x7fb00013de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 0x7fb00013de00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e770 tx=0x7faffc00eb70 comp rx=0 tx=0).stop 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 msgr2=0x7fb00013de00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013da20 0x7fb00013de00 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e770 tx=0x7faffc00eb70 comp rx=0 tx=0).stop 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7faff1ffb640 10 client.54278.objecter ms_handle_connect 0x7fafe403e550 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.329+0000 7fb004b30640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafe403e550 0x7fafe4040a10 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7fb00013eb50 tx=0x7faff4008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb007595640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb007595640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 0x7fb0000a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb007595640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0001200c0 con 0x7fb00013e340 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 0x7fb0000a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient: get_auth_request con 0x7fb00013e340 auth_method 0 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 10 monclient(hunting): handle_auth_done global_id 35622 payload 824 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 1 -- 172.21.15.105:0/2260355638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb0000a7c20 con 0x7fb00013e340 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7fb005331640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 0x7fb0000a0e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e770 tx=0x7faffc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 client.54278.objecter ms_handle_connect 0x7fb00013e340 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 client.54278.objecter resend_mon_ops 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faffc017070 con 0x7fb00013e340 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 monclient: dump: 2024-09-17T16:37:22.364 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7faffc0063f0 con 0x7fb00013e340 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 msgr2=0x7fb0000a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 0x7fb0000a0e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faffc00e770 tx=0x7faffc0046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 msgr2=0x7fb0000a0e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.333+0000 7faff1ffb640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb00013e340 0x7fb0000a0e50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:22.366 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 433137, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:22.367 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.369 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:22.370 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.371 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.372 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.373 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.374 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:22.375 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:22.376 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:22.377 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.378 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:22.379 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.380 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.381 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.382 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.383 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:22.384 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.385 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.386 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.387 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.388 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.390 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:22.391 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 10 librados: watch_flush enter 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 10 librados: watch_flush exit 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 20 client.54278.objecter shutdown clearing up homeless session... 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 10 client.54278.objecter successfully canceled tick 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafe403e550 msgr2=0x7fafe4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fafe403e550 0x7fafe4040a10 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7fb00013eb50 tx=0x7faff4008040 comp rx=0 tx=0).stop 2024-09-17T16:37:22.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 10 monclient: shutdown 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 msgr2=0x7fb00013d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 --2- 172.21.15.105:0/2260355638 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb0001363f0 0x7fb00013d4e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faff801c7b0 tx=0x7faff8019720 comp rx=0 tx=0).stop 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 reap_dead start 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb005b32640 1 -- 172.21.15.105:0/2260355638 reap_dead start 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 shutdown_connections 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 >> 172.21.15.105:0/2260355638 conn(0x7fb000068dd0 msgr2=0x7fb0000a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 shutdown_connections 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 -- 172.21.15.105:0/2260355638 wait complete. 2024-09-17T16:37:22.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.337+0000 7fb007595640 1 librados: shutdown 2024-09-17T16:37:22.395 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:22.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:37:22.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:37:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: get_monmap_and_config 2024-09-17T16:37:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: monmap: 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:22.505443+0000 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:22.505443+0000 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.509 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:22.510 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:22.510 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 Processor -- start 2024-09-17T16:37:22.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 -- start start 2024-09-17T16:37:22.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: init 2024-09-17T16:37:22.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient: _add_conns ranks=[6,0,8,1,5,9,7,10,2,3,4] 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 0x7ff5c40a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-d con 0x7ff5c40a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5c40a18f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-e con 0x7ff5c40a4070 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c40a1d70 con 0x7ff5c40a18f0 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff5c40a66c0 con 0x7ff5c40a5860 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff5c40a33d0 con 0x7ff5c40a4070 2024-09-17T16:37:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c8f8c640 10 monclient(hunting): authenticate will time out at 1525.771593s 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 0x7ff5c40a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c1d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff5c40a18f0 0x7ff5c4137610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37620/0 (socket says 172.21.15.105:37620) 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c1d9b640 1 -- 172.21.15.105:0/3531501072 learned_addr learned my addr 172.21.15.105:0/3531501072 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43436/0 (socket says 172.21.15.105:43436) 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): get_auth_request con 0x7ff5c40a4070 auth_method 0 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): get_auth_request con 0x7ff5c40a5860 auth_method 0 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.501+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 1 -- 172.21.15.105:0/3531501072 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2295644331 0 0) 0x7ff5ac001120 con 0x7ff5c40a18f0 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient(hunting): _negotiate 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient(hunting): my global_id is 35631 2024-09-17T16:37:22.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 1 -- 172.21.15.105:0/3531501072 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5b0003530 con 0x7ff5c40a18f0 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 10 monclient(hunting): handle_auth_done global_id 24405 payload 274 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 -- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 msgr2=0x7ff5c40a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 -- 172.21.15.105:0/3531501072 >> v1:172.21.15.105:6789/0 conn(0x7ff5c40a18f0 legacy=0x7ff5c4137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 monclient: found mon.noname-d 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 -- 172.21.15.105:0/3531501072 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c40a36b0 con 0x7ff5c40a5860 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c2d9d640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 0x7ff5c40a5c40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009df0 tx=0x7ff5b401c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=aed29f8fd08b9024 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 1 -- 172.21.15.105:0/3531501072 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b40b2070 con 0x7ff5c40a5860 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: dump: 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 5 monclient: authenticate success, global_id 24405 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 1 -- 172.21.15.105:0/3531501072 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b40a5a80 con 0x7ff5c40a5860 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 1 -- 172.21.15.105:0/3531501072 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b40a5c70 con 0x7ff5c40a5860 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 4 set_mon_vals no callback set 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c159a640 10 monclient: dump: 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.527 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 10 monclient: shutdown 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 msgr2=0x7ff5c40a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 0x7ff5c40a5c40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009df0 tx=0x7ff5b401c850 comp rx=0 tx=0).stop 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3531501072 shutdown_connections 2024-09-17T16:37:22.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a4070 0x7ff5c40a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3531501072 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff5c40a5860 0x7ff5c40a5c40 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3531501072 >> 172.21.15.105:0/3531501072 conn(0x7ff5c40aa450 msgr2=0x7ff5c40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3531501072 shutdown_connections 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3531501072 wait complete. 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.505+0000 7ff5c8f8c640 10 monclient: monmap: 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:22.529 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 librados: starting msgr at 2024-09-17T16:37:22.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 librados: starting objecter 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 Processor -- start 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- start start 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 librados: setting wanted keys 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 librados: calling monclient init 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: init 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _add_conns ranks=[8,4,0,7,5,2,1,6,3] 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a18f0 0x7ff5c40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-e con 0x7ff5c40a18f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 0x7ff5c4136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-c con 0x7ff5c40a4070 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 0x7ff5c4136b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): picked mon.noname-a con 0x7ff5c40a5860 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:22.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4142490 con 0x7ff5c40a5860 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff5c4142310 con 0x7ff5c40a4070 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff5c4142610 con 0x7ff5c40a18f0 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): authenticate will time out at 1525.776790s 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 0x7ff5c4136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 0x7ff5c4136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a18f0 0x7ff5c40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 0x7ff5c4136b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46238/0 (socket says 172.21.15.105:46238) 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 0x7ff5c4136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51816/0 (socket says 172.21.15.105:51816) 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 learned_addr learned my addr 172.21.15.105:0/3118473730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): get_auth_request con 0x7ff5c40a5860 auth_method 0 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): get_auth_request con 0x7ff5c40a4070 auth_method 0 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): get_auth_request con 0x7ff5c40a18f0 auth_method 0 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient(hunting): handle_auth_done global_id 35640 payload 824 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a18f0 msgr2=0x7ff5c40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a18f0 0x7ff5c40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 msgr2=0x7ff5c4136620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 0x7ff5c4136620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 monclient: found mon.noname-a 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c4122ff0 con 0x7ff5c40a5860 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff5c40a18f0 0x7ff5c40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff5c40a4070 0x7ff5c4136620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c2d9d640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 0x7ff5c4136b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e950 tx=0x7ff5bc00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 client.?.objecter ms_handle_connect 0x7ff5c40a5860 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5bc017070 con 0x7ff5c40a5860 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: dump: 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.537 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.538 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5bc013470 con 0x7ff5c40a5860 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5bc013a90 con 0x7ff5c40a5860 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 5 monclient: authenticate success, global_id 35640 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _renew_subs 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c406f0d0 con 0x7ff5c40a5860 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:22.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c0d99640 4 set_mon_vals no callback set 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: dump: 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.540 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 client.35640.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _renew_subs 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c4122d10 con 0x7ff5c40a5860 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 librados: init done 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 0x7ff5c4120880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c4051e80 con 0x7ff5c41204a0 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff5bc013c30 con 0x7ff5c40a5860 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 0x7ff5c4120880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient: get_auth_request con 0x7ff5c41204a0 auth_method 0 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff5a403e580 0x7ff5a4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 10 monclient(hunting): handle_auth_done global_id 44696 payload 824 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(376..376 src has 1..376) ==== 6202+0+0 (secure 0 0 0) 0x7ff5bc00f040 con 0x7ff5c40a5860 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 client.35640.objecter ms_dispatch 0x7ff5c4000ba0 osd_map(376..376 src has 1..376) 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 3 client.35640.objecter handle_osd_map got epochs [376,376] > 0 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 3 client.35640.objecter handle_osd_map decoding full epoch 376 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5c406ec10 con 0x7ff5c41204a0 2024-09-17T16:37:22.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 20 client.35640.objecter dump_active .. 0 homeless 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 client.35640.objecter ms_handle_connect 0x7ff5c41204a0 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 client.35640.objecter resend_mon_ops 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 0x7ff5c4120880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40a72c0 tx=0x7ff5b40195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b40bc070 con 0x7ff5c41204a0 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff5a403e580 0x7ff5a4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: dump: 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.543 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5c1d9b640 10 monclient: get_auth_request con 0x7ff5a403e580 auth_method 0 2024-09-17T16:37:22.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff5b40b3040 con 0x7ff5c41204a0 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 msgr2=0x7ff5c4120880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 0x7ff5c4120880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40a72c0 tx=0x7ff5b40195f0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 msgr2=0x7ff5c4120880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.509+0000 7ff5baffd640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c41204a0 0x7ff5c4120880 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40a72c0 tx=0x7ff5b40195f0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.513+0000 7ff5baffd640 10 client.35640.objecter ms_handle_connect 0x7ff5a403e580 2024-09-17T16:37:22.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.513+0000 7ff5c1d9b640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff5a403e580 0x7ff5a4040a40 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4073970 tx=0x7ff5ac009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 0x7ff5c409da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c4137610 con 0x7ff5c4120dc0 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 0x7ff5c409da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient: get_auth_request con 0x7ff5c4120dc0 auth_method 0 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 10 monclient(hunting): handle_auth_done global_id 44705 payload 824 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 1 -- 172.21.15.105:0/3118473730 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5c40a60a0 con 0x7ff5c4120dc0 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c259c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 0x7ff5c409da00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5b401d0f0 tx=0x7ff5b40a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 client.35640.objecter ms_handle_connect 0x7ff5c4120dc0 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 client.35640.objecter resend_mon_ops 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b40b2070 con 0x7ff5c4120dc0 2024-09-17T16:37:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 monclient: dump: 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.549 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7ff5b40a5dd0 con 0x7ff5c4120dc0 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 msgr2=0x7ff5c409da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 0x7ff5c409da00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5b401d0f0 tx=0x7ff5b40a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 msgr2=0x7ff5c409da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5baffd640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5c4120dc0 0x7ff5c409da00 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5b401d0f0 tx=0x7ff5b40a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 293170, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:22.551 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.552 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.553 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:22.554 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.555 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:22.556 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.557 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.558 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.559 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.561 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.562 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.563 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.564 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:22.565 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.566 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:22.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.569 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.571 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.572 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.574 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:22.574 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:22.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 librados: watch_flush enter 2024-09-17T16:37:22.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 librados: watch_flush exit 2024-09-17T16:37:22.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 20 client.35640.objecter shutdown clearing up homeless session... 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 client.35640.objecter successfully canceled tick 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff5a403e580 msgr2=0x7ff5a4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff5a403e580 0x7ff5a4040a40 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4073970 tx=0x7ff5ac009040 comp rx=0 tx=0).stop 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 10 monclient: shutdown 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 msgr2=0x7ff5c4136b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 --2- 172.21.15.105:0/3118473730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5c40a5860 0x7ff5c4136b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e950 tx=0x7ff5bc00ee10 comp rx=0 tx=0).stop 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 reap_dead start 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c2d9d640 1 -- 172.21.15.105:0/3118473730 reap_dead start 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 shutdown_connections 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 >> 172.21.15.105:0/3118473730 conn(0x7ff5c40aa450 msgr2=0x7ff5c40a8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 shutdown_connections 2024-09-17T16:37:22.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 -- 172.21.15.105:0/3118473730 wait complete. 2024-09-17T16:37:22.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.517+0000 7ff5c8f8c640 1 librados: shutdown 2024-09-17T16:37:22.577 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:22.577 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:37:22.577 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.681+0000 7fa58c734640 10 monclient: get_monmap_and_config 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.681+0000 7fa58c734640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient: monmap: 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.691 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:22.687787+0000 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:22.687787+0000 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.692 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:22.693 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:22.693 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 Processor -- start 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 -- start start 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient: init 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient: _add_conns ranks=[2,7,6,4,0,5,1,10,8,9,3] 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-b con 0x7fa5840a5860 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:22.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 0x7fa5840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-i con 0x7fa5840a18f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa584137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-d con 0x7fa5840a4070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840a66c0 con 0x7fa5840a5860 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa5840a1d70 con 0x7fa5840a4070 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa5840a33d0 con 0x7fa5840a18f0 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58c734640 10 monclient(hunting): authenticate will time out at 1525.953649s 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50204/0 (socket says 172.21.15.105:50204) 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1948301014 learned_addr learned my addr 172.21.15.105:0/1948301014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): get_auth_request con 0x7fa5840a5860 auth_method 0 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa584137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): get_auth_request con 0x7fa5840a4070 auth_method 0 2024-09-17T16:37:22.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_done global_id 44714 payload 274 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1948301014 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 msgr2=0x7fa5840a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 0x7fa5840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 0x7fa5840a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 msgr2=0x7fa584137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa584137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 monclient: found mon.noname-b 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1948301014 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840a36b0 con 0x7fa5840a5860 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa58acd1640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa584137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa589ccf640 10 monclient: get_auth_request con 0x7fa5840a18f0 auth_method 0 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.685+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 0x7fa5840a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5800098d0 tx=0x7fa58001c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f330926c285d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 1 -- 172.21.15.105:0/1948301014 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5800b0070 con 0x7fa5840a5860 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: dump: 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.711 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 1 -- 172.21.15.105:0/1948301014 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5800aa810 con 0x7fa5840a5860 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 5 monclient: authenticate success, global_id 44714 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 1 -- 172.21.15.105:0/1948301014 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5800a9300 con 0x7fa5840a5860 2024-09-17T16:37:22.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 4 set_mon_vals no callback set 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa5894ce640 10 monclient: dump: 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.713 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: shutdown 2024-09-17T16:37:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 msgr2=0x7fa5840a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5800098d0 tx=0x7fa58001c120 comp rx=0 tx=0).stop 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- 172.21.15.105:0/1948301014 shutdown_connections 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa5840a18f0 0x7fa5840a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa584137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- 172.21.15.105:0/1948301014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a5860 0x7fa5840a5c40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- 172.21.15.105:0/1948301014 >> 172.21.15.105:0/1948301014 conn(0x7fa5840aa450 msgr2=0x7fa5840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- 172.21.15.105:0/1948301014 shutdown_connections 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- 172.21.15.105:0/1948301014 wait complete. 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: monmap: 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 librados: starting msgr at 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 librados: starting objecter 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 Processor -- start 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- start start 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 librados: setting wanted keys 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 librados: calling monclient init 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: init 2024-09-17T16:37:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient: _add_conns ranks=[2,6,3,8,0,1,5,4,7] 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 0x7fa584135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-b con 0x7fa5840a18f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa5841360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-d con 0x7fa5840a4070 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5840a5860 0x7fa58413b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): picked mon.noname-g con 0x7fa5840a5860 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa584141480 con 0x7fa5840a18f0 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa584141300 con 0x7fa5840a5860 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa584141600 con 0x7fa5840a4070 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58c734640 10 monclient(hunting): authenticate will time out at 1525.959811s 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 0x7fa584135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa589ccf640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa5841360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 0x7fa584135bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50216/0 (socket says 172.21.15.105:50216) 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1895686921 learned_addr learned my addr 172.21.15.105:0/1895686921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 10 monclient(hunting): get_auth_request con 0x7fa5840a18f0 auth_method 0 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa58a4d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa589ccf640 10 monclient(hunting): get_auth_request con 0x7fa5840a4070 auth_method 0 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa589ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa589ccf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.689+0000 7fa589ccf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient(hunting): handle_auth_done global_id 44723 payload 824 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 msgr2=0x7fa5841360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa5840a4070 0x7fa5841360f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5840a5860 msgr2=0x7fa58413b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5840a5860 0x7fa58413b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 monclient: found mon.noname-b 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58413da10 con 0x7fa5840a18f0 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58a4d0640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 0x7fa584135bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ae040 tx=0x7fa580018f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.?.objecter ms_handle_connect 0x7fa5840a18f0 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5800b0040 con 0x7fa5840a18f0 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: dump: 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.722 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5800aae30 con 0x7fa5840a18f0 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 5 monclient: authenticate success, global_id 44723 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 monclient: _renew_subs 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840a5f40 con 0x7fa5840a18f0 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5800044b0 con 0x7fa5840a18f0 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa588ccd640 4 set_mon_vals no callback set 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: dump: 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 client.44723.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 monclient: _renew_subs 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58413ce80 con 0x7fa5840a18f0 2024-09-17T16:37:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 1 librados: init done 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 0x7fa58413bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa584051e80 con 0x7fa584137330 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 0x7fa58413bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient: get_auth_request con 0x7fa584137330 auth_method 0 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa580004040 con 0x7fa5840a18f0 2024-09-17T16:37:22.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 10 monclient(hunting): handle_auth_done global_id 54328 payload 824 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa56803e540 0x7fa568040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5840008e0 con 0x7fa584137330 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(376..376 src has 1..376) ==== 6202+0+0 (secure 0 0 0) 0x7fa5800ea120 con 0x7fa5840a18f0 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.44723.objecter ms_dispatch 0x7fa584000ba0 osd_map(376..376 src has 1..376) 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 3 client.44723.objecter handle_osd_map got epochs [376,376] > 0 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 3 client.44723.objecter handle_osd_map decoding full epoch 376 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 0x7fa58413bab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa57400f830 tx=0x7fa57400fcf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 20 client.44723.objecter dump_active .. 0 homeless 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.44723.objecter ms_handle_connect 0x7fa584137330 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.44723.objecter resend_mon_ops 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa57400de90 con 0x7fa584137330 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: dump: 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.729 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58acd1640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa56803e540 0x7fa568040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58acd1640 10 monclient: get_auth_request con 0x7fa56803e540 auth_method 0 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa57400da20 con 0x7fa584137330 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 msgr2=0x7fa58413bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 0x7fa58413bab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa57400f830 tx=0x7fa57400fcf0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 msgr2=0x7fa58413bab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa584137330 0x7fa58413bab0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa57400f830 tx=0x7fa57400fcf0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa57affd640 10 client.44723.objecter ms_handle_connect 0x7fa56803e540 2024-09-17T16:37:22.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.693+0000 7fa58acd1640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa56803e540 0x7fa568040a00 secure :-1 s=READY pgs=3519 cs=0 l=1 rev1=1 crypto rx=0x7fa5841206b0 tx=0x7fa57c00d040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.697+0000 7fa58c734640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.697+0000 7fa58c734640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 0x7fa58413c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa58413c940 con 0x7fa58413bff0 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 0x7fa58413c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient: get_auth_request con 0x7fa58413bff0 auth_method 0 2024-09-17T16:37:22.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 10 monclient(hunting): handle_auth_done global_id 54337 payload 824 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 1 -- 172.21.15.105:0/1895686921 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa584136630 con 0x7fa58413bff0 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 client.44723.objecter ms_handle_connect 0x7fa58413bff0 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 client.44723.objecter resend_mon_ops 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa589ccf640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 0x7fa58413c400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa574008010 tx=0x7fa5740046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5740065f0 con 0x7fa58413bff0 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 monclient: dump: 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.737 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fa574006180 con 0x7fa58413bff0 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:22.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 msgr2=0x7fa58413c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 0x7fa58413c400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa574008010 tx=0x7fa5740046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 msgr2=0x7fa58413c400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa57affd640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa58413bff0 0x7fa58413c400 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa574008010 tx=0x7fa5740046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:22.739 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 149699, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:22.740 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.741 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.742 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:22.744 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.745 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:22.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:22.747 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.748 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:22.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:22.750 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.755 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.757 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.758 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.761 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.763 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.764 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.765 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.766 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:22.766 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 10 librados: watch_flush enter 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 10 librados: watch_flush exit 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 20 client.44723.objecter shutdown clearing up homeless session... 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 10 client.44723.objecter successfully canceled tick 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa56803e540 msgr2=0x7fa568040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa56803e540 0x7fa568040a00 secure :-1 s=READY pgs=3519 cs=0 l=1 rev1=1 crypto rx=0x7fa5841206b0 tx=0x7fa57c00d040 comp rx=0 tx=0).stop 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 10 monclient: shutdown 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 msgr2=0x7fa584135bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 --2- 172.21.15.105:0/1895686921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa5840a18f0 0x7fa584135bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5800ae040 tx=0x7fa580018f20 comp rx=0 tx=0).stop 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58acd1640 1 -- 172.21.15.105:0/1895686921 reap_dead start 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58acd1640 1 -- 172.21.15.105:0/1895686921 reap_dead start 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 shutdown_connections 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 >> 172.21.15.105:0/1895686921 conn(0x7fa5840aa450 msgr2=0x7fa58409dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 shutdown_connections 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 -- 172.21.15.105:0/1895686921 wait complete. 2024-09-17T16:37:22.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.701+0000 7fa58c734640 1 librados: shutdown 2024-09-17T16:37:22.769 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:22.770 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:37:22.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:37:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: get_monmap_and_config 2024-09-17T16:37:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: monmap: 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:22.872417+0000 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:22.872417+0000 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.876 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:22.877 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:22.878 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 Processor -- start 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 -- start start 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: init 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient: _add_conns ranks=[1,7,3,4,6,5,9,8,10,0,2] 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f71b00a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 0x7f71b013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-i con 0x7f71b00a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:22.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-g con 0x7f71b00a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71b006ebc0 con 0x7f71b00a68e0 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f71b00a3fc0 con 0x7f71b00a73d0 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f71b00a63c0 con 0x7f71b00a4910 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71b4edc640 10 monclient(hunting): authenticate will time out at 1526.138606s 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ae59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f71b00a68e0 0x7f71b00a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:48668/0 (socket says 172.21.15.105:48668) 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ae59c640 1 -- 172.21.15.105:0/2571876079 learned_addr learned my addr 172.21.15.105:0/2571876079 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:22.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 0x7f71b013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): get_auth_request con 0x7f71b00a73d0 auth_method 0 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): get_auth_request con 0x7f71b00a4910 auth_method 0 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 1 -- 172.21.15.105:0/2571876079 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260040344 0 0) 0x7f71b006ebc0 con 0x7f71b00a68e0 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 10 monclient(hunting): _negotiate 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 10 monclient(hunting): my global_id is 54307 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71ad59a640 1 -- 172.21.15.105:0/2571876079 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7198003530 con 0x7f71b00a68e0 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient(hunting): handle_auth_done global_id 64123 payload 274 2024-09-17T16:37:22.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 -- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 msgr2=0x7f71b0141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 -- 172.21.15.105:0/2571876079 >> v1:172.21.15.160:6789/0 conn(0x7f71b00a68e0 legacy=0x7f71b00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 monclient: found mon.noname-i 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71add9b640 1 -- 172.21.15.105:0/2571876079 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b00a3690 con 0x7f71b00a4910 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.869+0000 7f71aed9d640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71add9b640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 0x7f71b013eaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097f0 tx=0x7f71a401c370 comp rx=0 tx=0).ready entity=mon.7 client_cookie=84e53d5238a70390 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 1 -- 172.21.15.105:0/2571876079 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a40b0070 con 0x7f71b00a4910 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: dump: 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.894 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 1 -- 172.21.15.105:0/2571876079 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a40aa810 con 0x7f71b00a4910 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 5 monclient: authenticate success, global_id 64123 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 4 set_mon_vals no callback set 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 1 -- 172.21.15.105:0/2571876079 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a40aa9c0 con 0x7f71b00a4910 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71ad59a640 10 monclient: dump: 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.896 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.897 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 10 monclient: shutdown 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 msgr2=0x7f71b013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 0x7f71b013eaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097f0 tx=0x7f71a401c370 comp rx=0 tx=0).stop 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- 172.21.15.105:0/2571876079 shutdown_connections 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 --2- 172.21.15.105:0/2571876079 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f71b00a4910 0x7f71b013eaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- 172.21.15.105:0/2571876079 >> 172.21.15.105:0/2571876079 conn(0x7f71b0068ba0 msgr2=0x7f71b0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- 172.21.15.105:0/2571876079 shutdown_connections 2024-09-17T16:37:22.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- 172.21.15.105:0/2571876079 wait complete. 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 10 monclient: build_initial_monmap 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 10 monclient: monmap: 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:22.899 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 librados: starting msgr at 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 librados: starting objecter 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 Processor -- start 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 -- start start 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 librados: setting wanted keys 2024-09-17T16:37:22.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 1 librados: calling monclient init 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.873+0000 7f71b4edc640 10 monclient: init 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _add_conns ranks=[2,6,3,1,8,7,5,0,4] 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 0x7f71b0126210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-b con 0x7f71b00a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b00a68e0 0x7f71b0126750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-d con 0x7f71b00a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0126c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): picked mon.noname-g con 0x7f71b00a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f71b0143f20 con 0x7f71b00a4910 2024-09-17T16:37:22.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f71b0143da0 con 0x7f71b00a73d0 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f71b01440a0 con 0x7f71b00a68e0 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): authenticate will time out at 1526.144270s 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b00a68e0 0x7f71b0126750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 0x7f71b0126210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 0x7f71b0126210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50230/0 (socket says 172.21.15.105:50230) 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 -- 172.21.15.105:0/3832961527 learned_addr learned my addr 172.21.15.105:0/3832961527 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): get_auth_request con 0x7f71b00a4910 auth_method 0 2024-09-17T16:37:22.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): get_auth_request con 0x7f71b00a68e0 auth_method 0 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0126c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 10 monclient(hunting): get_auth_request con 0x7f71b00a73d0 auth_method 0 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient(hunting): handle_auth_done global_id 44732 payload 824 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b00a68e0 msgr2=0x7f71b0126750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b00a68e0 0x7f71b0126750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 msgr2=0x7f71b0126c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71b00a73d0 0x7f71b0126c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 monclient: found mon.noname-b 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:22.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b0129080 con 0x7f71b00a4910 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b00a68e0 0x7f71b0126750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71ae59c640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 0x7f71b0126210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c500 tx=0x7f71a000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.?.objecter ms_handle_connect 0x7f71b00a4910 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a0017070 con 0x7f71b00a4910 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: dump: 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.906 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.907 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: _finish_auth 0 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0007070 con 0x7f71b00a4910 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 5 monclient: authenticate success, global_id 44732 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _renew_subs 2024-09-17T16:37:22.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b0127b90 con 0x7f71b00a4910 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a0007690 con 0x7f71b00a4910 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 client.44732.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71acd99640 4 set_mon_vals no callback set 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: dump: 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.909 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.910 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _renew_subs 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b0129650 con 0x7f71b00a4910 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 librados: init done 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 0x7f71b01d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f71b013fcb0 con 0x7f71b01408a0 2024-09-17T16:37:22.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 0x7f71b01d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient: get_auth_request con 0x7f71b01408a0 auth_method 0 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f71a0007220 con 0x7f71b00a4910 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f719003e550 0x7f7190040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 10 monclient(hunting): handle_auth_done global_id 24441 payload 824 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71b006de40 con 0x7f71b01408a0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(376..376 src has 1..376) ==== 6202+0+0 (secure 0 0 0) 0x7f71a004f0e0 con 0x7f71b00a4910 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.44732.objecter ms_dispatch 0x7f71b0000ba0 osd_map(376..376 src has 1..376) 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 3 client.44732.objecter handle_osd_map got epochs [376,376] > 0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 3 client.44732.objecter handle_osd_map decoding full epoch 376 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 20 client.44732.objecter dump_active .. 0 homeless 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.44732.objecter ms_handle_connect 0x7f71b01408a0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.44732.objecter resend_mon_ops 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71add9b640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 0x7f71b01d0fb0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f71b0074a10 tx=0x7f71a4019110 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a40b0040 con 0x7f71b01408a0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: dump: 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.914 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f719003e550 0x7f7190040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 10 monclient: get_auth_request con 0x7f719003e550 auth_method 0 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f71a40aa750 con 0x7f71b01408a0 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:22.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 msgr2=0x7f71b01d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 0x7f71b01d0fb0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f71b0074a10 tx=0x7f71a4019110 comp rx=0 tx=0).stop 2024-09-17T16:37:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 msgr2=0x7f71b01d0fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01408a0 0x7f71b01d0fb0 secure :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f71b0074a10 tx=0x7f71a4019110 comp rx=0 tx=0).stop 2024-09-17T16:37:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f719effd640 10 client.44732.objecter ms_handle_connect 0x7f719003e550 2024-09-17T16:37:22.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.877+0000 7f71aed9d640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f719003e550 0x7f7190040a10 secure :-1 s=READY pgs=3520 cs=0 l=1 rev1=1 crypto rx=0x7f71b0141240 tx=0x7f71a800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 0x7f71b01d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f71b01d1e40 con 0x7f71b01d14f0 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 0x7f71b01d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient: get_auth_request con 0x7f71b01d14f0 auth_method 0 2024-09-17T16:37:22.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 10 monclient(hunting): handle_auth_done global_id 24450 payload 824 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 1 -- 172.21.15.105:0/3832961527 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f71b0129ac0 con 0x7f71b01d14f0 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 client.44732.objecter ms_handle_connect 0x7f71b01d14f0 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 client.44732.objecter resend_mon_ops 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71add9b640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 0x7f71b01d1900 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f71a401aa20 tx=0x7f71a40a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:22.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71a40b0070 con 0x7f71b01d14f0 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 monclient: dump: 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:22.920 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f71a4003bf0 con 0x7f71b01d14f0 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 msgr2=0x7f71b01d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 0x7f71b01d1900 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f71a401aa20 tx=0x7f71a40a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 msgr2=0x7f71b01d1900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f719effd640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f71b01d14f0 0x7f71b01d1900 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 590524, 2024-09-17T16:37:22.922 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:22.923 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.924 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.925 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:22.926 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:22.927 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:22.928 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:22.929 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:22.930 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.931 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.934 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.935 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.937 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.939 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.941 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:22.942 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.944 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.945 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.946 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.947 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:22.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:22.949 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:22.949 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:22.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 librados: watch_flush enter 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 librados: watch_flush exit 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 20 client.44732.objecter shutdown clearing up homeless session... 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 client.44732.objecter successfully canceled tick 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f719003e550 msgr2=0x7f7190040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f719003e550 0x7f7190040a10 secure :-1 s=READY pgs=3520 cs=0 l=1 rev1=1 crypto rx=0x7f71b0141240 tx=0x7f71a800b040 comp rx=0 tx=0).stop 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 10 monclient: shutdown 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 msgr2=0x7f71b0126210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 --2- 172.21.15.105:0/3832961527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b00a4910 0x7f71b0126210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c500 tx=0x7f71a000c9c0 comp rx=0 tx=0).stop 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71aed9d640 1 -- 172.21.15.105:0/3832961527 reap_dead start 2024-09-17T16:37:22.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71aed9d640 1 -- 172.21.15.105:0/3832961527 reap_dead start 2024-09-17T16:37:22.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 shutdown_connections 2024-09-17T16:37:22.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 >> 172.21.15.105:0/3832961527 conn(0x7f71b0068ba0 msgr2=0x7f71b009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:22.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.885+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 shutdown_connections 2024-09-17T16:37:22.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.889+0000 7f71b4edc640 1 -- 172.21.15.105:0/3832961527 wait complete. 2024-09-17T16:37:22.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:22.889+0000 7f71b4edc640 1 librados: shutdown 2024-09-17T16:37:22.952 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:22.952 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:37:22.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:37:23.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.057+0000 7f6e002ed640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.057+0000 7f6e002ed640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.057+0000 7f6e002ed640 10 monclient: monmap: 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.063715+0000 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.063715+0000 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.068 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 Processor -- start 2024-09-17T16:37:23.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 -- start start 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient: init 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient: _add_conns ranks=[7,4,1,6,10,0,3,2,8,5,9] 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-i con 0x7f6df80a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-c con 0x7f6df80a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6df81396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df806ebc0 con 0x7f6df81396a0 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6df80a3fc0 con 0x7f6df80a4910 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6df80a63c0 con 0x7f6df80a68e0 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6e002ed640 10 monclient(hunting): authenticate will time out at 1526.329504s 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51850/0 (socket says 172.21.15.105:51850) 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 -- 172.21.15.105:0/1157055543 learned_addr learned my addr 172.21.15.105:0/1157055543 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): get_auth_request con 0x7f6df80a4910 auth_method 0 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfe089640 10 monclient(hunting): get_auth_request con 0x7f6df80a68e0 auth_method 0 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfe089640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfe089640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfe089640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient(hunting): handle_auth_done global_id 54346 payload 274 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 -- 172.21.15.105:0/1157055543 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 msgr2=0x7f6df80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 -- 172.21.15.105:0/1157055543 >> v1:172.21.15.160:6789/0 conn(0x7f6df81396a0 legacy=0x7f6df8139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 monclient: found mon.noname-c 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.061+0000 7f6dfd888640 1 -- 172.21.15.105:0/1157055543 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df80a3690 con 0x7f6df80a4910 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6de80098a0 tx=0x7f6de801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4f97e66f3cd4c153 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 1 -- 172.21.15.105:0/1157055543 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6de80b0070 con 0x7f6df80a4910 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: dump: 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.082 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 5 monclient: authenticate success, global_id 54346 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 1 -- 172.21.15.105:0/1157055543 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de80aa810 con 0x7f6df80a4910 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 4 set_mon_vals no callback set 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 1 -- 172.21.15.105:0/1157055543 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6de80a9300 con 0x7f6df80a4910 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd087640 10 monclient: dump: 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.084 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.085 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: shutdown 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1157055543 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 msgr2=0x7f6df80a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6de80098a0 tx=0x7f6de801c440 comp rx=0 tx=0).stop 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1157055543 shutdown_connections 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6df80a4910 0x7f6df80a7200 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1157055543 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1157055543 >> 172.21.15.105:0/1157055543 conn(0x7f6df8068ba0 msgr2=0x7f6df8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1157055543 shutdown_connections 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1157055543 wait complete. 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: monmap: 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.087 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 librados: starting msgr at 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 librados: starting objecter 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 Processor -- start 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- start start 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 librados: setting wanted keys 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 librados: calling monclient init 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: init 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _add_conns ranks=[3,7,2,8,6,1,5,4,0] 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6df80a4910 0x7f6df813d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-g con 0x7f6df80a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df813dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-i con 0x7f6df80a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 0x7f6df813e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): picked mon.noname-b con 0x7f6df81396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6df81444b0 con 0x7f6df81396a0 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6df8144330 con 0x7f6df80a4910 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6df8144630 con 0x7f6df80a68e0 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient(hunting): authenticate will time out at 1526.334494s 2024-09-17T16:37:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 0x7f6df813e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 0x7f6df813e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50236/0 (socket says 172.21.15.105:50236) 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 learned_addr learned my addr 172.21.15.105:0/1214651599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): get_auth_request con 0x7f6df81396a0 auth_method 0 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df813dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 10 monclient(hunting): get_auth_request con 0x7f6df80a68e0 auth_method 0 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6df80a4910 0x7f6df813d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe089640 10 monclient(hunting): get_auth_request con 0x7f6df80a4910 auth_method 0 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe089640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe089640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe089640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient(hunting): handle_auth_done global_id 44741 payload 824 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 msgr2=0x7f6df813dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df813dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6df80a4910 msgr2=0x7f6df813d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6df80a4910 0x7f6df813d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 monclient: found mon.noname-b 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df81402f0 con 0x7f6df81396a0 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6df80a68e0 0x7f6df813dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfe88a640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 0x7f6df813e130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c710 tx=0x7f6df400cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 client.?.objecter ms_handle_connect 0x7f6df81396a0 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6df40017e0 con 0x7f6df81396a0 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: dump: 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df400ce20 con 0x7f6df81396a0 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 5 monclient: authenticate success, global_id 44741 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _renew_subs 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df8122660 con 0x7f6df81396a0 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6df400acc0 con 0x7f6df81396a0 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6dfc886640 4 set_mon_vals no callback set 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 client.44741.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6deeffd640 10 monclient: dump: 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.095 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _renew_subs 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df8140b30 con 0x7f6df81396a0 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.065+0000 7f6e002ed640 1 librados: init done 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6e002ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6df400a850 con 0x7f6df81396a0 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 0x7f6df80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6df8051e80 con 0x7f6df80a0a20 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 0x7f6df80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient: get_auth_request con 0x7f6df80a0a20 auth_method 0 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6ddc03e550 0x7f6ddc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(377..377 src has 1..377) ==== 6202+0+0 (secure 0 0 0) 0x7f6df404fe20 con 0x7f6df81396a0 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 client.44741.objecter ms_dispatch 0x7f6df8000ba0 osd_map(377..377 src has 1..377) 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 3 client.44741.objecter handle_osd_map got epochs [377,377] > 0 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 3 client.44741.objecter handle_osd_map decoding full epoch 377 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 20 client.44741.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 10 monclient(hunting): handle_auth_done global_id 54305 payload 824 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6df813f550 con 0x7f6df80a0a20 2024-09-17T16:37:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6ddc03e550 0x7f6ddc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfd888640 10 monclient: get_auth_request con 0x7f6ddc03e550 auth_method 0 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 0x7f6df80a0e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f4b0 tx=0x7f6df000f970 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 client.44741.objecter ms_handle_connect 0x7f6df80a0a20 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 client.44741.objecter resend_mon_ops 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6df0019020 con 0x7f6df80a0a20 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 monclient: dump: 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6df0014670 con 0x7f6df80a0a20 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 msgr2=0x7f6df80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 0x7f6df80a0e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f4b0 tx=0x7f6df000f970 comp rx=0 tx=0).stop 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 msgr2=0x7f6df80a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a0a20 0x7f6df80a0e00 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f4b0 tx=0x7f6df000f970 comp rx=0 tx=0).stop 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6deeffd640 10 client.44741.objecter ms_handle_connect 0x7f6ddc03e550 2024-09-17T16:37:23.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.069+0000 7f6dfd888640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6ddc03e550 0x7f6ddc040a10 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7f6de80ad000 tx=0x7f6de80a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6e002ed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 0x7f6df81db590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6e002ed640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6df80a9de0 con 0x7f6df80a1340 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 0x7f6df81db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient: get_auth_request con 0x7f6df80a1340 auth_method 0 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.073+0000 7f6dfe089640 10 monclient(hunting): handle_auth_done global_id 54314 payload 824 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6dfe089640 1 -- 172.21.15.105:0/1214651599 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6df8139ae0 con 0x7f6df80a1340 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 client.44741.objecter ms_handle_connect 0x7f6df80a1340 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6dfe089640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 0x7f6df81db590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f4b0 tx=0x7f6df00046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 client.44741.objecter resend_mon_ops 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6df0019040 con 0x7f6df80a1340 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 monclient: dump: 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.103 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.104 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f6df0004050 con 0x7f6df80a1340 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 msgr2=0x7f6df81db590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 0x7f6df81db590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6df000f4b0 tx=0x7f6df00046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 msgr2=0x7f6df81db590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6deeffd640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6df80a1340 0x7f6df81db590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.105 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150075, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:23.106 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.107 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.108 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:23.109 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.110 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:23.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:23.118 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.119 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:23.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:23.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.127 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:23.128 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:23.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 10 librados: watch_flush enter 2024-09-17T16:37:23.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 10 librados: watch_flush exit 2024-09-17T16:37:23.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 20 client.44741.objecter shutdown clearing up homeless session... 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 10 client.44741.objecter successfully canceled tick 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6ddc03e550 msgr2=0x7f6ddc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6ddc03e550 0x7f6ddc040a10 secure :-1 s=READY pgs=3521 cs=0 l=1 rev1=1 crypto rx=0x7f6de80ad000 tx=0x7f6de80a6000 comp rx=0 tx=0).stop 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 10 monclient: shutdown 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 msgr2=0x7f6df813e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 --2- 172.21.15.105:0/1214651599 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6df81396a0 0x7f6df813e130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c710 tx=0x7f6df400cbd0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 reap_dead start 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6dfe88a640 1 -- 172.21.15.105:0/1214651599 reap_dead start 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 shutdown_connections 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 >> 172.21.15.105:0/1214651599 conn(0x7f6df8068ba0 msgr2=0x7f6df80a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 shutdown_connections 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 -- 172.21.15.105:0/1214651599 wait complete. 2024-09-17T16:37:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.077+0000 7f6e002ed640 1 librados: shutdown 2024-09-17T16:37:23.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:23.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:37:23.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: monmap: 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.237071+0000 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.237071+0000 2024-09-17T16:37:23.240 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.241 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 Processor -- start 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 -- start start 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: init 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient: _add_conns ranks=[2,8,3,9,0,6,5,4,10,7,1] 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-b con 0x7ff8c00a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 0x7ff8c00a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-e con 0x7ff8c00a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8c01396a0 0x7ff8c0139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-g con 0x7ff8c01396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c00a3fc0 con 0x7ff8c00a68e0 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c00a63c0 con 0x7ff8c01396a0 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff8c00a7740 con 0x7ff8c00a4910 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8c503d640 10 monclient(hunting): authenticate will time out at 1526.502779s 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 0x7ff8c00a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50238/0 (socket says 172.21.15.105:50238) 2024-09-17T16:37:23.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 -- 172.21.15.105:0/2647699976 learned_addr learned my addr 172.21.15.105:0/2647699976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): get_auth_request con 0x7ff8c00a68e0 auth_method 0 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): get_auth_request con 0x7ff8c00a4910 auth_method 0 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bf59e640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8c01396a0 0x7ff8c0139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bf59e640 10 monclient(hunting): get_auth_request con 0x7ff8c01396a0 auth_method 0 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bf59e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_done global_id 44750 payload 274 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 -- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 msgr2=0x7ff8c00a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 0x7ff8c00a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 -- 172.21.15.105:0/2647699976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8c01396a0 msgr2=0x7ff8c0139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8c01396a0 0x7ff8c0139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 monclient: found mon.noname-b 2024-09-17T16:37:23.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8bed9d640 1 -- 172.21.15.105:0/2647699976 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c006ec10 con 0x7ff8c00a68e0 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.233+0000 7ff8be59c640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 0x7ff8c00a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009eb0 tx=0x7ff8b801d350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240f547544a6f11b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 1 -- 172.21.15.105:0/2647699976 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b80090d0 con 0x7ff8c00a68e0 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: dump: 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.250 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 1 -- 172.21.15.105:0/2647699976 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b80a5a50 con 0x7ff8c00a68e0 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 1 -- 172.21.15.105:0/2647699976 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b80a5c40 con 0x7ff8c00a68e0 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 5 monclient: authenticate success, global_id 44750 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 4 set_mon_vals no callback set 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8bdd9b640 10 monclient: dump: 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.252 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: shutdown 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 msgr2=0x7ff8c00a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009eb0 tx=0x7ff8b801d350 comp rx=0 tx=0).stop 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- 172.21.15.105:0/2647699976 shutdown_connections 2024-09-17T16:37:23.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8c01396a0 0x7ff8c0139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff8c00a4910 0x7ff8c00a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- 172.21.15.105:0/2647699976 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8c00a68e0 0x7ff8c00a6cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- 172.21.15.105:0/2647699976 >> 172.21.15.105:0/2647699976 conn(0x7ff8c0068ba0 msgr2=0x7ff8c0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- 172.21.15.105:0/2647699976 shutdown_connections 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- 172.21.15.105:0/2647699976 wait complete. 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: monmap: 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 librados: starting msgr at 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 librados: starting objecter 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 Processor -- start 2024-09-17T16:37:23.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 -- start start 2024-09-17T16:37:23.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 librados: setting wanted keys 2024-09-17T16:37:23.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 librados: calling monclient init 2024-09-17T16:37:23.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: init 2024-09-17T16:37:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient: _add_conns ranks=[1,4,5,0,2,3,8,6,7] 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c00a4910 0x7ff8c01d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-f con 0x7ff8c00a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.237+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-c con 0x7ff8c00a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff8c01396a0 0x7ff8c013d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): picked mon.noname-h con 0x7ff8c01396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0143320 con 0x7ff8c00a4910 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff8c006e630 con 0x7ff8c00a68e0 2024-09-17T16:37:23.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff8c01434a0 con 0x7ff8c01396a0 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): authenticate will time out at 1526.508043s 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51854/0 (socket says 172.21.15.105:51854) 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 -- 172.21.15.105:0/4000305133 learned_addr learned my addr 172.21.15.105:0/4000305133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): get_auth_request con 0x7ff8c00a68e0 auth_method 0 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient(hunting): handle_auth_done global_id 54355 payload 824 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff8c01396a0 0x7ff8c013d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff8c01396a0 msgr2=0x7ff8c013d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c00a4910 0x7ff8c01d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff8c01396a0 0x7ff8c013d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c00a4910 msgr2=0x7ff8c01d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c00a4910 0x7ff8c01d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 monclient: found mon.noname-c 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c00ab970 con 0x7ff8c00a68e0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 10 monclient: get_auth_request con 0x7ff8c01396a0 auth_method 0 2024-09-17T16:37:23.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff8c01396a0 0x7ff8c013d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8be59c640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009fd0 tx=0x7ff8b400ee10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.?.objecter ms_handle_connect 0x7ff8c00a68e0 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8c00a68e0 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: dump: 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.266 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 5 monclient: authenticate success, global_id 54355 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b4013470 con 0x7ff8c00a68e0 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient: _renew_subs 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c00a9de0 con 0x7ff8c00a68e0 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b4013a90 con 0x7ff8c00a68e0 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 client.54355.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: dump: 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.268 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bd59a640 4 set_mon_vals no callback set 2024-09-17T16:37:23.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient: _renew_subs 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c0123c10 con 0x7ff8c00a68e0 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 librados: init done 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 0x7ff8c013db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c0051e80 con 0x7ff8c0123370 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff8b401a070 con 0x7ff8c00a68e0 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 0x7ff8c013db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient: get_auth_request con 0x7ff8c0123370 auth_method 0 2024-09-17T16:37:23.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8ac03e550 0x7ff8ac040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(377..377 src has 1..377) ==== 6202+0+0 (secure 0 0 0) 0x7ff8b400f040 con 0x7ff8c00a68e0 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.54355.objecter ms_dispatch 0x7ff8c0000ba0 osd_map(377..377 src has 1..377) 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 3 client.54355.objecter handle_osd_map got epochs [377,377] > 0 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 3 client.54355.objecter handle_osd_map decoding full epoch 377 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 20 client.54355.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8ac03e550 0x7ff8ac040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 10 monclient: get_auth_request con 0x7ff8ac03e550 auth_method 0 2024-09-17T16:37:23.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_done global_id 54316 payload 824 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c0139ae0 con 0x7ff8c0123370 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.54355.objecter ms_handle_connect 0x7ff8c0123370 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 0x7ff8c013db10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009e80 tx=0x7ff8b8019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.54355.objecter resend_mon_ops 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b80bc030 con 0x7ff8c0123370 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: dump: 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.272 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 client.54355.objecter ms_handle_connect 0x7ff8ac03e550 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8bf59e640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8ac03e550 0x7ff8ac040a10 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0074620 tx=0x7ff8b0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff8b8004040 con 0x7ff8c0123370 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 msgr2=0x7ff8c013db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 0x7ff8c013db10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009e80 tx=0x7ff8b8019690 comp rx=0 tx=0).stop 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 msgr2=0x7ff8c013db10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.241+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c0123370 0x7ff8c013db10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8c503d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8c503d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 0x7ff8c013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8c503d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c013ee90 con 0x7ff8c013e050 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 0x7ff8c013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient: get_auth_request con 0x7ff8c013e050 auth_method 0 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 10 monclient(hunting): handle_auth_done global_id 54325 payload 824 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 1 -- 172.21.15.105:0/4000305133 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff8c013e9a0 con 0x7ff8c013e050 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8bed9d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 0x7ff8c013e460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009eb0 tx=0x7ff8b80a7070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 client.54355.objecter ms_handle_connect 0x7ff8c013e050 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 client.54355.objecter resend_mon_ops 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b80c6070 con 0x7ff8c013e050 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 monclient: dump: 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.275 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff8b80c2810 con 0x7ff8c013e050 2024-09-17T16:37:23.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 msgr2=0x7ff8c013e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 0x7ff8c013e460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009eb0 tx=0x7ff8b80a7070 comp rx=0 tx=0).stop 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 msgr2=0x7ff8c013e460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.249+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8c013e050 0x7ff8c013e460 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009eb0 tx=0x7ff8b80a7070 comp rx=0 tx=0).stop 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:23.277 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150258, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.278 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.279 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.280 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:23.281 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.282 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:23.283 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:23.284 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.285 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.286 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:23.287 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.288 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:23.290 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:23.296 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.298 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.299 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:23.300 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 10 librados: watch_flush enter 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 10 librados: watch_flush exit 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 20 client.54355.objecter shutdown clearing up homeless session... 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 10 client.54355.objecter successfully canceled tick 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8ac03e550 msgr2=0x7ff8ac040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8ac03e550 0x7ff8ac040a10 secure :-1 s=READY pgs=3522 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0074620 tx=0x7ff8b0008040 comp rx=0 tx=0).stop 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 10 monclient: shutdown 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 msgr2=0x7ff8c01d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009fd0 tx=0x7ff8b400ee10 comp rx=0 tx=0).stop 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8bf59e640 1 -- 172.21.15.105:0/4000305133 reap_dead start 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 shutdown_connections 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 --2- 172.21.15.105:0/4000305133 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff8c00a68e0 0x7ff8c01d1b90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 >> 172.21.15.105:0/4000305133 conn(0x7ff8c0068ba0 msgr2=0x7ff8c00a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 shutdown_connections 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 -- 172.21.15.105:0/4000305133 wait complete. 2024-09-17T16:37:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.253+0000 7ff8c503d640 1 librados: shutdown 2024-09-17T16:37:23.303 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:23.303 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:37:23.304 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.413+0000 7f132a0a3640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.413+0000 7f132a0a3640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient: monmap: 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.419765+0000 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.419765+0000 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.423 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.424 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 Processor -- start 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 -- start start 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient: init 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient: _add_conns ranks=[7,4,8,9,3,10,1,2,5,6,0] 2024-09-17T16:37:23.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f13240a4750 0x7f13240a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-i con 0x7f13240a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-c con 0x7f132406c2e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-e con 0x7f1324138520 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f13240a55b0 con 0x7f132406c2e0 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f13240a6610 con 0x7f13240a4750 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f13240a61d0 con 0x7f1324138520 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f132a0a3640 10 monclient(hunting): authenticate will time out at 1526.685739s 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51862/0 (socket says 172.21.15.105:51862) 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43480/0 (socket says 172.21.15.105:43480) 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 -- 172.21.15.105:0/2280219608 learned_addr learned my addr 172.21.15.105:0/2280219608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): get_auth_request con 0x7f132406c2e0 auth_method 0 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): get_auth_request con 0x7f1324138520 auth_method 0 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f13237fe640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f13240a4750 0x7f13240a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f13237fe640 10 monclient(hunting): get_auth_request con 0x7f13240a4750 auth_method 0 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f13237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f13237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f13237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient(hunting): handle_auth_done global_id 54364 payload 274 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 -- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 msgr2=0x7f1324138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 -- 172.21.15.105:0/2280219608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f13240a4750 msgr2=0x7f13240a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f13240a4750 0x7f13240a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 monclient: found mon.noname-c 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1322ffd640 1 -- 172.21.15.105:0/2280219608 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13240a5f90 con 0x7f132406c2e0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.417+0000 7f1323fff640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f131801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f72a65c6a45a1031 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 1 -- 172.21.15.105:0/2280219608 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13180b0070 con 0x7f132406c2e0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: dump: 2024-09-17T16:37:23.432 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.433 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 1 -- 172.21.15.105:0/2280219608 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13180aa810 con 0x7f132406c2e0 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 5 monclient: authenticate success, global_id 54364 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 1 -- 172.21.15.105:0/2280219608 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13180a9300 con 0x7f132406c2e0 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 4 set_mon_vals no callback set 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13227fc640 10 monclient: dump: 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.435 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: shutdown 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 msgr2=0x7f13240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13180098a0 tx=0x7f131801c440 comp rx=0 tx=0).stop 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2280219608 shutdown_connections 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1324138520 0x7f1324138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f132406c2e0 0x7f13240a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2280219608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f13240a4750 0x7f13240a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2280219608 >> 172.21.15.105:0/2280219608 conn(0x7f13240a94d0 msgr2=0x7f13240a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2280219608 shutdown_connections 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2280219608 wait complete. 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: monmap: 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.437 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.438 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.438 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 librados: starting msgr at 2024-09-17T16:37:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 librados: starting objecter 2024-09-17T16:37:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 Processor -- start 2024-09-17T16:37:23.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- start start 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 librados: setting wanted keys 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 librados: calling monclient init 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: init 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _add_conns ranks=[2,8,7,1,0,3,6,4,5] 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 0x7f1324136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-b con 0x7f132406c2e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 0x7f1324136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-e con 0x7f13240a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1324138520 0x7f1324137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): picked mon.noname-i con 0x7f1324138520 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1324143370 con 0x7f132406c2e0 2024-09-17T16:37:23.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f132409ee20 con 0x7f1324138520 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f13241434f0 con 0x7f13240a4750 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient(hunting): authenticate will time out at 1526.690676s 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 0x7f1324136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 0x7f1324136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 0x7f1324136900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50254/0 (socket says 172.21.15.105:50254) 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 0x7f1324136e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43482/0 (socket says 172.21.15.105:43482) 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 1 -- 172.21.15.105:0/2291497601 learned_addr learned my addr 172.21.15.105:0/2291497601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): get_auth_request con 0x7f132406c2e0 auth_method 0 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): get_auth_request con 0x7f13240a4750 auth_method 0 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient(hunting): handle_auth_done global_id 54341 payload 824 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1324138520 msgr2=0x7f1324137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1324138520 0x7f1324137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 msgr2=0x7f1324136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 0x7f1324136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 monclient: found mon.noname-e 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1324122680 con 0x7f13240a4750 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f13237fe640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f132406c2e0 0x7f1324136900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1322ffd640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 0x7f1324136e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13180ab000 tx=0x7f13180191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 client.?.objecter ms_handle_connect 0x7f13240a4750 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13180b0040 con 0x7f13240a4750 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: dump: 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.449 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13180044b0 con 0x7f13240a4750 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 5 monclient: authenticate success, global_id 54341 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _renew_subs 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1324121d60 con 0x7f13240a4750 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13180aacb0 con 0x7f13240a4750 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:23.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f1321ffb640 4 set_mon_vals no callback set 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f130bfff640 10 monclient: dump: 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 client.54341.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _renew_subs 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f13240a7100 con 0x7f13240a4750 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.421+0000 7f132a0a3640 1 librados: init done 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f132a0a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 0x7f132413bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1324051e80 con 0x7f132413b920 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1318004040 con 0x7f13240a4750 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 0x7f132413bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient: get_auth_request con 0x7f132413b920 auth_method 0 2024-09-17T16:37:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f131003e540 0x7f1310040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(377..377 src has 1..377) ==== 6202+0+0 (secure 0 0 0) 0x7f13180e9100 con 0x7f13240a4750 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 client.54341.objecter ms_dispatch 0x7f1324000ba0 osd_map(377..377 src has 1..377) 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 3 client.54341.objecter handle_osd_map got epochs [377,377] > 0 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 3 client.54341.objecter handle_osd_map decoding full epoch 377 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 20 client.54341.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f1323fff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f131003e540 0x7f1310040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f1323fff640 10 monclient: get_auth_request con 0x7f131003e540 auth_method 0 2024-09-17T16:37:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 10 monclient(hunting): handle_auth_done global_id 54282 payload 824 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1324120fc0 con 0x7f132413b920 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 client.54341.objecter ms_handle_connect 0x7f132413b920 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 client.54341.objecter resend_mon_ops 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f13237fe640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 0x7f132413bd00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1314010cb0 tx=0x7f131400e5a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13140044a0 con 0x7f132413b920 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 monclient: dump: 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 client.54341.objecter ms_handle_connect 0x7f131003e540 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f1323fff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f131003e540 0x7f1310040a00 secure :-1 s=READY pgs=3523 cs=0 l=1 rev1=1 crypto rx=0x7f13240a6dd0 tx=0x7f131c00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1314004030 con 0x7f132413b920 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 msgr2=0x7f132413bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 0x7f132413bd00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1314010cb0 tx=0x7f131400e5a0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 msgr2=0x7f132413bd00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.425+0000 7f130bfff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413b920 0x7f132413bd00 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1314010cb0 tx=0x7f131400e5a0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 0x7f132411f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f132406a200 con 0x7f132413c240 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 0x7f132411f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient: get_auth_request con 0x7f132413c240 auth_method 0 2024-09-17T16:37:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 10 monclient(hunting): handle_auth_done global_id 54291 payload 824 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 1 -- 172.21.15.105:0/2291497601 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1324138960 con 0x7f132413c240 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 client.54341.objecter ms_handle_connect 0x7f132413c240 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f13237fe640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 0x7f132411f390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1314010cb0 tx=0x7f13140046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 client.54341.objecter resend_mon_ops 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f131402a020 con 0x7f132413c240 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 monclient: dump: 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f1314014430 con 0x7f132413c240 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 msgr2=0x7f132411f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 0x7f132411f390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1314010cb0 tx=0x7f13140046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 msgr2=0x7f132411f390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f130bfff640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f132413c240 0x7f132411f390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.459 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150442, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.460 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.461 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:23.463 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.464 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.466 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:23.467 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.468 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.470 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.472 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.473 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.475 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:23.476 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.477 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.480 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.481 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:23.482 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 librados: watch_flush enter 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 librados: watch_flush exit 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 20 client.54341.objecter shutdown clearing up homeless session... 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 client.54341.objecter successfully canceled tick 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f131003e540 msgr2=0x7f1310040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f131003e540 0x7f1310040a00 secure :-1 s=READY pgs=3523 cs=0 l=1 rev1=1 crypto rx=0x7f13240a6dd0 tx=0x7f131c00b040 comp rx=0 tx=0).stop 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 10 monclient: shutdown 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 msgr2=0x7f1324136e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 --2- 172.21.15.105:0/2291497601 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f13240a4750 0x7f1324136e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13180ab000 tx=0x7f13180191e0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f1323fff640 1 -- 172.21.15.105:0/2291497601 reap_dead start 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f1323fff640 1 -- 172.21.15.105:0/2291497601 reap_dead start 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 shutdown_connections 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 >> 172.21.15.105:0/2291497601 conn(0x7f13240a94d0 msgr2=0x7f13241394f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 shutdown_connections 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 -- 172.21.15.105:0/2291497601 wait complete. 2024-09-17T16:37:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.433+0000 7f132a0a3640 1 librados: shutdown 2024-09-17T16:37:23.485 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:23.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:37:23.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:37:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: monmap: 2024-09-17T16:37:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.592449+0000 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.592449+0000 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.596 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.597 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.597 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 Processor -- start 2024-09-17T16:37:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 -- start start 2024-09-17T16:37:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: init 2024-09-17T16:37:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient: _add_conns ranks=[9,1,6,2,5,3,7,8,10,4,0] 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13c40a5860 0x7f13c40a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-a con 0x7f13c40a5860 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f13c40a3f80 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 0x7f13c4137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-d con 0x7f13c40a6180 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c40a35a0 con 0x7f13c40a3f80 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f13c40746f0 con 0x7f13c40a5860 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f13c40a19e0 con 0x7f13c40a6180 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c8e2d640 10 monclient(hunting): authenticate will time out at 1526.857837s 2024-09-17T16:37:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13c40a5860 0x7f13c40a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f13c40a5860 0x7f13c40a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46244/0 (socket says 172.21.15.105:46244) 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 -- 172.21.15.105:0/3254801634 learned_addr learned my addr 172.21.15.105:0/3254801634 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): get_auth_request con 0x7f13c40a5860 auth_method 0 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 0x7f13c4137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): get_auth_request con 0x7f13c40a6180 auth_method 0 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient(hunting): handle_auth_done global_id 35649 payload 274 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 -- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 msgr2=0x7f13c4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 0x7f13c4137d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 -- 172.21.15.105:0/3254801634 >> v1:172.21.15.160:6789/0 conn(0x7f13c40a3f80 legacy=0x7f13c4137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 monclient: found mon.noname-a 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c259c640 1 -- 172.21.15.105:0/3254801634 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c40a2d00 con 0x7f13c40a5860 2024-09-17T16:37:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.589+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 0x7f13c4137d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40a5860 0x7f13c40a5c40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009e50 tx=0x7f13bc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8ede0a48edf89e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 1 -- 172.21.15.105:0/3254801634 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13bc0090d0 con 0x7f13c40a5860 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: dump: 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 1 -- 172.21.15.105:0/3254801634 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc0a5be0 con 0x7f13c40a5860 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 1 -- 172.21.15.105:0/3254801634 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13bc0a5dd0 con 0x7f13c40a5860 2024-09-17T16:37:23.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 5 monclient: authenticate success, global_id 35649 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 4 set_mon_vals no callback set 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c159a640 10 monclient: dump: 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.612 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: shutdown 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40a5860 msgr2=0x7f13c40a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40a5860 0x7f13c40a5c40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009e50 tx=0x7f13bc01c950 comp rx=0 tx=0).stop 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/3254801634 shutdown_connections 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f13c40a6180 0x7f13c4137d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/3254801634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c40a5860 0x7f13c40a5c40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/3254801634 >> 172.21.15.105:0/3254801634 conn(0x7f13c40aa450 msgr2=0x7f13c40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/3254801634 shutdown_connections 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/3254801634 wait complete. 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: monmap: 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.614 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 librados: starting msgr at 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 librados: starting objecter 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 Processor -- start 2024-09-17T16:37:23.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- start start 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 librados: setting wanted keys 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 librados: calling monclient init 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: init 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _add_conns ranks=[3,4,0,8,6,5,7,1,2] 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13c40a5860 0x7f13c41cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-g con 0x7f13c40a5860 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 0x7f13c41cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-c con 0x7f13c40a6180 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 0x7f13c413c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): picked mon.noname-a con 0x7f13c413c180 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40a6560 con 0x7f13c413c180 2024-09-17T16:37:23.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f13c4138260 con 0x7f13c40a5860 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f13c4121630 con 0x7f13c40a6180 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient(hunting): authenticate will time out at 1526.862558s 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 0x7f13c41cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 0x7f13c413c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 0x7f13c413c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46250/0 (socket says 172.21.15.105:46250) 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 0x7f13c41cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51874/0 (socket says 172.21.15.105:51874) 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 learned_addr learned my addr 172.21.15.105:0/2683936788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): get_auth_request con 0x7f13c413c180 auth_method 0 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): get_auth_request con 0x7f13c40a6180 auth_method 0 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13c40a5860 0x7f13c41cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 10 monclient(hunting): get_auth_request con 0x7f13c40a5860 auth_method 0 2024-09-17T16:37:23.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient(hunting): handle_auth_done global_id 35658 payload 824 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 msgr2=0x7f13c41cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 0x7f13c41cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13c40a5860 msgr2=0x7f13c41cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13c40a5860 0x7f13c41cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 monclient: found mon.noname-a 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c40a9490 con 0x7f13c413c180 2024-09-17T16:37:23.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f13c40a6180 0x7f13c41cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c2d9d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 0x7f13c413c5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e930 tx=0x7f13b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 client.?.objecter ms_handle_connect 0x7f13c413c180 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b4017070 con 0x7f13c413c180 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: dump: 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.620 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b400cdb0 con 0x7f13c413c180 2024-09-17T16:37:23.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 5 monclient: authenticate success, global_id 35658 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _renew_subs 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c40a0460 con 0x7f13c413c180 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13b40138e0 con 0x7f13c413c180 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13baffd640 10 monclient: dump: 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.622 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c0d99640 4 set_mon_vals no callback set 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 client.35658.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _renew_subs 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c41209f0 con 0x7f13c413c180 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.593+0000 7f13c8e2d640 1 librados: init done 2024-09-17T16:37:23.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c8e2d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 0x7f13c413cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f13c40a66e0 con 0x7f13c40a3f80 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f13b4013470 con 0x7f13c413c180 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 0x7f13c413cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13a403e550 0x7f13a4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient: get_auth_request con 0x7f13c40a3f80 auth_method 0 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(377..377 src has 1..377) ==== 6202+0+0 (secure 0 0 0) 0x7f13b400f040 con 0x7f13c413c180 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 client.35658.objecter ms_dispatch 0x7f13c4000ba0 osd_map(377..377 src has 1..377) 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 3 client.35658.objecter handle_osd_map got epochs [377,377] > 0 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 3 client.35658.objecter handle_osd_map decoding full epoch 377 2024-09-17T16:37:23.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 20 client.35658.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13a403e550 0x7f13a4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c1d9b640 10 monclient: get_auth_request con 0x7f13a403e550 auth_method 0 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 10 monclient(hunting): handle_auth_done global_id 20021 payload 824 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13c4138040 con 0x7f13c40a3f80 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 client.35658.objecter ms_handle_connect 0x7f13c40a3f80 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 client.35658.objecter resend_mon_ops 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c259c640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 0x7f13c413cae0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f13c40aaed0 tx=0x7f13bc0196f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 client.35658.objecter ms_handle_connect 0x7f13a403e550 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13c1d9b640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13a403e550 0x7f13a4040a10 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7f13c413d8e0 tx=0x7f13ac007e40 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13bc0bd070 con 0x7f13c40a3f80 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 monclient: dump: 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f13bc0a5770 con 0x7f13c40a3f80 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 msgr2=0x7f13c413cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 0x7f13c413cae0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f13c40aaed0 tx=0x7f13bc0196f0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 msgr2=0x7f13c413cae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.597+0000 7f13baffd640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c40a3f80 0x7f13c413cae0 secure :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f13c40aaed0 tx=0x7f13bc0196f0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 0x7f13c40a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f13c4051e80 con 0x7f13c413d020 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 0x7f13c40a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient: get_auth_request con 0x7f13c413d020 auth_method 0 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 10 monclient(hunting): handle_auth_done global_id 20030 payload 824 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 1 -- 172.21.15.105:0/2683936788 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13c4069f10 con 0x7f13c413d020 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c259c640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 0x7f13c40a7d50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f13bc000f90 tx=0x7f13bc0a7000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 client.35658.objecter ms_handle_connect 0x7f13c413d020 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 client.35658.objecter resend_mon_ops 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f13bc0c2c20 con 0x7f13c413d020 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 monclient: dump: 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.630 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f13bc004000 con 0x7f13c413d020 2024-09-17T16:37:23.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 msgr2=0x7f13c40a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 0x7f13c40a7d50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f13bc000f90 tx=0x7f13bc0a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 msgr2=0x7f13c40a7d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13baffd640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f13c413d020 0x7f13c40a7d50 secure :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f13bc000f90 tx=0x7f13bc0a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:37:23.632 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 591212, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:23.633 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.635 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:23.636 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.637 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.639 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.641 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.642 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.644 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.648 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.650 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.652 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.653 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:23.654 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 librados: watch_flush enter 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 librados: watch_flush exit 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 20 client.35658.objecter shutdown clearing up homeless session... 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 client.35658.objecter successfully canceled tick 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13a403e550 msgr2=0x7f13a4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f13a403e550 0x7f13a4040a10 secure :-1 s=READY pgs=3524 cs=0 l=1 rev1=1 crypto rx=0x7f13c413d8e0 tx=0x7f13ac007e40 comp rx=0 tx=0).stop 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 10 monclient: shutdown 2024-09-17T16:37:23.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 msgr2=0x7f13c413c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 --2- 172.21.15.105:0/2683936788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c413c180 0x7f13c413c5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e930 tx=0x7f13b400edf0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 reap_dead start 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c2d9d640 1 -- 172.21.15.105:0/2683936788 reap_dead start 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 shutdown_connections 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 >> 172.21.15.105:0/2683936788 conn(0x7f13c40aa450 msgr2=0x7f13c413f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 shutdown_connections 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 -- 172.21.15.105:0/2683936788 wait complete. 2024-09-17T16:37:23.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.605+0000 7f13c8e2d640 1 librados: shutdown 2024-09-17T16:37:23.657 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:23.657 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:37:23.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: monmap: 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.767848+0000 2024-09-17T16:37:23.771 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.767848+0000 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.772 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.773 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.773 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 Processor -- start 2024-09-17T16:37:23.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 -- start start 2024-09-17T16:37:23.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: init 2024-09-17T16:37:23.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient: _add_conns ranks=[1,5,7,3,0,2,8,10,4,6,9] 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5e2c0a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-h con 0x7f5e2c0a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e2c0a7520 0x7f5e2c140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-i con 0x7f5e2c0a7520 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2c0a4080 con 0x7f5e2c0a6a30 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5e2c0a7c40 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5e2c0a37e0 con 0x7f5e2c0a7520 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e30b96640 10 monclient(hunting): authenticate will time out at 1527.033262s 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e2c0a7520 0x7f5e2c140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5e2c0a6a30 0x7f5e2c0a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:48708/0 (socket says 172.21.15.105:48708) 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2a59c640 1 -- 172.21.15.105:0/3794736914 learned_addr learned my addr 172.21.15.105:0/3794736914 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a4ab0 auth_method 0 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a7520 auth_method 0 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_done global_id 20039 payload 274 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 -- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e2c0a7520 msgr2=0x7f5e2c140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 --2- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e2c0a7520 0x7f5e2c140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 -- 172.21.15.105:0/3794736914 >> v1:172.21.15.160:6789/0 conn(0x7f5e2c0a6a30 legacy=0x7f5e2c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 monclient: found mon.noname-h 2024-09-17T16:37:23.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.765+0000 7f5e29d9b640 1 -- 172.21.15.105:0/3794736914 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c0a3ac0 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 1 --2- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c13ea30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097f0 tx=0x7f5e2001c360 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7b0cd39ca20048d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 1 -- 172.21.15.105:0/3794736914 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e200b2070 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: dump: 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 5 monclient: authenticate success, global_id 20039 2024-09-17T16:37:23.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 1 -- 172.21.15.105:0/3794736914 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e200ab7f0 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 4 set_mon_vals no callback set 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 1 -- 172.21.15.105:0/3794736914 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e200abe50 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2959a640 10 monclient: dump: 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.781 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.782 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: shutdown 2024-09-17T16:37:23.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 msgr2=0x7f5e2c13ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c13ea30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5e200097f0 tx=0x7f5e2001c360 comp rx=0 tx=0).stop 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- 172.21.15.105:0/3794736914 shutdown_connections 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e2c0a7520 0x7f5e2c140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- 172.21.15.105:0/3794736914 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c13ea30 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- 172.21.15.105:0/3794736914 >> 172.21.15.105:0/3794736914 conn(0x7f5e2c0a8d00 msgr2=0x7f5e2c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- 172.21.15.105:0/3794736914 shutdown_connections 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- 172.21.15.105:0/3794736914 wait complete. 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: monmap: 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 librados: starting msgr at 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 librados: starting objecter 2024-09-17T16:37:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 Processor -- start 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- start start 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 librados: setting wanted keys 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 librados: calling monclient init 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: init 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient: _add_conns ranks=[4,0,2,7,3,5,1,8,6] 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c1d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2c0a4ab0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 0x7f5e2c1d1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-a con 0x7f5e2c0a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 0x7f5e2c1d1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): picked mon.noname-b con 0x7f5e2c0a7520 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c143e60 con 0x7f5e2c0a6a30 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c143ce0 con 0x7f5e2c0a7520 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f5e2c143fe0 con 0x7f5e2c0a4ab0 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e30b96640 10 monclient(hunting): authenticate will time out at 1527.039076s 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 0x7f5e2c1d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c1d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 0x7f5e2c1d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c1d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51880/0 (socket says 172.21.15.105:51880) 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 0x7f5e2c1d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46266/0 (socket says 172.21.15.105:46266) 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 1 -- 172.21.15.105:0/984988351 learned_addr learned my addr 172.21.15.105:0/984988351 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 0x7f5e2c1d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50260/0 (socket says 172.21.15.105:50260) 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a4ab0 auth_method 0 2024-09-17T16:37:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a6a30 auth_method 0 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): get_auth_request con 0x7f5e2c0a7520 auth_method 0 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient(hunting): handle_auth_done global_id 44768 payload 824 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 msgr2=0x7f5e2c1d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c1d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 msgr2=0x7f5e2c1d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 0x7f5e2c1d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 monclient: found mon.noname-b 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c129250 con 0x7f5e2c0a7520 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e29d9b640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2c0a6a30 0x7f5e2c1d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2a59c640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f5e2c0a4ab0 0x7f5e2c1d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e2ad9d640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 0x7f5e2c1d1970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400eb80 tx=0x7f5e2400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 10 client.?.objecter ms_handle_connect 0x7f5e2c0a7520 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e24018020 con 0x7f5e2c0a7520 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.769+0000 7f5e1affd640 10 monclient: dump: 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.797 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.798 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 5 monclient: authenticate success, global_id 44768 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24014480 con 0x7f5e2c0a7520 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 monclient: _renew_subs 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c12a0c0 con 0x7f5e2c0a7520 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 client.44768.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 monclient: _renew_subs 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c127d60 con 0x7f5e2c0a7520 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 1 librados: init done 2024-09-17T16:37:23.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e24014aa0 con 0x7f5e2c0a7520 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e28d99640 4 set_mon_vals no callback set 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: dump: 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.800 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 0x7f5e10004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5e10007cc0 con 0x7f5e10002890 2024-09-17T16:37:23.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5e2401b020 con 0x7f5e2c0a7520 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 0x7f5e10004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient: get_auth_request con 0x7f5e10002890 auth_method 0 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5e0c03e550 0x7f5e0c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(377..377 src has 1..377) ==== 6202+0+0 (secure 0 0 0) 0x7f5e24050810 con 0x7f5e2c0a7520 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 client.44768.objecter ms_dispatch 0x7f5e2c000ba0 osd_map(377..377 src has 1..377) 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 3 client.44768.objecter handle_osd_map got epochs [377,377] > 0 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 3 client.44768.objecter handle_osd_map decoding full epoch 377 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 20 client.44768.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e29d9b640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5e0c03e550 0x7f5e0c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e29d9b640 10 monclient: get_auth_request con 0x7f5e0c03e550 auth_method 0 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_done global_id 64168 payload 824 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e10007ec0 con 0x7f5e10002890 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 client.44768.objecter ms_handle_connect 0x7f5e10002890 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 client.44768.objecter resend_mon_ops 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e2a59c640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 0x7f5e10004d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007270 tx=0x7f5e1c007730 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 client.44768.objecter ms_handle_connect 0x7f5e0c03e550 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e29d9b640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5e0c03e550 0x7f5e0c040a10 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f5e200af000 tx=0x7f5e20008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e1c005860 con 0x7f5e10002890 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: dump: 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.804 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5e1c00fa50 con 0x7f5e10002890 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 msgr2=0x7f5e10004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 0x7f5e10004d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007270 tx=0x7f5e1c007730 comp rx=0 tx=0).stop 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 msgr2=0x7f5e10004d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.773+0000 7f5e1affd640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10002890 0x7f5e10004d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 0x7f5e10007170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5e1000a200 con 0x7f5e10006ce0 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 0x7f5e10007170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient: get_auth_request con 0x7f5e10006ce0 auth_method 0 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 10 monclient(hunting): handle_auth_done global_id 64177 payload 824 2024-09-17T16:37:23.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 1 -- 172.21.15.105:0/984988351 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5e100076b0 con 0x7f5e10006ce0 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 client.44768.objecter ms_handle_connect 0x7f5e10006ce0 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 client.44768.objecter resend_mon_ops 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e2a59c640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 0x7f5e10007170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007270 tx=0x7f5e1c000ee0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e1c0064a0 con 0x7f5e10006ce0 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 monclient: dump: 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.807 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f5e1c006030 con 0x7f5e10006ce0 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 msgr2=0x7f5e10007170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 0x7f5e10007170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007270 tx=0x7f5e1c000ee0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 msgr2=0x7f5e10007170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e1affd640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5e10006ce0 0x7f5e10007170 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007270 tx=0x7f5e1c000ee0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:23.809 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16883, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.810 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.811 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:23.813 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.814 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:23.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.820 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.822 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:23.824 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.827 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.829 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:23.830 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:23.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:23.831 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:23.831 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 librados: watch_flush enter 2024-09-17T16:37:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 librados: watch_flush exit 2024-09-17T16:37:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 20 client.44768.objecter shutdown clearing up homeless session... 2024-09-17T16:37:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 client.44768.objecter successfully canceled tick 2024-09-17T16:37:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5e0c03e550 msgr2=0x7f5e0c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5e0c03e550 0x7f5e0c040a10 secure :-1 s=READY pgs=3525 cs=0 l=1 rev1=1 crypto rx=0x7f5e200af000 tx=0x7f5e20008040 comp rx=0 tx=0).stop 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 10 monclient: shutdown 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.781+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 msgr2=0x7f5e2c1d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 --2- 172.21.15.105:0/984988351 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e2c0a7520 0x7f5e2c1d1970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400eb80 tx=0x7f5e2400c490 comp rx=0 tx=0).stop 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e2ad9d640 1 -- 172.21.15.105:0/984988351 reap_dead start 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e2ad9d640 1 -- 172.21.15.105:0/984988351 reap_dead start 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 shutdown_connections 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 >> 172.21.15.105:0/984988351 conn(0x7f5e2c0a8d00 msgr2=0x7f5e2c09fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 shutdown_connections 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 -- 172.21.15.105:0/984988351 wait complete. 2024-09-17T16:37:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.785+0000 7f5e30b96640 1 librados: shutdown 2024-09-17T16:37:23.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:23.834 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:37:23.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.941+0000 7f72226db640 10 monclient: get_monmap_and_config 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.941+0000 7f72226db640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.941+0000 7f72226db640 10 monclient: monmap: 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:23.947035+0000 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:23.947035+0000 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:23.950 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:23.951 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.941+0000 7f72226db640 1 Processor -- start 2024-09-17T16:37:23.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 -- start start 2024-09-17T16:37:23.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient: init 2024-09-17T16:37:23.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient: _add_conns ranks=[2,0,5,3,1,6,9,10,7,4,8] 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 0x7f721c0a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): picked mon.noname-b con 0x7f721c0a3720 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f721c0a3b00 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 0x7f721c0a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): picked mon.noname-h con 0x7f721c0a3f00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721c06db50 con 0x7f721c0a3b00 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f721c0a0e80 con 0x7f721c0a3720 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f721c0a1490 con 0x7f721c0a3f00 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient(hunting): authenticate will time out at 1527.212725s 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 0x7f721c0a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f7213fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f721c0a3b00 0x7f721c0a56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37632/0 (socket says 172.21.15.105:37632) 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f7213fff640 1 -- 172.21.15.105:0/62421945 learned_addr learned my addr 172.21.15.105:0/62421945 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): get_auth_request con 0x7f721c0a3720 auth_method 0 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 1 -- 172.21.15.105:0/62421945 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458282636 0 0) 0x7f721c06db50 con 0x7f721c0a3b00 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient(hunting): _negotiate 2024-09-17T16:37:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient(hunting): my global_id is 35676 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f7220c78640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 0x7f721c0a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 1 -- 172.21.15.105:0/62421945 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7208003530 con 0x7f721c0a3b00 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient(hunting): handle_auth_done global_id 44777 payload 274 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 -- 172.21.15.105:0/62421945 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 msgr2=0x7f721c0a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 0x7f721c0a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 -- 172.21.15.105:0/62421945 >> v1:172.21.15.105:6789/0 conn(0x7f721c0a3b00 legacy=0x7f721c0a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 monclient: found mon.noname-b 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 -- 172.21.15.105:0/62421945 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c0a7240 con 0x7f721c0a3720 2024-09-17T16:37:23.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f7220c78640 10 monclient: get_auth_request con 0x7f721c0a3f00 auth_method 0 2024-09-17T16:37:23.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f7220c78640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 0x7f721c0a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721bfff640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 0x7f721c0a1e70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72040097f0 tx=0x7f720401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d02bf23aa9cc6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 1 -- 172.21.15.105:0/62421945 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72040b0070 con 0x7f721c0a3720 2024-09-17T16:37:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: dump: 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.963 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 1 -- 172.21.15.105:0/62421945 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72040aa810 con 0x7f721c0a3720 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 5 monclient: authenticate success, global_id 44777 2024-09-17T16:37:23.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 1 -- 172.21.15.105:0/62421945 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72040a9300 con 0x7f721c0a3720 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f721b7fe640 10 monclient: dump: 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.945+0000 7f72226db640 4 set_mon_vals no callback set 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: shutdown 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/62421945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 msgr2=0x7f721c0a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 0x7f721c0a1e70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72040097f0 tx=0x7f720401c430 comp rx=0 tx=0).stop 2024-09-17T16:37:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/62421945 shutdown_connections 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f721c0a3f00 0x7f721c0a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- 172.21.15.105:0/62421945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f721c0a3720 0x7f721c0a1e70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/62421945 >> 172.21.15.105:0/62421945 conn(0x7f721c0aa390 msgr2=0x7f721c0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/62421945 shutdown_connections 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/62421945 wait complete. 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: build_initial_monmap 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: monmap: 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 librados: starting msgr at 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 librados: starting objecter 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 Processor -- start 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- start start 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 librados: setting wanted keys 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 librados: calling monclient init 2024-09-17T16:37:23.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: init 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _add_conns ranks=[1,4,6,5,2,7,3,8,0] 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 0x7f721c13e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): picked mon.noname-f con 0x7f721c0a3720 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): picked mon.noname-c con 0x7f721c0a3b00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): picked mon.noname-d con 0x7f721c0a3f00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1442e0 con 0x7f721c0a3720 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f721c0a6520 con 0x7f721c0a3b00 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:23.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f721c144460 con 0x7f721c0a3f00 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient(hunting): authenticate will time out at 1527.217821s 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51884/0 (socket says 172.21.15.105:51884) 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:59394/0 (socket says 172.21.15.105:59394) 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 -- 172.21.15.105:0/3280728349 learned_addr learned my addr 172.21.15.105:0/3280728349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): get_auth_request con 0x7f721c0a3b00 auth_method 0 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): get_auth_request con 0x7f721c0a3f00 auth_method 0 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f721bfff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 0x7f721c13e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient(hunting): handle_auth_done global_id 54391 payload 824 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 -- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 msgr2=0x7f721c13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 -- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 msgr2=0x7f721c13e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 0x7f721c13e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 monclient: found mon.noname-c 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 -- 172.21.15.105:0/3280728349 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c0a5cd0 con 0x7f721c0a3b00 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f721bfff640 10 monclient: get_auth_request con 0x7f721c0a3720 auth_method 0 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7220c78640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f721bfff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 0x7f721c13e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f7213fff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f720c001ec0 tx=0x7f720c00c960 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 client.?.objecter ms_handle_connect 0x7f721c0a3b00 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:23.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f720c017070 con 0x7f721c0a3b00 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: dump: 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.973 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: _finish_auth 0 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: _check_auth_tickets 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f720c007070 con 0x7f721c0a3b00 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 5 monclient: authenticate success, global_id 54391 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _renew_subs 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c13bcb0 con 0x7f721c0a3b00 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f720c007710 con 0x7f721c0a3b00 2024-09-17T16:37:23.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 monclient: dump: 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:23.975 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f721a7fc640 4 set_mon_vals no callback set 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 client.54391.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _renew_subs 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c1256a0 con 0x7f721c0a3b00 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 librados: init done 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f720c0072a0 con 0x7f721c0a3b00 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:23.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c0a1e70 con 0x7f721c0a3b00 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 0x7f71fc040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(378..378 src has 1..378) ==== 6202+0+0 (secure 0 0 0) 0x7f720c04f030 con 0x7f721c0a3b00 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 10 client.54391.objecter ms_dispatch 0x7f721c000ba0 osd_map(378..378 src has 1..378) 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 3 client.54391.objecter handle_osd_map got epochs [378,378] > 0 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.949+0000 7f72197fa640 3 client.54391.objecter handle_osd_map decoding full epoch 378 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.953+0000 7f72197fa640 20 client.54391.objecter dump_active .. 0 homeless 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.953+0000 7f721bfff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 0x7f71fc040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.953+0000 7f721bfff640 10 monclient: get_auth_request con 0x7f71fc03e5b0 auth_method 0 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.953+0000 7f72197fa640 10 client.54391.objecter ms_handle_connect 0x7f71fc03e5b0 2024-09-17T16:37:23.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.953+0000 7f721bfff640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 0x7f71fc040a70 secure :-1 s=READY pgs=3526 cs=0 l=1 rev1=1 crypto rx=0x7f72040ad000 tx=0x7f72040a4f90 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:23.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.957+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f721c0a5cd0 con 0x7f721c0a3b00 2024-09-17T16:37:23.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.957+0000 7f72197fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:23.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:23.957+0000 7f72197fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.189+0000 7f72226db640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:37:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.189+0000 7f72226db640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:37:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.189+0000 7f72226db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.189+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f721c140630 con 0x7f721c0a3b00 2024-09-17T16:37:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.193+0000 7f72197fa640 1 -- 172.21.15.105:0/3280728349 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f720c019d30 con 0x7f721c0a3b00 2024-09-17T16:37:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.193+0000 7f72197fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:37:24.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.193+0000 7f72197fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:24.203 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.203 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.203 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=11,kv=9,logm=66} crc {auth=3695829149,config=692619927,health=136006561,kv=2863082231,logm=1788388728}) 2024-09-17T16:37:24.203 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=2,health=10,kv=10,logm=65} crc {auth=1896023847,config=692619927,health=2038331059,kv=2863082231,logm=263428338}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=10,logm=64} crc {auth=1615980756,config=692619927,health=3397957174,kv=2863082231,logm=536176280}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=2890728830,config=692619927,health=3397957174,kv=2863082231,logm=1042223290}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=13,health=12,kv=10,logm=65} crc {auth=3131142063,health=3397957174,kv=2863082231,logm=3784237136}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.204 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=2890728830,config=692619927,health=3397957174,kv=2863082231,logm=2787487477}) 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=14,config=2,health=11,kv=10,logm=63} crc {auth=3216336269,config=692619927,health=136006561,kv=2863082231,logm=689169679}) 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=14,config=1,health=10,kv=9,logm=66} crc {auth=2890728830,config=3678701417,health=2573281241,kv=3449169938,logm=730092387}) 2024-09-17T16:37:24.205 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.199+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=953106980,config=692619927,health=3397957174,kv=3818608592,logm=2265708414}) 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 10 librados: watch_flush enter 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 10 librados: watch_flush exit 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 20 client.54391.objecter shutdown clearing up homeless session... 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 10 client.54391.objecter successfully canceled tick 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 msgr2=0x7f71fc040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 0x7f71fc040a70 secure :-1 s=READY pgs=3526 cs=0 l=1 rev1=1 crypto rx=0x7f72040ad000 tx=0x7f72040a4f90 comp rx=0 tx=0).stop 2024-09-17T16:37:24.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 10 monclient: shutdown 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 msgr2=0x7f721c13ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f720c001ec0 tx=0x7f720c00c960 comp rx=0 tx=0).stop 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 shutdown_connections 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f71fc03e5b0 0x7f71fc040a70 secure :-1 s=CLOSED pgs=3526 cs=0 l=1 rev1=1 crypto rx=0x7f72040ad000 tx=0x7f72040a4f90 comp rx=0 tx=0).stop 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f721c0a3f00 0x7f721c13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f721c0a3b00 0x7f721c13ec20 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f720c001ec0 tx=0x7f720c00c960 comp rx=0 tx=0).stop 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 --2- 172.21.15.105:0/3280728349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f721c0a3720 0x7f721c13e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 >> 172.21.15.105:0/3280728349 conn(0x7f721c0aa390 msgr2=0x7f721c0a8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 shutdown_connections 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 -- 172.21.15.105:0/3280728349 wait complete. 2024-09-17T16:37:24.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:24.197+0000 7f72226db640 1 librados: shutdown 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554624363}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1628405353}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2110653699}) 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.210 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1942809982}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4152936716}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=463837693}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2313420219}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=720131285}) 2024-09-17T16:37:24.211 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.207+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=835752348}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=950374450}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2903044119}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3638358834}) 2024-09-17T16:37:24.216 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2090288142}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1938483793}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1631777009}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=209816325}) 2024-09-17T16:37:24.217 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.218 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3015407268}) 2024-09-17T16:37:24.218 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.211+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4125033933}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2787216778}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3466640892}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3937790758}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.221 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3338831237}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3838808165}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2092800253}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3284170618}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476330963}) 2024-09-17T16:37:24.222 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.215+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=459510627}) 2024-09-17T16:37:24.223 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2844498843}) 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3102793942}) 2024-09-17T16:37:24.226 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580130670}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1875416407}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=5077290}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2474360181}) 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.227 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.228 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=737796393}) 2024-09-17T16:37:24.228 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.228 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999914868}) 2024-09-17T16:37:24.228 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.219+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165546967}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=644727812}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2521261528}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4224672138}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1743381538}) 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.231 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.223+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4000166240}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=435538288}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2850847731}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2138590008}) 2024-09-17T16:37:24.232 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1752766967}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.227+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2542634835}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1439518739}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.234 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=293796952}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562349536}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2480343593}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1843906105}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1505113804}) 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.235 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=505324780}) 2024-09-17T16:37:24.236 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.231+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2809420612}) 2024-09-17T16:37:24.240 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124580985}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933300331}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536650652}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4132198752}) 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.241 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2006984903}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1675537185}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1296298831}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1659523519}) 2024-09-17T16:37:24.242 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.235+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3670045887}) 2024-09-17T16:37:24.256 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.256 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.256 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1246449248}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=814686841}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2771992836}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1641385965}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4017961592}) 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.257 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2820851310}) 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1424759033}) 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331165334}) 2024-09-17T16:37:24.258 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.251+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1358805093}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=745914292}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3856321143}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=973476017}) 2024-09-17T16:37:24.266 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1988338651}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.259+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1261155825}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3569597149}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2963109601}) 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.267 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019968335}) 2024-09-17T16:37:24.268 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676172086}) 2024-09-17T16:37:24.269 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.269 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.269 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2118236358}) 2024-09-17T16:37:24.269 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.269 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2087889365}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=710227528}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3450188221}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=325674764}) 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.270 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.263+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1019012600}) 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=78096213}) 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4039711338}) 2024-09-17T16:37:24.271 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2814232749}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1632250794}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1226877403}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.273 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468694879}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1327132512}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3666731132}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018472567}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3675544509}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.267+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.271+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1468891875}) 2024-09-17T16:37:24.274 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.271+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1304628685}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=548519358}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196546402}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1074640747}) 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.278 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2004591540}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=298098036}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2361150370}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=944654949}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1726252748}) 2024-09-17T16:37:24.279 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=55229519}) 2024-09-17T16:37:24.282 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.275+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.282 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.282 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=558208139}) 2024-09-17T16:37:24.282 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.282 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054910461}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1418222326}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1402394469}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1338488175}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2216919285}) 2024-09-17T16:37:24.283 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.284 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.284 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=69599583}) 2024-09-17T16:37:24.284 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.284 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838647726}) 2024-09-17T16:37:24.284 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.279+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4097359541}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505908934}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605239504}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603872237}) 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.291 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3493588826}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1166175173}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3254370340}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2215146297}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342803404}) 2024-09-17T16:37:24.292 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.287+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=787818024}) 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3161059637}) 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3827375672}) 2024-09-17T16:37:24.295 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4169559709}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=299969898}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=146951121}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1999214174}) 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.296 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.297 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1181552627}) 2024-09-17T16:37:24.297 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.297 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559505503}) 2024-09-17T16:37:24.297 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.291+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2088137260}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646348589}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=474190231}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4106130491}) 2024-09-17T16:37:24.299 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2017889715}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1492716836}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2855434881}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3694918038}) 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.300 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=635589665}) 2024-09-17T16:37:24.301 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.295+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=260678747}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2603645903}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3322213766}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3116656252}) 2024-09-17T16:37:24.302 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431391418}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4257349773}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990554777}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=484900251}) 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.303 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4240602198}) 2024-09-17T16:37:24.304 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3914452100}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1861576277}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3894116301}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=5153114}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1396146306}) 2024-09-17T16:37:24.306 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1346244014}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3072692240}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1365067614}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859805968}) 2024-09-17T16:37:24.307 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.299+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2880271424}) 2024-09-17T16:37:24.308 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3289250887}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3934016204}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1068286332}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2433348801}) 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.309 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2243037116}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2828589069}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3578201965}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2864419733}) 2024-09-17T16:37:24.310 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.303+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2261582328}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4122209630}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2775294692}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=921985581}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.313 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1318741445}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2553938354}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4091413301}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4064557568}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321650627}) 2024-09-17T16:37:24.314 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.307+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3869384881}) 2024-09-17T16:37:24.317 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.317 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1628251588}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2219635452}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3350785580}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=69167864}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.318 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2109983179}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1411386721}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3972510821}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642472292}) 2024-09-17T16:37:24.319 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.311+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4144061265}) 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3263778224}) 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.321 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3804401726}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1837576997}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1737419574}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3148519117}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=667541721}) 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.322 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.323 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2547909325}) 2024-09-17T16:37:24.323 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.323 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2836448907}) 2024-09-17T16:37:24.323 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.315+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3724702350}) 2024-09-17T16:37:24.326 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923698445}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3930777305}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=893702261}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=723056818}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.327 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2628746364}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=944040956}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4207606747}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002399102}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.319+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=141719641}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2159813754}) 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.328 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2960923445}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=462835999}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3618620376}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3914175180}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948612168}) 2024-09-17T16:37:24.329 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.330 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.330 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3457905629}) 2024-09-17T16:37:24.330 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.330 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1090719322}) 2024-09-17T16:37:24.330 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2255028358}) 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1827586575}) 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2407044136}) 2024-09-17T16:37:24.333 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=433385792}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1490669899}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3569730975}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=566924807}) 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.334 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2424548594}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3376507256}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.323+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3104186844}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2136720832}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=474355489}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=374844323}) 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.335 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1787596578}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3094708128}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803423555}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2857989126}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390072899}) 2024-09-17T16:37:24.336 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.327+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2455490821}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380878332}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1454865108}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=518984965}) 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.342 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=619216371}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3714261093}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=14115387}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=977122678}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2989561210}) 2024-09-17T16:37:24.343 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.331+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=578400217}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722038703}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=425213276}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=667538605}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2858938603}) 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.344 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1913099031}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2138565417}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=864572008}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3712244390}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1663732837}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.345 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2071149841}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2275068114}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=309756114}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4063564645}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1212789993}) 2024-09-17T16:37:24.346 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3165545816}) 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=295827250}) 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1716788354}) 2024-09-17T16:37:24.347 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.335+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3671063941}) 2024-09-17T16:37:24.355 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.355 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.355 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3032476889}) 2024-09-17T16:37:24.355 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.355 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3023719768}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3862364375}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3752832453}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=146831721}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3569128849}) 2024-09-17T16:37:24.356 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1936732780}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144884289}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.339+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4077873869}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614554212}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1527378059}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.357 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=374072537}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3927980450}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2063796412}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=141728586}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=327217501}) 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.358 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=500788258}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2194301652}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774656085}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3426219787}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4222921163}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.359 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4108171529}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2689107571}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3446006723}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4125485600}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139869853}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.343+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=487922740}) 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.360 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3426433753}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1676565622}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=835962005}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=711872813}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1749644647}) 2024-09-17T16:37:24.361 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2102360581}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3332262271}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047169184}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3180276323}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087555562}) 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.362 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798225575}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1933206511}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=936375725}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=484625953}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4238855629}) 2024-09-17T16:37:24.363 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3517308258}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561709887}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.347+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=651031395}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780815069}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2139694559}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.364 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=997295402}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=833706503}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2777810513}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2683185592}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4221294503}) 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.365 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1800664180}) 2024-09-17T16:37:24.366 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1715890168}) 2024-09-17T16:37:24.383 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.383 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.351+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.383 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4080819634}) 2024-09-17T16:37:24.383 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.383 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432653947}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341656197}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2074751689}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1576712115}) 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.384 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1899116168}) 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2738834714}) 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.385 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741068535}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1233146085}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=725845184}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=892875249}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3670251113}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.386 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3353007623}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2576598966}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=646038756}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3720448902}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.355+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=176556514}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3128248584}) 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.387 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586070402}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2176183300}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3296815888}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3485073424}) 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.388 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1828425823}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1179916456}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2091872590}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=44524237}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.359+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=318070901}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.389 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2940396996}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=889492554}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2858928793}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1273521855}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2203785379}) 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.390 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=13652076}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=217143526}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2985615729}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.363+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3085395561}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=721547495}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1736620438}) 2024-09-17T16:37:24.391 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=46820971}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3393142936}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1458495621}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=601574428}) 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.392 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1497613406}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724921982}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421533285}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2096602547}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3931209460}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2714822460}) 2024-09-17T16:37:24.393 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2223013368}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1476396294}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1661890338}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3258013210}) 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.394 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=797601022}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.367+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=387162155}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1585176019}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2542950694}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3072929705}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2536292239}) 2024-09-17T16:37:24.395 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=495010788}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=868450659}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=322131983}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2752625181}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1927156962}) 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.396 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2210384618}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3364489368}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=590269508}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=479248306}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3034357412}) 2024-09-17T16:37:24.397 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2084571605}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3713986700}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140702992}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.371+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1354782152}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3985734190}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.398 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4020171151}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2885992621}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3559950554}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2108017529}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=886434929}) 2024-09-17T16:37:24.399 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2912599332}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=195207995}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2687230626}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4009576733}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853725194}) 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.400 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4270957275}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=578284712}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2148387676}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2456907915}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3333038751}) 2024-09-17T16:37:24.401 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2758074233}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.375+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1478774260}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2828882894}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2726842931}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1012175019}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=914287300}) 2024-09-17T16:37:24.402 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1686320694}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2903586469}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=783655809}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286346389}) 2024-09-17T16:37:24.403 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1826199055}) 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046227871}) 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2714432161}) 2024-09-17T16:37:24.448 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=214659750}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4184696729}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1954919268}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3218387750}) 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.449 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3805418382}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2329762196}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.379+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3078225962}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1016675426}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838841020}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.450 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=564075870}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2622363378}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3584567978}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1029144676}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=326369443}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154201116}) 2024-09-17T16:37:24.451 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2892896342}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604708554}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=813170136}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4187615199}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1565226799}) 2024-09-17T16:37:24.452 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=85560413}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2609264339}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3776814422}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3947857073}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.383+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=601437102}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.453 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493175204}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=216356616}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3829848066}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3268553362}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2034855111}) 2024-09-17T16:37:24.454 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=593757910}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3923009865}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=405875727}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1255903234}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1994516260}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.455 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=258116133}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3045392183}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1846957273}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=41129676}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1737087306}) 2024-09-17T16:37:24.456 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2381168032}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=198683090}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.387+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3288195833}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3734170673}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3293293348}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.457 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1305729658}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3629903299}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=87438868}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2976419663}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=574817429}) 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.458 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3639296374}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2237551617}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1101391139}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2660718592}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2907854927}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2917785110}) 2024-09-17T16:37:24.459 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2099987154}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1735701038}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=657476724}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163715070}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.391+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3881519104}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.460 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4187058698}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3214653295}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3018454885}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2013656917}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3333393510}) 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.461 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=612869001}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1778066545}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943659768}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2336941511}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3349919922}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=716900131}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.462 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2648924869}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1322201505}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=82140352}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3728901043}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=550978905}) 2024-09-17T16:37:24.463 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602417453}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.395+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4086538880}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1100317223}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1946419339}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=182853306}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.464 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3758971328}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2045686165}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1999454772}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2313414879}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=45633577}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=7823174}) 2024-09-17T16:37:24.465 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3911342829}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=596175037}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1244224176}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2453397219}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.466 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2072232408}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=535480333}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3903662101}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=821661894}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.399+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2326114871}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2723555613}) 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.467 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=667285494}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1221715613}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=892599476}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2958583383}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1928350782}) 2024-09-17T16:37:24.468 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1959887695}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371233655}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1009167488}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1749363381}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3050893496}) 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.469 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3846412589}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3476137680}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3598018164}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3480420926}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.470 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3974407965}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1032880337}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=659937911}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2576152447}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073640232}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2610197421}) 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.471 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2615181060}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1293391530}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3613193040}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=443100785}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669518799}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.403+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2491401982}) 2024-09-17T16:37:24.472 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2074492256}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3613691277}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1441586691}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4142249338}) 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.473 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=899944779}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3564072368}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2111994796}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218086517}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4136224194}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.474 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2894833295}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2624907804}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3913103348}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3567724340}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1836482145}) 2024-09-17T16:37:24.475 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3628340116}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1918665487}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1667198125}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.407+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=124294553}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867359037}) 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.476 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2151130319}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=374997116}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1342445734}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3082784887}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.477 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1589404406}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1730659291}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646487729}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1995555105}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725594700}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2826649540}) 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.478 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=59324211}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3731065166}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1621711002}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3557311627}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.479 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2032399313}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540901843}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.411+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428409270}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3845457746}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=567269162}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1935662450}) 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.480 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3006131652}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=84074151}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3461373261}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=456679978}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054456234}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3567410170}) 2024-09-17T16:37:24.481 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3180922850}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2653408370}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=550204067}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1450966191}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4010343354}) 2024-09-17T16:37:24.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1612450697}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4022219608}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1647296364}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.415+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3173483733}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=11} crc {logm=1519543206,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=4059254324,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1728204244,osd_snap=1614297645}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.483 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=40,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2828599447,mdsmap=1248375647,mgr=1348586770,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=346208105,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1728204244,osd_snap=464223838}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=44,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=12} crc {logm=3386706432,mdsmap=1248375647,mgr=1754239133,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_snap=3465228449}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=44,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=2625517870,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=432780167,mon_config_key=1190249571,monmap=1360384041,nvmeofgw=598337686,osd_metadata=1070247634,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=44,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=3190006169,mdsmap=3140514009,mgr=4234951994,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1105456853,mon_config_key=3381100223,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1728204244,osd_snap=2669840769}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=3917803424,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2288430746,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=1728204244,osd_snap=1193576899}) 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.484 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=46,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=9} crc {logm=2575648591,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=346208105,monmap=859292559,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=755866839}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=12} crc {logm=4210020604,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3925574058,mon_config_key=1734705363,monmap=835142563,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=1728204244,osd_snap=1521843086}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=44,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=12} crc {logm=1706590758,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=397939364,mon_config_key=1589589211,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=1728204244,osd_snap=3239896881}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=693691745,osdmap=4011305031}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=1204183605,osdmap=3000493449}) 2024-09-17T16:37:24.485 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=1943746139,osdmap=2123248364}) 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=345685481,osdmap=876165874}) 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.486 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=1204183605,osdmap=842205944}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=3088986355,osdmap=4240317230}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=21,osdmap=79} crc {osd_snap=1204183605,osdmap=2862520082}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=265568371,osdmap=1845805880}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.419+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=20,osdmap=80} crc {osd_snap=2940960704,osdmap=907446203}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3337795666}) 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.487 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2875441436}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3970514614}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2073648340}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3190383514}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=374724008}) 2024-09-17T16:37:24.488 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2745288235}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4056233271}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2604117232}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=233112714}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2006396482}) 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.489 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=115391105}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1309303345}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=207948666}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=201529125}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.490 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=998783959}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=237820309}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.423+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1987196690}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2139479129}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2362574502}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2087003710}) 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.491 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2912390562}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3468213708}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1300011402}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3338163534}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3813367381}) 2024-09-17T16:37:24.492 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1528063442}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1039267256}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.427+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=596702225}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=433021303}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2058995390}) 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.493 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=419712369}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3131873824}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=686269114}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3858276856}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.431+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=637757574}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3815310448}) 2024-09-17T16:37:24.494 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=80} crc {osdmap=2195952065}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=78} crc {osdmap=930916958}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=78} crc {osdmap=217734502}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=79} crc {osdmap=3294170420}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.495 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=80} crc {osdmap=726563120}) 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=77} crc {osdmap=2765212245}) 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=640382708}) 2024-09-17T16:37:24.496 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:37:24.439+0000 7fa54f46f640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=79} crc {osdmap=2520123422}) 2024-09-17T16:37:25.224 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:37:25.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:37:25.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: get_monmap_and_config 2024-09-17T16:37:25.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: monmap: 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:25.356570+0000 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:25.356570+0000 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.360 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:25.361 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 Processor -- start 2024-09-17T16:37:25.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 -- start start 2024-09-17T16:37:25.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: init 2024-09-17T16:37:25.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient: _add_conns ranks=[2,3,5,7,9,1,0,4,6,8,10] 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-b con 0x7f36a40a2de0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36a40a41d0 0x7f36a40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-g con 0x7f36a40a41d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f36a409b050 0x7f36a409b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-h con 0x7f36a409b050 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f36a409c730 con 0x7f36a40a2de0 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f36a409c9a0 con 0x7f36a40a41d0 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f36a40a2770 con 0x7f36a409b050 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36ab3cb640 10 monclient(hunting): authenticate will time out at 1528.623099s 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50280/0 (socket says 172.21.15.105:50280) 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 -- 172.21.15.105:0/3599697459 learned_addr learned my addr 172.21.15.105:0/3599697459 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): get_auth_request con 0x7f36a40a2de0 auth_method 0 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9968640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f36a409b050 0x7f36a409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a8966640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36a40a41d0 0x7f36a40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9968640 10 monclient(hunting): get_auth_request con 0x7f36a409b050 auth_method 0 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9968640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9968640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a8966640 10 monclient(hunting): get_auth_request con 0x7f36a40a41d0 auth_method 0 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a8966640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a8966640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a8966640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient(hunting): handle_auth_done global_id 44786 payload 274 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 -- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f36a409b050 msgr2=0x7f36a409b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f36a409b050 0x7f36a409b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 -- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36a40a41d0 msgr2=0x7f36a40a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36a40a41d0 0x7f36a40a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 monclient: found mon.noname-b 2024-09-17T16:37:25.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:25.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.353+0000 7f36a9167640 1 -- 172.21.15.105:0/3599697459 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a40a20b0 con 0x7f36a40a2de0 2024-09-17T16:37:25.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36a9167640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36a00098d0 tx=0x7f36a001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1bc245e87267f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 1 -- 172.21.15.105:0/3599697459 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36a00b0070 con 0x7f36a40a2de0 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: dump: 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.373 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 5 monclient: authenticate success, global_id 44786 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 1 -- 172.21.15.105:0/3599697459 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a00044a0 con 0x7f36a40a2de0 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 1 -- 172.21.15.105:0/3599697459 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36a0003a90 con 0x7f36a40a2de0 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 4 set_mon_vals no callback set 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f369bfff640 10 monclient: dump: 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.375 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 10 monclient: shutdown 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- 172.21.15.105:0/3599697459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 msgr2=0x7f36a40a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36a00098d0 tx=0x7f36a001c410 comp rx=0 tx=0).stop 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- 172.21.15.105:0/3599697459 shutdown_connections 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f36a409b050 0x7f36a409b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36a40a41d0 0x7f36a40a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/3599697459 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36a40a2de0 0x7f36a40a31c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- 172.21.15.105:0/3599697459 >> 172.21.15.105:0/3599697459 conn(0x7f36a40a83e0 msgr2=0x7f36a40a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- 172.21.15.105:0/3599697459 shutdown_connections 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- 172.21.15.105:0/3599697459 wait complete. 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 10 monclient: monmap: 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:25.377 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 librados: starting msgr at 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 librados: starting objecter 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 Processor -- start 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 -- start start 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 librados: setting wanted keys 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 1 librados: calling monclient init 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.357+0000 7f36ab3cb640 10 monclient: init 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _add_conns ranks=[0,6,1,5,4,2,8,3,7] 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-a con 0x7f36a409b050 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f36a40a2de0 0x7f36a409f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-d con 0x7f36a40a2de0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a40a41d0 0x7f36a409fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): picked mon.noname-f con 0x7f36a40a41d0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40a3dc0 con 0x7f36a409b050 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40a3c40 con 0x7f36a40a41d0 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f36a41411c0 con 0x7f36a40a2de0 2024-09-17T16:37:25.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient(hunting): authenticate will time out at 1528.628308s 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f36a40a2de0 0x7f36a409f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46278/0 (socket says 172.21.15.105:46278) 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 -- 172.21.15.105:0/2770073785 learned_addr learned my addr 172.21.15.105:0/2770073785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): get_auth_request con 0x7f36a409b050 auth_method 0 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): get_auth_request con 0x7f36a40a2de0 auth_method 0 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9968640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a40a41d0 0x7f36a409fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9968640 10 monclient(hunting): get_auth_request con 0x7f36a40a41d0 auth_method 0 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9968640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9968640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9968640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient(hunting): handle_auth_done global_id 35685 payload 824 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 -- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f36a40a2de0 msgr2=0x7f36a409f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f36a40a2de0 0x7f36a409f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 -- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a40a41d0 msgr2=0x7f36a409fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a40a41d0 0x7f36a409fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 monclient: found mon.noname-a 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 -- 172.21.15.105:0/2770073785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a413b150 con 0x7f36a409b050 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a9167640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c10 tx=0x7f36a00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 client.?.objecter ms_handle_connect 0x7f36a409b050 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36a00b0040 con 0x7f36a409b050 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: dump: 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 5 monclient: authenticate success, global_id 35685 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0003db0 con 0x7f36a409b050 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _renew_subs 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a40a14c0 con 0x7f36a409b050 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36a00ab8b0 con 0x7f36a409b050 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f369b7fe640 4 set_mon_vals no callback set 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 client.35685.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 monclient: dump: 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _renew_subs 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a413dac0 con 0x7f36a409b050 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 librados: init done 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f36a00ab440 con 0x7f36a409b050 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a40a62e0 con 0x7f36a409b050 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 0x7f3688040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(380..380 src has 1..380) ==== 5817+0+0 (secure 0 0 0) 0x7f36a00e9270 con 0x7f36a409b050 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 client.35685.objecter ms_dispatch 0x7f36a4000ba0 osd_map(380..380 src has 1..380) 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 3 client.35685.objecter handle_osd_map got epochs [380,380] > 0 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 3 client.35685.objecter handle_osd_map decoding full epoch 380 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 20 client.35685.objecter dump_active .. 0 homeless 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 0x7f3688040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 10 monclient: get_auth_request con 0x7f368803e580 auth_method 0 2024-09-17T16:37:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f3699ffb640 10 client.35685.objecter ms_handle_connect 0x7f368803e580 2024-09-17T16:37:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.361+0000 7f36a8966640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 0x7f3688040a40 secure :-1 s=READY pgs=3527 cs=0 l=1 rev1=1 crypto rx=0x7f36a4074b20 tx=0x7f3694008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.365+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36a0016150 con 0x7f36a409b050 2024-09-17T16:37:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.365+0000 7f3699ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.365+0000 7f3699ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:37:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:25.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f36a409b450 con 0x7f36a409b050 2024-09-17T16:37:25.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f3699ffb640 1 -- 172.21.15.105:0/2770073785 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f36a00aba50 con 0x7f36a409b050 2024-09-17T16:37:25.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f3699ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:25.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f3699ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:25.608 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":62,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:37:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 librados: watch_flush enter 2024-09-17T16:37:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 librados: watch_flush exit 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 20 client.35685.objecter shutdown clearing up homeless session... 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 client.35685.objecter successfully canceled tick 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 msgr2=0x7f3688040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 0x7f3688040a40 secure :-1 s=READY pgs=3527 cs=0 l=1 rev1=1 crypto rx=0x7f36a4074b20 tx=0x7f3694008040 comp rx=0 tx=0).stop 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 10 monclient: shutdown 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 msgr2=0x7f36a413cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c10 tx=0x7f36a00191b0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 shutdown_connections 2024-09-17T16:37:25.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f368803e580 0x7f3688040a40 unknown :-1 s=CLOSED pgs=3527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a40a41d0 0x7f36a409fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f36a40a2de0 0x7f36a409f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 --2- 172.21.15.105:0/2770073785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36a409b050 0x7f36a413cc00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 >> 172.21.15.105:0/2770073785 conn(0x7f36a40a83e0 msgr2=0x7f36a40a4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 shutdown_connections 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 -- 172.21.15.105:0/2770073785 wait complete. 2024-09-17T16:37:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.601+0000 7f36ab3cb640 1 librados: shutdown 2024-09-17T16:37:25.627 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:37:25.627 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:37:25.628 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:37:25.628 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:37:25.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: get_monmap_and_config 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: monmap: 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:25.744798+0000 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:25.744798+0000 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:25.748 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:25.749 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 Processor -- start 2024-09-17T16:37:25.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 -- start start 2024-09-17T16:37:25.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: init 2024-09-17T16:37:25.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient: _add_conns ranks=[0,8,10,4,9,6,3,5,2,1,7] 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ccc0a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-e con 0x7f3ccc09e740 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ccc09e0b0 0x7f3ccc13dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-f con 0x7f3ccc09e0b0 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ccc0a0a80 con 0x7f3ccc0a4750 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3ccc09ef60 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3ccc0a31c0 con 0x7f3ccc09e740 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd47e4640 10 monclient(hunting): authenticate will time out at 1529.010758s 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43498/0 (socket says 172.21.15.105:43498) 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/741952434 learned_addr learned my addr 172.21.15.105:0/741952434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request con 0x7f3ccc09e740 auth_method 0 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd2d81640 1 --2- 172.21.15.105:0/741952434 >> v2:172.21.15.160:3300/0 conn(0x7f3ccc09e0b0 0x7f3ccc13dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd2d81640 10 monclient(hunting): get_auth_request con 0x7f3ccc09e0b0 auth_method 0 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd2d81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd2d81640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd2d81640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_done global_id 54350 payload 274 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/741952434 >> v2:172.21.15.160:3300/0 conn(0x7f3ccc09e0b0 msgr2=0x7f3ccc13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/741952434 >> v2:172.21.15.160:3300/0 conn(0x7f3ccc09e0b0 0x7f3ccc13dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/741952434 >> v1:172.21.15.105:6789/0 conn(0x7f3ccc0a4750 legacy=0x7f3ccc0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 monclient: found mon.noname-e 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:25.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.741+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/741952434 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc0a6710 con 0x7f3ccc09e740 2024-09-17T16:37:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/741952434 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009870 tx=0x7f3cbc01c3c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=babf58a9e5b16be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 1 -- 172.21.15.105:0/741952434 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cbc0b0070 con 0x7f3ccc09e740 2024-09-17T16:37:25.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: dump: 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 1 -- 172.21.15.105:0/741952434 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0a4e50 con 0x7f3ccc09e740 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 5 monclient: authenticate success, global_id 54350 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 1 -- 172.21.15.105:0/741952434 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cbc0aac80 con 0x7f3ccc09e740 2024-09-17T16:37:25.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 4 set_mon_vals no callback set 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd157e640 10 monclient: dump: 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 10 monclient: shutdown 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- 172.21.15.105:0/741952434 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 msgr2=0x7f3ccc13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/741952434 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009870 tx=0x7f3cbc01c3c0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- 172.21.15.105:0/741952434 shutdown_connections 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/741952434 >> v2:172.21.15.160:3300/0 conn(0x7f3ccc09e0b0 0x7f3ccc13dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/741952434 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3ccc09e740 0x7f3ccc13b740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- 172.21.15.105:0/741952434 >> 172.21.15.105:0/741952434 conn(0x7f3ccc0a5840 msgr2=0x7f3ccc0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- 172.21.15.105:0/741952434 shutdown_connections 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- 172.21.15.105:0/741952434 wait complete. 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 10 monclient: monmap: 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:25.766 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 librados: starting msgr at 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 librados: starting objecter 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 Processor -- start 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 -- start start 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 librados: setting wanted keys 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 1 librados: calling monclient init 2024-09-17T16:37:25.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.745+0000 7f3cd47e4640 10 monclient: init 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _add_conns ranks=[4,5,1,8,3,6,0,2,7] 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 0x7f3ccc1362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-c con 0x7f3ccc09e0b0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ccc09e740 0x7f3ccc136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-h con 0x7f3ccc09e740 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ccc0a4750 0x7f3ccc120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): picked mon.noname-f con 0x7f3ccc0a4750 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc140d80 con 0x7f3ccc0a4750 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3ccc140c00 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f3ccc140f00 con 0x7f3ccc09e740 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): authenticate will time out at 1529.016165s 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 0x7f3ccc1362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 0x7f3ccc1362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51886/0 (socket says 172.21.15.105:51886) 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 -- 172.21.15.105:0/312482988 learned_addr learned my addr 172.21.15.105:0/312482988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): get_auth_request con 0x7f3ccc09e0b0 auth_method 0 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ccc0a4750 0x7f3ccc120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ccc09e740 0x7f3ccc136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 10 monclient(hunting): get_auth_request con 0x7f3ccc0a4750 auth_method 0 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request con 0x7f3ccc09e740 auth_method 0 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient(hunting): handle_auth_done global_id 54400 payload 824 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ccc09e740 msgr2=0x7f3ccc136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f3ccc09e740 0x7f3ccc136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ccc0a4750 msgr2=0x7f3ccc120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ccc0a4750 0x7f3ccc120150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 monclient: found mon.noname-c 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc0a4170 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ccc0a4750 0x7f3ccc120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2580640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 0x7f3ccc1362f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c3c0 tx=0x7f3cc400c880 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.?.objecter ms_handle_connect 0x7f3ccc09e0b0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cc4017070 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: dump: 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.771 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4003600 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 5 monclient: authenticate success, global_id 54400 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _renew_subs 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc13df90 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cc4003ca0 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 client.54400.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd0d7d640 4 set_mon_vals no callback set 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: dump: 2024-09-17T16:37:25.773 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _renew_subs 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc1236c0 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 librados: init done 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 0x7f3ccc02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc051e80 con 0x7f3ccc120e50 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 0x7f3ccc02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3cc4003830 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient: get_auth_request con 0x7f3ccc120e50 auth_method 0 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3cb003e550 0x7f3cb0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_done global_id 35703 payload 824 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(380..380 src has 1..380) ==== 5817+0+0 (secure 0 0 0) 0x7f3cc404ee90 con 0x7f3ccc09e0b0 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.54400.objecter ms_dispatch 0x7f3ccc000ba0 osd_map(380..380 src has 1..380) 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ccc123b30 con 0x7f3ccc120e50 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 3 client.54400.objecter handle_osd_map got epochs [380,380] > 0 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 3 client.54400.objecter handle_osd_map decoding full epoch 380 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 20 client.54400.objecter dump_active .. 0 homeless 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.54400.objecter ms_handle_connect 0x7f3ccc120e50 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.54400.objecter resend_mon_ops 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 0x7f3ccc02d450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc069ea0 tx=0x7f3cbc0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3cb003e550 0x7f3cb0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 10 monclient: get_auth_request con 0x7f3cb003e550 auth_method 0 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cbc0b0050 con 0x7f3ccc120e50 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: dump: 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.777 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3cbc0b1040 con 0x7f3ccc120e50 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 msgr2=0x7f3ccc02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 0x7f3ccc02d450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc069ea0 tx=0x7f3cbc0191c0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 msgr2=0x7f3ccc02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:25.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc120e50 0x7f3ccc02d450 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc069ea0 tx=0x7f3cbc0191c0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cc2ffd640 10 client.54400.objecter ms_handle_connect 0x7f3cb003e550 2024-09-17T16:37:25.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.749+0000 7f3cd2d81640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3cb003e550 0x7f3cb0040a10 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc09e4c0 tx=0x7f3cc800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:25.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 0x7f3ccc02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc0a4b30 con 0x7f3ccc121230 2024-09-17T16:37:25.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 0x7f3ccc02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient: get_auth_request con 0x7f3ccc121230 auth_method 0 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 10 monclient(hunting): handle_auth_done global_id 35712 payload 824 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 1 -- 172.21.15.105:0/312482988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3ccc0717e0 con 0x7f3ccc121230 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 client.54400.objecter ms_handle_connect 0x7f3ccc121230 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 client.54400.objecter resend_mon_ops 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd1d7f640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 0x7f3ccc02d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc01a050 tx=0x7f3cbc0a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cbc0abc20 con 0x7f3ccc121230 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 monclient: dump: 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.782 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f3cbc0c1070 con 0x7f3ccc121230 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 msgr2=0x7f3ccc02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 0x7f3ccc02d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc01a050 tx=0x7f3cbc0a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 msgr2=0x7f3ccc02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cc2ffd640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ccc121230 0x7f3ccc02d990 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc01a050 tx=0x7f3cbc0a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:25.783 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 436561, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:25.784 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.788 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.789 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.791 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:25.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:25.798 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.800 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.801 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:25.802 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.803 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.804 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.805 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:25.806 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 librados: watch_flush enter 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 librados: watch_flush exit 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 20 client.54400.objecter shutdown clearing up homeless session... 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 client.54400.objecter successfully canceled tick 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3cb003e550 msgr2=0x7f3cb0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3cb003e550 0x7f3cb0040a10 secure :-1 s=READY pgs=3528 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc09e4c0 tx=0x7f3cc800b040 comp rx=0 tx=0).stop 2024-09-17T16:37:25.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 10 monclient: shutdown 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 msgr2=0x7f3ccc1362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 --2- 172.21.15.105:0/312482988 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3ccc09e0b0 0x7f3ccc1362f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c3c0 tx=0x7f3cc400c880 comp rx=0 tx=0).stop 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd2d81640 1 -- 172.21.15.105:0/312482988 reap_dead start 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd2d81640 1 -- 172.21.15.105:0/312482988 reap_dead start 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 shutdown_connections 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 >> 172.21.15.105:0/312482988 conn(0x7f3ccc0a5840 msgr2=0x7f3ccc0a7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 shutdown_connections 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 -- 172.21.15.105:0/312482988 wait complete. 2024-09-17T16:37:25.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.757+0000 7f3cd47e4640 1 librados: shutdown 2024-09-17T16:37:25.810 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:25.810 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:37:25.810 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:37:25.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.917+0000 7f9e2f57c640 10 monclient: get_monmap_and_config 2024-09-17T16:37:25.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.917+0000 7f9e2f57c640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.917+0000 7f9e2f57c640 10 monclient: monmap: 2024-09-17T16:37:25.926 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:25.923544+0000 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:25.923544+0000 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.927 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:25.928 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 Processor -- start 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 -- start start 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient: init 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient: _add_conns ranks=[10,6,1,5,9,2,4,0,7,3,8] 2024-09-17T16:37:25.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 0x7f9e280a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-f con 0x7f9e280a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-d con 0x7f9e280a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9e281397f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e280a4080 con 0x7f9e281397f0 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9e280a7c40 con 0x7f9e280a6a30 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9e280a7890 con 0x7f9e280a4ab0 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2f57c640 10 monclient(hunting): authenticate will time out at 1529.189325s 2024-09-17T16:37:25.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:59402/0 (socket says 172.21.15.105:59402) 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 -- 172.21.15.105:0/3230899736 learned_addr learned my addr 172.21.15.105:0/3230899736 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): get_auth_request con 0x7f9e280a4ab0 auth_method 0 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2d318640 1 --2- 172.21.15.105:0/3230899736 >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 0x7f9e280a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_done global_id 24486 payload 274 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 -- 172.21.15.105:0/3230899736 >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 msgr2=0x7f9e280a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/3230899736 >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 0x7f9e280a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 -- 172.21.15.105:0/3230899736 >> v1:172.21.15.160:6789/0 conn(0x7f9e281397f0 legacy=0x7f9e28139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 monclient: found mon.noname-d 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2cb17640 1 -- 172.21.15.105:0/3230899736 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e280a37e0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2d318640 10 monclient: get_auth_request con 0x7f9e280a6a30 auth_method 0 2024-09-17T16:37:25.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.921+0000 7f9e2d318640 1 --2- 172.21.15.105:0/3230899736 >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 0x7f9e280a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/3230899736 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009870 tx=0x7f9e1801c3f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b0fe505d4ca5d645 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 1 -- 172.21.15.105:0/3230899736 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e180b0070 con 0x7f9e280a4ab0 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: dump: 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.941 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 1 -- 172.21.15.105:0/3230899736 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e180044a0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 1 -- 172.21.15.105:0/3230899736 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e18003a90 con 0x7f9e280a4ab0 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 5 monclient: authenticate success, global_id 24486 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 4 set_mon_vals no callback set 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1ffff640 10 monclient: dump: 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.943 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: shutdown 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/3230899736 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 msgr2=0x7f9e280a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/3230899736 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009870 tx=0x7f9e1801c3f0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/3230899736 shutdown_connections 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/3230899736 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9e280a4ab0 0x7f9e280a7350 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/3230899736 >> v2:172.21.15.160:3300/0 conn(0x7f9e280a6a30 0x7f9e280a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/3230899736 >> 172.21.15.105:0/3230899736 conn(0x7f9e280a8d00 msgr2=0x7f9e280a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/3230899736 shutdown_connections 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/3230899736 wait complete. 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: build_initial_monmap 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: monmap: 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:25.945 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 librados: starting msgr at 2024-09-17T16:37:25.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 librados: starting objecter 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 Processor -- start 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- start start 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 librados: setting wanted keys 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 librados: calling monclient init 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: init 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: _add_conns ranks=[2,5,0,8,1,3,6,7,4] 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 0x7f9e2813fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-b con 0x7f9e280a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9e280a6a30 0x7f9e281d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-h con 0x7f9e280a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 0x7f9e2813d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): picked mon.noname-a con 0x7f9e281397f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e28144270 con 0x7f9e281397f0 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2806e770 con 0x7f9e280a4ab0 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9e281443f0 con 0x7f9e280a6a30 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient(hunting): authenticate will time out at 1529.194998s 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 0x7f9e2813fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 0x7f9e2813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 0x7f9e2813fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50294/0 (socket says 172.21.15.105:50294) 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 -- 172.21.15.105:0/2815823538 learned_addr learned my addr 172.21.15.105:0/2815823538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:25.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): get_auth_request con 0x7f9e280a4ab0 auth_method 0 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 0x7f9e2813d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46304/0 (socket says 172.21.15.105:46304) 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): get_auth_request con 0x7f9e281397f0 auth_method 0 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9e280a6a30 0x7f9e281d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient(hunting): handle_auth_done global_id 44795 payload 824 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9e280a6a30 msgr2=0x7f9e281d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9e280a6a30 0x7f9e281d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 msgr2=0x7f9e2813d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 0x7f9e2813d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient: _finish_hunting 0 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 monclient: found mon.noname-b 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e280a9f90 con 0x7f9e280a4ab0 2024-09-17T16:37:25.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2db19640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e281397f0 0x7f9e2813d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2d318640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 0x7f9e2813fa60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c7f0 tx=0x7f9e2400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 client.?.objecter ms_handle_connect 0x7f9e280a4ab0 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e24007ca0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: dump: 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:25.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e240113f0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 5 monclient: authenticate success, global_id 44795 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: _renew_subs 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e28122e70 con 0x7f9e280a4ab0 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.925+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e24011a10 con 0x7f9e280a4ab0 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1f7fe640 4 set_mon_vals no callback set 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: dump: 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.953 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 10 client.44795.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:25.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 10 monclient: _renew_subs 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e281403f0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 1 librados: init done 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9e24011bb0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 0x7f9e2813d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9e28139610 con 0x7f9e28122a90 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 0x7f9e2813d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient: get_auth_request con 0x7f9e28122a90 auth_method 0 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9e0c03e550 0x7f9e0c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_done global_id 44804 payload 824 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(380..380 src has 1..380) ==== 5817+0+0 (secure 0 0 0) 0x7f9e2404ffe0 con 0x7f9e280a4ab0 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 client.44795.objecter ms_dispatch 0x7f9e28000ba0 osd_map(380..380 src has 1..380) 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e28139ca0 con 0x7f9e28122a90 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 3 client.44795.objecter handle_osd_map got epochs [380,380] > 0 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 3 client.44795.objecter handle_osd_map decoding full epoch 380 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 20 client.44795.objecter dump_active .. 0 homeless 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 client.44795.objecter ms_handle_connect 0x7f9e28122a90 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 client.44795.objecter resend_mon_ops 2024-09-17T16:37:25.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 0x7f9e2813d980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e2806ebb0 tx=0x7f9e18019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2db19640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9e0c03e550 0x7f9e0c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2db19640 10 monclient: get_auth_request con 0x7f9e0c03e550 auth_method 0 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e180b9030 con 0x7f9e28122a90 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: dump: 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.957 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9e180b1040 con 0x7f9e28122a90 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 msgr2=0x7f9e2813d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 0x7f9e2813d980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e2806ebb0 tx=0x7f9e18019190 comp rx=0 tx=0).stop 2024-09-17T16:37:25.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 msgr2=0x7f9e2813d980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:25.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e28122a90 0x7f9e2813d980 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e2806ebb0 tx=0x7f9e18019190 comp rx=0 tx=0).stop 2024-09-17T16:37:25.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e1dffb640 10 client.44795.objecter ms_handle_connect 0x7f9e0c03e550 2024-09-17T16:37:25.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.929+0000 7f9e2db19640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9e0c03e550 0x7f9e0c040a10 secure :-1 s=READY pgs=3529 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009fd0 tx=0x7f9e2000d040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:25.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 0x7f9e2813dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:25.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:25.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9e28051e80 con 0x7f9e281d1e00 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 0x7f9e2813dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient: get_auth_request con 0x7f9e281d1e00 auth_method 0 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 10 monclient(hunting): handle_auth_done global_id 44813 payload 824 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 1 -- 172.21.15.105:0/2815823538 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9e280008e0 con 0x7f9e281d1e00 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 client.44795.objecter ms_handle_connect 0x7f9e281d1e00 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 client.44795.objecter resend_mon_ops 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2cb17640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 0x7f9e2813dec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e1801c220 tx=0x7f9e180a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e180b9070 con 0x7f9e281d1e00 2024-09-17T16:37:25.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 monclient: dump: 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:25.962 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f9e180ab650 con 0x7f9e281d1e00 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 msgr2=0x7f9e2813dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 0x7f9e2813dec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e1801c220 tx=0x7f9e180a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 msgr2=0x7f9e2813dec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:25.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e1dffb640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e281d1e00 0x7f9e2813dec0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e1801c220 tx=0x7f9e180a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 296590, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:25.964 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.965 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:25.967 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:25.968 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:25.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:25.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:25.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:25.973 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:25.974 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:25.977 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:25.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:25.980 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:25.981 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:25.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:25.983 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.984 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:37:25.985 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:25.986 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:25.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:25.986 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:25.986 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 librados: watch_flush enter 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 librados: watch_flush exit 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 20 client.44795.objecter shutdown clearing up homeless session... 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 client.44795.objecter successfully canceled tick 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9e0c03e550 msgr2=0x7f9e0c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9e0c03e550 0x7f9e0c040a10 secure :-1 s=READY pgs=3529 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009fd0 tx=0x7f9e2000d040 comp rx=0 tx=0).stop 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 10 monclient: shutdown 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 msgr2=0x7f9e2813fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 --2- 172.21.15.105:0/2815823538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e280a4ab0 0x7f9e2813fa60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c7f0 tx=0x7f9e2400ccb0 comp rx=0 tx=0).stop 2024-09-17T16:37:25.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2db19640 1 -- 172.21.15.105:0/2815823538 reap_dead start 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2db19640 1 -- 172.21.15.105:0/2815823538 reap_dead start 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 shutdown_connections 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 >> 172.21.15.105:0/2815823538 conn(0x7f9e280a8d00 msgr2=0x7f9e280a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 shutdown_connections 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 -- 172.21.15.105:0/2815823538 wait complete. 2024-09-17T16:37:25.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:25.937+0000 7f9e2f57c640 1 librados: shutdown 2024-09-17T16:37:25.989 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:25.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:37:25.989 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: get_monmap_and_config 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: monmap: 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:26.100978+0000 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:26.100978+0000 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:26.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 Processor -- start 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 -- start start 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: init 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient: _add_conns ranks=[3,1,4,5,10,9,7,8,0,2,6] 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c0a3720 0x7fa24c0a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-g con 0x7fa24c0a3720 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa24c0a3b00 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-c con 0x7fa24c0a3f00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c06db50 con 0x7fa24c0a3b00 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0a0e80 con 0x7fa24c0a3720 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa24c0a1490 con 0x7fa24c0a3f00 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa251f19640 10 monclient(hunting): authenticate will time out at 1529.366654s 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51888/0 (socket says 172.21.15.105:51888) 2024-09-17T16:37:26.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 learned_addr learned my addr 172.21.15.105:0/3956335516 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): get_auth_request con 0x7fa24c0a3f00 auth_method 0 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c0a3720 0x7fa24c0a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request con 0x7fa24c0a3720 auth_method 0 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient(hunting): handle_auth_done global_id 54409 payload 274 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c0a3720 msgr2=0x7fa24c0a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c0a3720 0x7fa24c0a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> v1:172.21.15.160:6789/0 conn(0x7fa24c0a3b00 legacy=0x7fa24c0a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 monclient: found mon.noname-c 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:26.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.097+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0a7240 con 0x7fa24c0a3f00 2024-09-17T16:37:26.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.101+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 msgr2=0x7fa24c0a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:37:26.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.101+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 msgr2=0x7fa24c0a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:26.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.101+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fa2440098a0 tx=0x7fa24401c4d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:26.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.101+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:26.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient: get_auth_request con 0x7fa24c0a3f00 auth_method 0 2024-09-17T16:37:26.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 0 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: ms_handle_reset current mon [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 msgr2=0x7fa24c0a5dd0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0a3f00 0x7fa24c0a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: _add_conns ranks=[7,5,9,6,10,8,2,0] 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa238003640 0x7fa238005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): picked mon.noname-i con 0x7fa238003640 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa238006350 0x7fa2380087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): picked mon.noname-h con 0x7fa238006350 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 --2- 172.21.15.105:0/3956335516 >> v2:172.21.15.105:3300/0 conn(0x7fa2380090f0 0x7fa23800b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): picked mon.noname-a con 0x7fa2380090f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa23800bec0 con 0x7fa2380090f0 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa23800c040 con 0x7fa238006350 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa23800c240 con 0x7fa238003640 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> v2:172.21.15.105:3300/0 conn(0x7fa2380090f0 0x7fa23800b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): get_auth_request con 0x7fa2380090f0 auth_method 0 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa238006350 0x7fa2380087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): get_auth_request con 0x7fa238006350 auth_method 0 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa238003640 0x7fa238005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request con 0x7fa238003640 auth_method 0 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient(hunting): handle_auth_done global_id 54409 payload 293 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa238003640 msgr2=0x7fa238005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa238003640 0x7fa238005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa238006350 msgr2=0x7fa2380087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa238006350 0x7fa2380087e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 monclient: found mon.noname-a 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23800c500 con 0x7fa2380090f0 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa238003640 0x7fa238005a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 -- 172.21.15.105:0/3956335516 reap_dead start 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2380090f0 0x7fa23800b5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0a1080 tx=0x7fa244009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84efb96c993072c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2440ab5e0 con 0x7fa2380090f0 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: dump: 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa251f19640 5 monclient: authenticate success, global_id 54409 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2440abdf0 con 0x7fa2380090f0 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa251f19640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa251f19640 4 set_mon_vals no callback set 2024-09-17T16:37:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 1 -- 172.21.15.105:0/3956335516 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2440af8b0 con 0x7fa2380090f0 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.305+0000 7fa24a7fc640 10 monclient: dump: 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.322 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: shutdown 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2380090f0 msgr2=0x7fa23800b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2380090f0 0x7fa23800b5b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0a1080 tx=0x7fa244009240 comp rx=0 tx=0).stop 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- 172.21.15.105:0/3956335516 shutdown_connections 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 --2- 172.21.15.105:0/3956335516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2380090f0 0x7fa23800b5b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- 172.21.15.105:0/3956335516 >> 172.21.15.105:0/3956335516 conn(0x7fa24c0aa390 msgr2=0x7fa24c0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- 172.21.15.105:0/3956335516 shutdown_connections 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- 172.21.15.105:0/3956335516 wait complete. 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: monmap: 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.330 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 librados: starting msgr at 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 librados: starting objecter 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 Processor -- start 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- start start 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 librados: setting wanted keys 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 librados: calling monclient init 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: init 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient: _add_conns ranks=[7,1,3,5,6,2,8,4,0] 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-i con 0x7fa2380090f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa24c13bca0 0x7fa24c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-f con 0x7fa24c13bca0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c13f180 0x7fa24c13f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): picked mon.noname-g con 0x7fa24c13f180 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0a2200 con 0x7fa24c13bca0 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0a2080 con 0x7fa24c13f180 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa24c0a6520 con 0x7fa2380090f0 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa251f19640 10 monclient(hunting): authenticate will time out at 1529.579119s 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa24c13bca0 0x7fa24c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:46890/0 (socket says 172.21.15.105:46890) 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 1 -- 172.21.15.105:0/2028842062 learned_addr learned my addr 172.21.15.105:0/2028842062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request con 0x7fa2380090f0 auth_method 0 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): get_auth_request con 0x7fa24c13bca0 auth_method 0 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c13f180 0x7fa24c13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): get_auth_request con 0x7fa24c13f180 auth_method 0 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.309+0000 7fa24bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 10 monclient(hunting): handle_auth_done global_id 64186 payload 824 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c13f180 msgr2=0x7fa24c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa24c13f180 0x7fa24c13f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa24c13bca0 msgr2=0x7fa24c13ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa24c13bca0 0x7fa24c13ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 monclient: found mon.noname-i 2024-09-17T16:37:26.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c124e20 con 0x7fa2380090f0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24b7fe640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009d10 tx=0x7fa23c007700 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.?.objecter ms_handle_connect 0x7fa2380090f0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa23c007e10 con 0x7fa2380090f0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: dump: 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.336 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c019070 con 0x7fa2380090f0 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 5 monclient: authenticate success, global_id 64186 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 monclient: _renew_subs 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c1262c0 con 0x7fa2380090f0 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa23c014cf0 con 0x7fa2380090f0 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa249ffb640 4 set_mon_vals no callback set 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 client.64186.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: dump: 2024-09-17T16:37:26.338 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.339 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 monclient: _renew_subs 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c0a7c40 con 0x7fa2380090f0 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 1 librados: init done 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 0x7fa24c071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa24c13c080 con 0x7fa24c13fb40 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 0x7fa24c071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient: get_auth_request con 0x7fa24c13fb40 auth_method 0 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa23c014410 con 0x7fa2380090f0 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 10 monclient(hunting): handle_auth_done global_id 54418 payload 824 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa22c03e580 0x7fa22c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa24c0a8170 con 0x7fa24c13fb40 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7fa23c0505a0 con 0x7fa2380090f0 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.64186.objecter ms_dispatch 0x7fa24c000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 3 client.64186.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 3 client.64186.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 20 client.64186.objecter dump_active .. 0 homeless 2024-09-17T16:37:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24affd640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 0x7fa24c071090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa234007fd0 tx=0x7fa234012ac0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.64186.objecter ms_handle_connect 0x7fa24c13fb40 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.64186.objecter resend_mon_ops 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24bfff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa22c03e580 0x7fa22c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24bfff640 10 monclient: get_auth_request con 0x7fa22c03e580 auth_method 0 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa234019020 con 0x7fa24c13fb40 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: dump: 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.342 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa2340053a0 con 0x7fa24c13fb40 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 msgr2=0x7fa24c071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 0x7fa24c071090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa234007fd0 tx=0x7fa234012ac0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 msgr2=0x7fa24c071090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c13fb40 0x7fa24c071090 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa234007fd0 tx=0x7fa234012ac0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa243fff640 10 client.64186.objecter ms_handle_connect 0x7fa22c03e580 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.313+0000 7fa24bfff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa22c03e580 0x7fa22c040a40 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7fa238003610 tx=0x7fa24401c4d0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 0x7fa24c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa24c051e80 con 0x7fa24c0715d0 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 0x7fa24c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient: get_auth_request con 0x7fa24c0715d0 auth_method 0 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 10 monclient(hunting): handle_auth_done global_id 54427 payload 824 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 1 -- 172.21.15.105:0/2028842062 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa24c0a56c0 con 0x7fa24c0715d0 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24affd640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 0x7fa24c0719e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa234009010 tx=0x7fa234001130 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 client.64186.objecter ms_handle_connect 0x7fa24c0715d0 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 client.64186.objecter resend_mon_ops 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa234019040 con 0x7fa24c0715d0 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 monclient: dump: 2024-09-17T16:37:26.345 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.346 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7fa2340144b0 con 0x7fa24c0715d0 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 msgr2=0x7fa24c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 0x7fa24c0719e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa234009010 tx=0x7fa234001130 comp rx=0 tx=0).stop 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 msgr2=0x7fa24c0719e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa243fff640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa24c0715d0 0x7fa24c0719e0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa234009010 tx=0x7fa234001130 comp rx=0 tx=0).stop 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:26.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 153319, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:26.348 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.349 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.350 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.351 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:26.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.357 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.359 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.364 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:26.365 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.366 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.367 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.368 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.369 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:26.370 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 librados: watch_flush enter 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 librados: watch_flush exit 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 20 client.64186.objecter shutdown clearing up homeless session... 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 client.64186.objecter successfully canceled tick 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa22c03e580 msgr2=0x7fa22c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa22c03e580 0x7fa22c040a40 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7fa238003610 tx=0x7fa24401c4d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 10 monclient: shutdown 2024-09-17T16:37:26.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 msgr2=0x7fa24c13b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009d10 tx=0x7fa23c007700 comp rx=0 tx=0).stop 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa24bfff640 1 -- 172.21.15.105:0/2028842062 reap_dead start 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 shutdown_connections 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 --2- 172.21.15.105:0/2028842062 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa2380090f0 0x7fa24c13b760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 >> 172.21.15.105:0/2028842062 conn(0x7fa24c0aa390 msgr2=0x7fa24c0a04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 shutdown_connections 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 -- 172.21.15.105:0/2028842062 wait complete. 2024-09-17T16:37:26.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.321+0000 7fa251f19640 1 librados: shutdown 2024-09-17T16:37:26.373 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:26.374 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:37:26.374 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: get_monmap_and_config 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: monmap: 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:26.495972+0000 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:26.495972+0000 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:26.500 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 Processor -- start 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 -- start start 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: init 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient: _add_conns ranks=[10,5,3,9,8,4,6,2,1,0,7] 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02480a4750 0x7f02480a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-f con 0x7f02480a4750 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02480a5070 0x7f0248137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-h con 0x7f02480a5070 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f0248137e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-g con 0x7f0248137a50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f02481383a0 con 0x7f02480a4750 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f02480a0ee0 con 0x7f0248137a50 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f024806cf10 con 0x7f02480a5070 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024f48f640 10 monclient(hunting): authenticate will time out at 1529.761955s 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02480a4750 0x7f02480a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02480a4750 0x7f02480a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:34568/0 (socket says 172.21.15.105:34568) 2024-09-17T16:37:26.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 -- 172.21.15.105:0/3036993723 learned_addr learned my addr 172.21.15.105:0/3036993723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): get_auth_request con 0x7f02480a4750 auth_method 0 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f0248137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02480a5070 0x7f0248137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): get_auth_request con 0x7f0248137a50 auth_method 0 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): get_auth_request con 0x7f02480a5070 auth_method 0 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024da2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient(hunting): handle_auth_done global_id 54388 payload 274 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 -- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02480a5070 msgr2=0x7f0248137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02480a5070 0x7f0248137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 -- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 msgr2=0x7f0248137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f0248137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 monclient: found mon.noname-f 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.493+0000 7f024d22b640 1 -- 172.21.15.105:0/3036993723 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024806c370 con 0x7f02480a4750 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024d22b640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02480a4b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f024001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c6d97004c2e41a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 1 -- 172.21.15.105:0/3036993723 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02400b0070 con 0x7f02480a4750 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: dump: 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.508 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.509 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 1 -- 172.21.15.105:0/3036993723 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02400aa810 con 0x7f02480a4750 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 1 -- 172.21.15.105:0/3036993723 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02400a9300 con 0x7f02480a4750 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 5 monclient: authenticate success, global_id 54388 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 4 set_mon_vals no callback set 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f023ffff640 10 monclient: dump: 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.510 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.511 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 10 monclient: shutdown 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 msgr2=0x7f02480a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02480a4b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f024001c400 comp rx=0 tx=0).stop 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- 172.21.15.105:0/3036993723 shutdown_connections 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f0248137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02480a5070 0x7f0248137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 --2- 172.21.15.105:0/3036993723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02480a4b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- 172.21.15.105:0/3036993723 >> 172.21.15.105:0/3036993723 conn(0x7f02480a5840 msgr2=0x7f02480a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- 172.21.15.105:0/3036993723 shutdown_connections 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- 172.21.15.105:0/3036993723 wait complete. 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 10 monclient: monmap: 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 librados: starting msgr at 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 librados: starting objecter 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 Processor -- start 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 -- start start 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 librados: setting wanted keys 2024-09-17T16:37:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 1 librados: calling monclient init 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.497+0000 7f024f48f640 10 monclient: init 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _add_conns ranks=[1,0,3,2,6,5,4,8,7] 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02481d1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-f con 0x7f02480a4750 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 0x7f02481d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-a con 0x7f02480a5070 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f024813c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): picked mon.noname-g con 0x7f0248137a50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02480a5450 con 0x7f02480a5070 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f02480a11c0 con 0x7f02480a4750 2024-09-17T16:37:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f02481336d0 con 0x7f0248137a50 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): authenticate will time out at 1529.768380s 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 0x7f02481d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 0x7f02481d16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46326/0 (socket says 172.21.15.105:46326) 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 -- 172.21.15.105:0/3327068941 learned_addr learned my addr 172.21.15.105:0/3327068941 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): get_auth_request con 0x7f02480a5070 auth_method 0 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02481d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f024813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): get_auth_request con 0x7f02480a4750 auth_method 0 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 10 monclient(hunting): get_auth_request con 0x7f0248137a50 auth_method 0 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient(hunting): handle_auth_done global_id 35730 payload 824 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 msgr2=0x7f024813c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0248137a50 0x7f024813c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 msgr2=0x7f02481d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02480a4750 0x7f02481d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 monclient: found mon.noname-a 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0248138040 con 0x7f02480a5070 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024ca2a640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 0x7f02481d16d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0244009db0 tx=0x7f024400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.?.objecter ms_handle_connect 0x7f02480a5070 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0244018020 con 0x7f02480a5070 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: dump: 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024400cdb0 con 0x7f02480a5070 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0244014d60 con 0x7f02480a5070 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: dump: 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023f7fe640 4 set_mon_vals no callback set 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 5 monclient: authenticate success, global_id 35730 2024-09-17T16:37:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _renew_subs 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02480a4d30 con 0x7f02480a5070 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 client.35730.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _renew_subs 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024809f700 con 0x7f02480a5070 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 librados: init done 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 0x7f024813ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f024809f8d0 con 0x7f02481d1c10 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 0x7f024813ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient: get_auth_request con 0x7f02481d1c10 auth_method 0 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f024401b020 con 0x7f02480a5070 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 10 monclient(hunting): handle_auth_done global_id 24495 payload 824 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0248132160 con 0x7f02481d1c10 2024-09-17T16:37:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f023403e550 0x7f0234040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7f0244050890 con 0x7f02480a5070 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.35730.objecter ms_dispatch 0x7f0248000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 3 client.35730.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 3 client.35730.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024d22b640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 0x7f024813ce70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0248069ea0 tx=0x7f02400191a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 20 client.35730.objecter dump_active .. 0 homeless 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.35730.objecter ms_handle_connect 0x7f02481d1c10 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.35730.objecter resend_mon_ops 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f023403e550 0x7f0234040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 10 monclient: get_auth_request con 0x7f023403e550 auth_method 0 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02400b9030 con 0x7f02481d1c10 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: dump: 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f02400b1040 con 0x7f02481d1c10 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 msgr2=0x7f024813ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 0x7f024813ce70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0248069ea0 tx=0x7f02400191a0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 msgr2=0x7f024813ce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f02481d1c10 0x7f024813ce70 secure :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0248069ea0 tx=0x7f02400191a0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f023dffb640 10 client.35730.objecter ms_handle_connect 0x7f023403e550 2024-09-17T16:37:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.501+0000 7f024da2c640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f023403e550 0x7f0234040a10 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7f024809e900 tx=0x7f0238008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 0x7f02480700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f024813d7c0 con 0x7f024813d3b0 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 0x7f02480700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient: get_auth_request con 0x7f024813d3b0 auth_method 0 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 10 monclient(hunting): handle_auth_done global_id 24504 payload 824 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 1 -- 172.21.15.105:0/3327068941 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0248137510 con 0x7f024813d3b0 2024-09-17T16:37:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 client.35730.objecter ms_handle_connect 0x7f024813d3b0 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 client.35730.objecter resend_mon_ops 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024d22b640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 0x7f02480700d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f02400a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02400b9070 con 0x7f024813d3b0 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 monclient: dump: 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f02400b5400 con 0x7f024813d3b0 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 msgr2=0x7f02480700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 0x7f02480700d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f02400a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 msgr2=0x7f02480700d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f023dffb640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f024813d3b0 0x7f02480700d0 secure :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f02400a8000 comp rx=0 tx=0).stop 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 594148, 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:26.538 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.539 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:26.540 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.541 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.543 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:26.545 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.547 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.548 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:26.549 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.551 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.552 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:26.553 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.554 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.555 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.556 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.557 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.558 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.559 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:26.559 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:26.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 librados: watch_flush enter 2024-09-17T16:37:26.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 librados: watch_flush exit 2024-09-17T16:37:26.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 20 client.35730.objecter shutdown clearing up homeless session... 2024-09-17T16:37:26.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 client.35730.objecter successfully canceled tick 2024-09-17T16:37:26.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f023403e550 msgr2=0x7f0234040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f023403e550 0x7f0234040a10 secure :-1 s=READY pgs=3531 cs=0 l=1 rev1=1 crypto rx=0x7f024809e900 tx=0x7f0238008040 comp rx=0 tx=0).stop 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 10 monclient: shutdown 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 msgr2=0x7f02481d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 --2- 172.21.15.105:0/3327068941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02480a5070 0x7f02481d16d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0244009db0 tx=0x7f024400c490 comp rx=0 tx=0).stop 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024da2c640 1 -- 172.21.15.105:0/3327068941 reap_dead start 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024da2c640 1 -- 172.21.15.105:0/3327068941 reap_dead start 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 shutdown_connections 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 >> 172.21.15.105:0/3327068941 conn(0x7f02480a5840 msgr2=0x7f02480a6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 shutdown_connections 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 -- 172.21.15.105:0/3327068941 wait complete. 2024-09-17T16:37:26.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.509+0000 7f024f48f640 1 librados: shutdown 2024-09-17T16:37:26.562 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:26.562 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:37:26.562 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:37:26.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 10 monclient: get_monmap_and_config 2024-09-17T16:37:26.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 10 monclient: monmap: 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:26.682544+0000 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:26.682544+0000 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.686 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.687 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.687 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.687 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:26.687 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:26.687 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 1 Processor -- start 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 1 -- start start 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.677+0000 7fa80066e640 10 monclient: init 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient: _add_conns ranks=[2,3,5,0,6,9,1,7,8,10,4] 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-b con 0x7fa7f80a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f80a6cc0 0x7fa7f80a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-g con 0x7fa7f80a6cc0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:26.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 0x7fa7f8139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-h con 0x7fa7f81396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f80a2ea0 con 0x7fa7f80a68e0 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f8069ab0 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa7f80a87d0 con 0x7fa7f81396a0 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient(hunting): authenticate will time out at 1529.948424s 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50322/0 (socket says 172.21.15.105:50322) 2024-09-17T16:37:26.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3516136835 learned_addr learned my addr 172.21.15.105:0/3516136835 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): get_auth_request con 0x7fa7f80a68e0 auth_method 0 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f80a6cc0 0x7fa7f80a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fec0b640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 0x7fa7f8139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request con 0x7fa7f80a6cc0 auth_method 0 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fdc09640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fdc09640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_done global_id 44822 payload 274 2024-09-17T16:37:26.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 msgr2=0x7fa7f8139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 0x7fa7f8139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f80a6cc0 msgr2=0x7fa7f80a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f80a6cc0 0x7fa7f80a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 monclient: found mon.noname-b 2024-09-17T16:37:26.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3516136835 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f80a7b80 con 0x7fa7f80a68e0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fec0b640 10 monclient: get_auth_request con 0x7fa7f81396a0 auth_method 0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fec0b640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 0x7fa7f8139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80098a0 tx=0x7fa7e801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98aa587ecc88f9da server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 1 -- 172.21.15.105:0/3516136835 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b0070 con 0x7fa7f80a68e0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: dump: 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.695 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 1 -- 172.21.15.105:0/3516136835 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e80aa810 con 0x7fa7f80a68e0 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 5 monclient: authenticate success, global_id 44822 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 1 -- 172.21.15.105:0/3516136835 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80a9300 con 0x7fa7f80a68e0 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 4 set_mon_vals no callback set 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa7fd408640 10 monclient: dump: 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.697 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 10 monclient: shutdown 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 -- 172.21.15.105:0/3516136835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 msgr2=0x7fa7f80a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80098a0 tx=0x7fa7e801c3b0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 -- 172.21.15.105:0/3516136835 shutdown_connections 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f81396a0 0x7fa7f8139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f80a6cc0 0x7fa7f80a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.681+0000 7fa80066e640 1 --2- 172.21.15.105:0/3516136835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f80a68e0 0x7fa7f80a48a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- 172.21.15.105:0/3516136835 >> 172.21.15.105:0/3516136835 conn(0x7fa7f80a96e0 msgr2=0x7fa7f8068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- 172.21.15.105:0/3516136835 shutdown_connections 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- 172.21.15.105:0/3516136835 wait complete. 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: monmap: 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:26.699 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.700 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 librados: starting msgr at 2024-09-17T16:37:26.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 librados: starting objecter 2024-09-17T16:37:26.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 Processor -- start 2024-09-17T16:37:26.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- start start 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 librados: setting wanted keys 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 librados: calling monclient init 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: init 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _add_conns ranks=[6,0,4,8,7,1,5,3,2] 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 0x7fa7f8136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-d con 0x7fa7f80a6cc0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f81396a0 0x7fa7f8136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-a con 0x7fa7f81396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7f813b810 0x7fa7f813bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): picked mon.noname-c con 0x7fa7f813b810 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f80a75e0 con 0x7fa7f81396a0 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa7f8069500 con 0x7fa7f813b810 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa7f80a4b10 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient(hunting): authenticate will time out at 1529.954057s 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 0x7fa7f8136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f81396a0 0x7fa7f8136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 0x7fa7f8136140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:59426/0 (socket says 172.21.15.105:59426) 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3871402201 learned_addr learned my addr 172.21.15.105:0/3871402201 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): get_auth_request con 0x7fa7f80a6cc0 auth_method 0 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request con 0x7fa7f81396a0 auth_method 0 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7f813b810 0x7fa7f813bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): get_auth_request con 0x7fa7f813b810 auth_method 0 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fec0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient(hunting): handle_auth_done global_id 24513 payload 824 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7f813b810 msgr2=0x7fa7f813bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7f813b810 0x7fa7f813bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f81396a0 msgr2=0x7fa7f8136680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f81396a0 0x7fa7f8136680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 monclient: found mon.noname-d 2024-09-17T16:37:26.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f813f380 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f81396a0 0x7fa7f8136680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fe40a640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 0x7fa7f8136140 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80097c0 tx=0x7fa7e80191b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 client.?.objecter ms_handle_connect 0x7fa7f80a6cc0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b9070 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: dump: 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e80044b0 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b0040 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7f6ffd640 10 monclient: dump: 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.714 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa7fcc07640 4 set_mon_vals no callback set 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 5 monclient: authenticate success, global_id 24513 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _renew_subs 2024-09-17T16:37:26.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f8139db0 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 client.24513.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _renew_subs 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f8137370 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.685+0000 7fa80066e640 1 librados: init done 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa80066e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa80066e640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 0x7fa7f813c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa7f80a7760 con 0x7fa7f80a96e0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 0x7fa7f813c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient: get_auth_request con 0x7fa7f80a96e0 auth_method 0 2024-09-17T16:37:26.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa7e8004040 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_done global_id 54359 payload 824 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7f813ef20 con 0x7fa7f80a96e0 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7e403e550 0x7fa7e4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7fa7e80e9060 con 0x7fa7f80a6cc0 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 client.24513.objecter ms_dispatch 0x7fa7f8000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 3 client.24513.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:26.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 3 client.24513.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 0x7fa7f813c1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec00e1d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 20 client.24513.objecter dump_active .. 0 homeless 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 client.24513.objecter ms_handle_connect 0x7fa7f80a96e0 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 client.24513.objecter resend_mon_ops 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7ec018020 con 0x7fa7f80a96e0 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 monclient: dump: 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.718 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fec0b640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7e403e550 0x7fa7e4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fec0b640 10 monclient: get_auth_request con 0x7fa7e403e550 auth_method 0 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa7ec00ed40 con 0x7fa7f80a96e0 2024-09-17T16:37:26.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 msgr2=0x7fa7f813c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 0x7fa7f813c1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec00e1d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 msgr2=0x7fa7f813c1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f80a96e0 0x7fa7f813c1b0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec00e1d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7f6ffd640 10 client.24513.objecter ms_handle_connect 0x7fa7e403e550 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.689+0000 7fa7fec0b640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7e403e550 0x7fa7e4040a10 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80a7a80 tx=0x7fa7f0007700 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa80066e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa80066e640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 0x7fa7f809fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa80066e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa7f8051e80 con 0x7fa7f813c6f0 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 0x7fa7f809fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient: get_auth_request con 0x7fa7f813c6f0 auth_method 0 2024-09-17T16:37:26.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 10 monclient(hunting): handle_auth_done global_id 54368 payload 824 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.693+0000 7fa7fdc09640 1 -- 172.21.15.105:0/3871402201 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa7f80a70a0 con 0x7fa7f813c6f0 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 client.24513.objecter ms_handle_connect 0x7fa7f813c6f0 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7fdc09640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 0x7fa7f809fa10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 client.24513.objecter resend_mon_ops 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7ec018040 con 0x7fa7f813c6f0 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 monclient: dump: 2024-09-17T16:37:26.721 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fa7ec003b90 con 0x7fa7f813c6f0 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 msgr2=0x7fa7f809fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 0x7fa7f809fa10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec0046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 msgr2=0x7fa7f809fa10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7f6ffd640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa7f813c6f0 0x7fa7f809fa10 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7f813cf70 tx=0x7fa7ec0046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 153695, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.728 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:26.729 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:26.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:26.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.737 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.738 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.740 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.743 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:26.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:26.745 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 10 librados: watch_flush enter 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 10 librados: watch_flush exit 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 20 client.24513.objecter shutdown clearing up homeless session... 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 10 client.24513.objecter successfully canceled tick 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7e403e550 msgr2=0x7fa7e4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7e403e550 0x7fa7e4040a10 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80a7a80 tx=0x7fa7f0007700 comp rx=0 tx=0).stop 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 10 monclient: shutdown 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 msgr2=0x7fa7f8136140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 --2- 172.21.15.105:0/3871402201 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7f80a6cc0 0x7fa7f8136140 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80097c0 tx=0x7fa7e80191b0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7fec0b640 1 -- 172.21.15.105:0/3871402201 reap_dead start 2024-09-17T16:37:26.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa7fec0b640 1 -- 172.21.15.105:0/3871402201 reap_dead start 2024-09-17T16:37:26.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 shutdown_connections 2024-09-17T16:37:26.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 >> 172.21.15.105:0/3871402201 conn(0x7fa7f80a68e0 msgr2=0x7fa7f80a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 shutdown_connections 2024-09-17T16:37:26.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 -- 172.21.15.105:0/3871402201 wait complete. 2024-09-17T16:37:26.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.697+0000 7fa80066e640 1 librados: shutdown 2024-09-17T16:37:26.748 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:26.748 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:37:26.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:37:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: get_monmap_and_config 2024-09-17T16:37:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: monmap: 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:26.864460+0000 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:26.864460+0000 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:26.869 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:26.869 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 Processor -- start 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 -- start start 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: init 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient: _add_conns ranks=[10,2,4,8,9,3,0,6,1,7,5] 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6d8c09b410 0x7f6d8c0691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-f con 0x7f6d8c09b410 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 0x7f6d8c09bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-b con 0x7f6d8c09b7f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:26.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c0a3e60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6d8c0698b0 con 0x7f6d8c09b410 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8c09ca10 con 0x7f6d8c09b7f0 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6d8c0a5df0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d93603640 10 monclient(hunting): authenticate will time out at 1530.130396s 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 0x7f6d8c09bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6d8c09b410 0x7f6d8c0691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51950/0 (socket says 172.21.15.105:51950) 2024-09-17T16:37:26.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 0x7f6d8c09bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50328/0 (socket says 172.21.15.105:50328) 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 -- 172.21.15.105:0/2597170896 learned_addr learned my addr 172.21.15.105:0/2597170896 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): get_auth_request con 0x7f6d8c0a3e60 auth_method 0 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): get_auth_request con 0x7f6d8c09b7f0 auth_method 0 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): get_auth_request con 0x7f6d8c09b410 auth_method 0 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d90b9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_done global_id 54445 payload 274 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 -- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 msgr2=0x7f6d8c09bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 0x7f6d8c09bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 -- 172.21.15.105:0/2597170896 >> v2:172.21.15.160:3300/0 conn(0x7f6d8c09b410 msgr2=0x7f6d8c0691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/2597170896 >> v2:172.21.15.160:3300/0 conn(0x7f6d8c09b410 0x7f6d8c0691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 monclient: found mon.noname-c 2024-09-17T16:37:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:26.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.861+0000 7f6d91ba0640 1 -- 172.21.15.105:0/2597170896 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c09c450 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009e80 tx=0x7f6d8401c8d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=50be093d15aa3c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 1 -- 172.21.15.105:0/2597170896 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840090d0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: dump: 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.882 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 5 monclient: authenticate success, global_id 54445 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 1 -- 172.21.15.105:0/2597170896 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840a59d0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 4 set_mon_vals no callback set 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 1 -- 172.21.15.105:0/2597170896 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840a5bc0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d7ffff640 10 monclient: dump: 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.884 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: shutdown 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 msgr2=0x7f6d8c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009e80 tx=0x7f6d8401c8d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- 172.21.15.105:0/2597170896 shutdown_connections 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c0a42c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- 172.21.15.105:0/2597170896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d8c09b7f0 0x7f6d8c09bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- 172.21.15.105:0/2597170896 >> v2:172.21.15.160:3300/0 conn(0x7f6d8c09b410 0x7f6d8c0691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- 172.21.15.105:0/2597170896 >> 172.21.15.105:0/2597170896 conn(0x7f6d8c0a4f50 msgr2=0x7f6d8c0a5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- 172.21.15.105:0/2597170896 shutdown_connections 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- 172.21.15.105:0/2597170896 wait complete. 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: build_initial_monmap 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: monmap: 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 librados: starting msgr at 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 librados: starting objecter 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 Processor -- start 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- start start 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 librados: setting wanted keys 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 librados: calling monclient init 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: init 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient: _add_conns ranks=[1,7,4,2,0,6,3,8,5] 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09b410 0x7f6d8c12d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-f con 0x7f6d8c09b410 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d8c09b7f0 0x7f6d8c12da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-i con 0x7f6d8c09b7f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): picked mon.noname-c con 0x7f6d8c0a3e60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1432a0 con 0x7f6d8c09b410 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6d8c143120 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6d8c143420 con 0x7f6d8c09b7f0 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d93603640 10 monclient(hunting): authenticate will time out at 1530.135626s 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51958/0 (socket says 172.21.15.105:51958) 2024-09-17T16:37:26.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 1 -- 172.21.15.105:0/1337454018 learned_addr learned my addr 172.21.15.105:0/1337454018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 10 monclient(hunting): get_auth_request con 0x7f6d8c0a3e60 auth_method 0 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.865+0000 7f6d91ba0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09b410 0x7f6d8c12d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): get_auth_request con 0x7f6d8c09b410 auth_method 0 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d8c09b7f0 0x7f6d8c12da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 10 monclient(hunting): get_auth_request con 0x7f6d8c09b7f0 auth_method 0 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient(hunting): handle_auth_done global_id 54454 payload 824 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d8c09b7f0 msgr2=0x7f6d8c12da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d8c09b7f0 0x7f6d8c12da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09b410 msgr2=0x7f6d8c12d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09b410 0x7f6d8c12d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 monclient: found mon.noname-c 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c130080 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d91ba0640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009040 tx=0x7f6d84019670 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.?.objecter ms_handle_connect 0x7f6d8c0a3e60 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840bc020 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: dump: 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840039a0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d84004040 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: dump: 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7f7fe640 4 set_mon_vals no callback set 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 5 monclient: authenticate success, global_id 54454 2024-09-17T16:37:26.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 monclient: _renew_subs 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c12e600 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 client.54454.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 monclient: _renew_subs 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c12fac0 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 1 librados: init done 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 0x7f6d8c09a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c051e80 con 0x7f6d8c09a040 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 0x7f6d8c09a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient: get_auth_request con 0x7f6d8c09a040 auth_method 0 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6d840bf020 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7003e580 0x7f6d70040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7f6d840b3040 con 0x7f6d8c0a3e60 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.54454.objecter ms_dispatch 0x7f6d8c000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 3 client.54454.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 3 client.54454.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:26.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 20 client.54454.objecter dump_active .. 0 homeless 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 10 monclient(hunting): handle_auth_done global_id 54424 payload 824 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d8c09bbd0 con 0x7f6d8c09a040 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7003e580 0x7f6d70040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 10 monclient: get_auth_request con 0x7f6d7003e580 auth_method 0 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.54454.objecter ms_handle_connect 0x7f6d8c09a040 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.54454.objecter resend_mon_ops 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d9139f640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 0x7f6d8c09a420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d8000fb30 con 0x7f6d8c09a040 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: dump: 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.899 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 client.54454.objecter ms_handle_connect 0x7f6d7003e580 2024-09-17T16:37:26.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d90b9e640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7003e580 0x7f6d70040a40 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c13c410 tx=0x7f6d88008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6d8000f6c0 con 0x7f6d8c09a040 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 msgr2=0x7f6d8c09a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 0x7f6d8c09a420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800119f0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 msgr2=0x7f6d8c09a420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.869+0000 7f6d7dffb640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a040 0x7f6d8c09a420 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800119f0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d93603640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d93603640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 0x7f6d8c139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d93603640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c12fd00 con 0x7f6d8c09a960 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 0x7f6d8c139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient: get_auth_request con 0x7f6d8c09a960 auth_method 0 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:26.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 10 monclient(hunting): handle_auth_done global_id 54433 payload 824 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 1 -- 172.21.15.105:0/1337454018 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6d8c06d810 con 0x7f6d8c09a960 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 10 client.54454.objecter ms_handle_connect 0x7f6d8c09a960 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 10 client.54454.objecter resend_mon_ops 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d9139f640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 0x7f6d8c139aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d800229e0 con 0x7f6d8c09a960 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.877+0000 7f6d7dffb640 10 monclient: dump: 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:26.904 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f6d80005f70 con 0x7f6d8c09a960 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 msgr2=0x7f6d8c139aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 0x7f6d8c139aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 msgr2=0x7f6d8c139aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d7dffb640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d8c09a960 0x7f6d8c139aa0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011530 tx=0x7f6d800046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 153886, 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:26.906 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:26.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.909 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:26.910 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:26.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:26.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.914 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:26.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.917 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:26.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:26.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.922 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:26.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:26.926 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:26.927 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:26.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:26.928 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:26.928 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 10 librados: watch_flush enter 2024-09-17T16:37:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 10 librados: watch_flush exit 2024-09-17T16:37:26.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 20 client.54454.objecter shutdown clearing up homeless session... 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 10 client.54454.objecter successfully canceled tick 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7003e580 msgr2=0x7f6d70040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7003e580 0x7f6d70040a40 secure :-1 s=READY pgs=3533 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c13c410 tx=0x7f6d88008040 comp rx=0 tx=0).stop 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 10 monclient: shutdown 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 msgr2=0x7f6d8c12df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d91ba0640 1 -- 172.21.15.105:0/1337454018 reap_dead start 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009040 tx=0x7f6d84019670 comp rx=0 tx=0).stop 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 shutdown_connections 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 --2- 172.21.15.105:0/1337454018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d8c0a3e60 0x7f6d8c12df90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 >> 172.21.15.105:0/1337454018 conn(0x7f6d8c0a4f50 msgr2=0x7f6d8c0a1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 shutdown_connections 2024-09-17T16:37:26.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 -- 172.21.15.105:0/1337454018 wait complete. 2024-09-17T16:37:26.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:26.881+0000 7f6d93603640 1 librados: shutdown 2024-09-17T16:37:26.931 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:26.931 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:37:26.931 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: get_monmap_and_config 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: monmap: 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:27.056754+0000 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:27.056754+0000 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:27.060 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:27.061 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 Processor -- start 2024-09-17T16:37:27.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 -- start start 2024-09-17T16:37:27.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: init 2024-09-17T16:37:27.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient: _add_conns ranks=[7,5,9,4,2,3,10,8,0,6,1] 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7feca40a3720 0x7feca40a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-i con 0x7feca40a3720 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 0x7feca40a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-h con 0x7feca40a3b00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feca40a3f00 0x7feca40a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-a con 0x7feca40a3f00 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7feca406db50 con 0x7feca40a3f00 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7feca40a0e80 con 0x7feca40a3b00 2024-09-17T16:37:27.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7feca40a1490 con 0x7feca40a3720 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca9f77640 10 monclient(hunting): authenticate will time out at 1530.322546s 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feca40a3f00 0x7feca40a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feca40a3f00 0x7feca40a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46346/0 (socket says 172.21.15.105:46346) 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 -- 172.21.15.105:0/1959795281 learned_addr learned my addr 172.21.15.105:0/1959795281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): get_auth_request con 0x7feca40a3f00 auth_method 0 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca37fe640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7feca40a3720 0x7feca40a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca2ffd640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 0x7feca40a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient(hunting): handle_auth_done global_id 35748 payload 274 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 -- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7feca40a3720 msgr2=0x7feca40a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7feca40a3720 0x7feca40a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 -- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 msgr2=0x7feca40a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 0x7feca40a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 monclient: found mon.noname-a 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca3fff640 1 -- 172.21.15.105:0/1959795281 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40a7240 con 0x7feca40a3f00 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca2ffd640 10 monclient: get_auth_request con 0x7feca40a3b00 auth_method 0 2024-09-17T16:37:27.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.053+0000 7feca2ffd640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 0x7feca40a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca3fff640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca40a5c00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe34e3ea8bf4dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 1 -- 172.21.15.105:0/1959795281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec940b0070 con 0x7feca40a3f00 2024-09-17T16:37:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: dump: 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 5 monclient: authenticate success, global_id 35748 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 1 -- 172.21.15.105:0/1959795281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec940aa810 con 0x7feca40a3f00 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 4 set_mon_vals no callback set 2024-09-17T16:37:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 1 -- 172.21.15.105:0/1959795281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec940a9300 con 0x7feca40a3f00 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca27fc640 10 monclient: dump: 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 10 monclient: shutdown 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- 172.21.15.105:0/1959795281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 msgr2=0x7feca40a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca40a5c00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9401c4b0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- 172.21.15.105:0/1959795281 shutdown_connections 2024-09-17T16:37:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca40a5c00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7feca40a3b00 0x7feca40a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 --2- 172.21.15.105:0/1959795281 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7feca40a3720 0x7feca40a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- 172.21.15.105:0/1959795281 >> 172.21.15.105:0/1959795281 conn(0x7feca40aa390 msgr2=0x7feca40a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- 172.21.15.105:0/1959795281 shutdown_connections 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- 172.21.15.105:0/1959795281 wait complete. 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 10 monclient: monmap: 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 librados: starting msgr at 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 librados: starting objecter 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 Processor -- start 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 -- start start 2024-09-17T16:37:27.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 librados: setting wanted keys 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 1 librados: calling monclient init 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.057+0000 7feca9f77640 10 monclient: init 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _add_conns ranks=[3,4,0,8,2,7,6,5,1] 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca40a3720 0x7feca41d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-g con 0x7feca40a3720 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 0x7feca41d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-c con 0x7feca40a3b00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca409e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): picked mon.noname-a con 0x7feca40a3f00 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feca41432d0 con 0x7feca40a3f00 2024-09-17T16:37:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7feca40a6550 con 0x7feca40a3720 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7feca4143450 con 0x7feca40a3b00 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): authenticate will time out at 1530.328538s 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 0x7feca41d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca409e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 0x7feca41d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:51972/0 (socket says 172.21.15.105:51972) 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 -- 172.21.15.105:0/387099946 learned_addr learned my addr 172.21.15.105:0/387099946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): get_auth_request con 0x7feca40a3b00 auth_method 0 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 10 monclient(hunting): get_auth_request con 0x7feca40a3f00 auth_method 0 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 msgr2=0x7feca409e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 msgr2=0x7feca409e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca409e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca409e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca40a3720 0x7feca41d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): get_auth_request con 0x7feca40a3720 auth_method 0 2024-09-17T16:37:27.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient(hunting): handle_auth_done global_id 54463 payload 824 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca40a3720 msgr2=0x7feca41d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca40a3720 0x7feca41d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 msgr2=0x7feca409e630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feca40a3f00 0x7feca409e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 monclient: found mon.noname-c 2024-09-17T16:37:27.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40a7f20 con 0x7feca40a3b00 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca2ffd640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 0x7feca41d1b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec9800cf80 tx=0x7fec9800a7d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 client.?.objecter ms_handle_connect 0x7feca40a3b00 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec9801a070 con 0x7feca40a3b00 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: dump: 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.085 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9800dab0 con 0x7feca40a3b00 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec9800dc60 con 0x7feca40a3b00 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 monclient: dump: 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.087 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca1ffb640 4 set_mon_vals no callback set 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 5 monclient: authenticate success, global_id 54463 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _renew_subs 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:27.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca4125b80 con 0x7feca40a3b00 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 client.54463.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _renew_subs 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7feca413ca20 con 0x7feca40a3b00 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 librados: init done 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 0x7feca409ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7feca4051e80 con 0x7feca409eb70 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fec9800d640 con 0x7feca40a3b00 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 0x7feca409ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient: get_auth_request con 0x7feca409eb70 auth_method 0 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fec9003e550 0x7fec90040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7fec980504d0 con 0x7feca40a3b00 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 10 client.54463.objecter ms_dispatch 0x7feca4000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 3 client.54463.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 3 client.54463.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7fec8bfff640 20 client.54463.objecter dump_active .. 0 homeless 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fec9003e550 0x7fec90040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 10 monclient(hunting): handle_auth_done global_id 54372 payload 824 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca37fe640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feca40a7b60 con 0x7feca409eb70 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.061+0000 7feca3fff640 10 monclient: get_auth_request con 0x7fec9003e550 auth_method 0 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 client.54463.objecter ms_handle_connect 0x7feca409eb70 2024-09-17T16:37:27.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7feca37fe640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 0x7feca409ef50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00f810 tx=0x7fec9c00fcd0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 client.54463.objecter resend_mon_ops 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec9c00de90 con 0x7feca409eb70 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 monclient: dump: 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.091 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 client.54463.objecter ms_handle_connect 0x7fec9003e550 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7feca3fff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fec9003e550 0x7fec90040a10 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7feca40a7800 tx=0x7fec940a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec9c00da20 con 0x7feca409eb70 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 msgr2=0x7feca409ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 0x7feca409ef50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00f810 tx=0x7fec9c00fcd0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 msgr2=0x7feca409ef50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.065+0000 7fec8bfff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409eb70 0x7feca409ef50 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00f810 tx=0x7fec9c00fcd0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca9f77640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca9f77640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 0x7feca413cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca9f77640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7feca40a1e70 con 0x7feca409f490 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 0x7feca413cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient: get_auth_request con 0x7feca409f490 auth_method 0 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.069+0000 7feca37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca37fe640 10 monclient(hunting): handle_auth_done global_id 54381 payload 824 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca37fe640 1 -- 172.21.15.105:0/387099946 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feca41262f0 con 0x7feca409f490 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca37fe640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 0x7feca413cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec9c008010 tx=0x7fec9c0046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 client.54463.objecter ms_handle_connect 0x7feca409f490 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 client.54463.objecter resend_mon_ops 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fec9c003a90 con 0x7feca409f490 2024-09-17T16:37:27.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 monclient: dump: 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.096 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fec9c003620 con 0x7feca409f490 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 msgr2=0x7feca413cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 0x7feca413cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec9c008010 tx=0x7fec9c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 msgr2=0x7feca413cc30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7fec8bfff640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feca409f490 0x7feca413cc30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 154078, 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:27.098 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.100 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:27.101 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:27.102 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.103 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.105 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:27.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:27.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.113 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:27.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:27.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.119 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:27.120 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 10 librados: watch_flush enter 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 10 librados: watch_flush exit 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 20 client.54463.objecter shutdown clearing up homeless session... 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 10 client.54463.objecter successfully canceled tick 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fec9003e550 msgr2=0x7fec90040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fec9003e550 0x7fec90040a10 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7feca40a7800 tx=0x7fec940a6000 comp rx=0 tx=0).stop 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 10 monclient: shutdown 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 msgr2=0x7feca41d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 --2- 172.21.15.105:0/387099946 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7feca40a3b00 0x7feca41d1b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fec9800cf80 tx=0x7fec9800a7d0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca3fff640 1 -- 172.21.15.105:0/387099946 reap_dead start 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca3fff640 1 -- 172.21.15.105:0/387099946 reap_dead start 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 shutdown_connections 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 >> 172.21.15.105:0/387099946 conn(0x7feca40aa390 msgr2=0x7feca40a8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 shutdown_connections 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 -- 172.21.15.105:0/387099946 wait complete. 2024-09-17T16:37:27.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.073+0000 7feca9f77640 1 librados: shutdown 2024-09-17T16:37:27.123 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:27.123 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:37:27.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:37:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: get_monmap_and_config 2024-09-17T16:37:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: monmap: 2024-09-17T16:37:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.243 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:27.240339+0000 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:27.240339+0000 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.244 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.245 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:27.245 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:27.245 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 Processor -- start 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 -- start start 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: init 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient: _add_conns ranks=[5,2,7,3,4,6,8,10,0,1,9] 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63140a68e0 0x7f63140a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-h con 0x7f63140a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-b con 0x7f63140a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63141396a0 0x7f6314139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-i con 0x7f63141396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63140a3fc0 con 0x7f63140a4910 2024-09-17T16:37:27.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f63140a63c0 con 0x7f63140a68e0 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f63140a7740 con 0x7f63141396a0 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6318b1c640 10 monclient(hunting): authenticate will time out at 1530.506065s 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50334/0 (socket says 172.21.15.105:50334) 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 -- 172.21.15.105:0/3078517580 learned_addr learned my addr 172.21.15.105:0/3078517580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): get_auth_request con 0x7f63140a4910 auth_method 0 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f631259c640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63140a68e0 0x7f63140a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6312d9d640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63141396a0 0x7f6314139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f631259c640 10 monclient(hunting): get_auth_request con 0x7f63140a68e0 auth_method 0 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f631259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f631259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f631259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6312d9d640 10 monclient(hunting): get_auth_request con 0x7f63141396a0 auth_method 0 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6312d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6312d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6312d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient(hunting): handle_auth_done global_id 44840 payload 274 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 -- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63141396a0 msgr2=0x7f6314139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63141396a0 0x7f6314139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 -- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63140a68e0 msgr2=0x7f63140a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63140a68e0 0x7f63140a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 monclient: found mon.noname-b 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.237+0000 7f6311d9b640 1 -- 172.21.15.105:0/3078517580 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631406ec10 con 0x7f63140a4910 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63080098d0 tx=0x7f630801cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b17785ae75f7b32b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 1 -- 172.21.15.105:0/3078517580 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63080b0070 con 0x7f63140a4910 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: dump: 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.252 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 1 -- 172.21.15.105:0/3078517580 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63080aa810 con 0x7f63140a4910 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 5 monclient: authenticate success, global_id 44840 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 1 -- 172.21.15.105:0/3078517580 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63080a9300 con 0x7f63140a4910 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:27.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 4 set_mon_vals no callback set 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631159a640 10 monclient: dump: 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.254 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.255 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: shutdown 2024-09-17T16:37:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- 172.21.15.105:0/3078517580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 msgr2=0x7f63140a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63080098d0 tx=0x7f630801cea0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- 172.21.15.105:0/3078517580 shutdown_connections 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f63141396a0 0x7f6314139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63140a4910 0x7f63140a7200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- 172.21.15.105:0/3078517580 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f63140a68e0 0x7f63140a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- 172.21.15.105:0/3078517580 >> 172.21.15.105:0/3078517580 conn(0x7f6314068ba0 msgr2=0x7f6314068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- 172.21.15.105:0/3078517580 shutdown_connections 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- 172.21.15.105:0/3078517580 wait complete. 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: monmap: 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:27.263 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 librados: starting msgr at 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 librados: starting objecter 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 Processor -- start 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- start start 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 librados: setting wanted keys 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 librados: calling monclient init 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: init 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient: _add_conns ranks=[1,3,2,5,6,7,8,0,4] 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63140a4910 0x7f63141361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-f con 0x7f63140a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63140a68e0 0x7f6314136710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-g con 0x7f63140a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:27.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 0x7f631413b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): picked mon.noname-b con 0x7f63141396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6314143320 con 0x7f63140a4910 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f631406e630 con 0x7f63141396a0 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f63141434a0 con 0x7f63140a68e0 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6318b1c640 10 monclient(hunting): authenticate will time out at 1530.511528s 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 0x7f631413b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 0x7f631413b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:50344/0 (socket says 172.21.15.105:50344) 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 learned_addr learned my addr 172.21.15.105:0/1256050615 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): get_auth_request con 0x7f63141396a0 auth_method 0 2024-09-17T16:37:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63140a68e0 0x7f6314136710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 10 monclient(hunting): get_auth_request con 0x7f63140a68e0 auth_method 0 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6311d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f631259c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63140a4910 0x7f63141361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 10 monclient(hunting): handle_auth_done global_id 44849 payload 824 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63140a68e0 msgr2=0x7f6314136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.241+0000 7f6312d9d640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63140a68e0 0x7f6314136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63140a4910 msgr2=0x7f63141361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63140a4910 0x7f63141361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 1 monclient: found mon.noname-b 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631413f510 con 0x7f63141396a0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6312d9d640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 0x7f631413b6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fd0 tx=0x7f630c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.?.objecter ms_handle_connect 0x7f63141396a0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f630c00ed70 con 0x7f63141396a0 2024-09-17T16:37:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: dump: 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c019070 con 0x7f63141396a0 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f630c014d00 con 0x7f63141396a0 2024-09-17T16:37:27.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 monclient: dump: 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.271 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6310d99640 4 set_mon_vals no callback set 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 5 monclient: authenticate success, global_id 44849 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 monclient: _renew_subs 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631413fd20 con 0x7f63141396a0 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 client.44849.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 monclient: _renew_subs 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:27.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f631413ef50 con 0x7f63141396a0 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 1 librados: init done 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 0x7f631413bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f631413dd10 con 0x7f631413f120 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f630c014420 con 0x7f63141396a0 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 0x7f631413bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f62f403e550 0x7f62f4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient: get_auth_request con 0x7f631413f120 auth_method 0 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(381..381 src has 1..381) ==== 6189+0+0 (secure 0 0 0) 0x7f630c0506a0 con 0x7f63141396a0 2024-09-17T16:37:27.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.44849.objecter ms_dispatch 0x7f6314000ba0 osd_map(381..381 src has 1..381) 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 3 client.44849.objecter handle_osd_map got epochs [381,381] > 0 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 3 client.44849.objecter handle_osd_map decoding full epoch 381 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 20 client.44849.objecter dump_active .. 0 homeless 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6311d9b640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f62f403e550 0x7f62f4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6311d9b640 10 monclient: get_auth_request con 0x7f62f403e550 auth_method 0 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 10 monclient(hunting): handle_auth_done global_id 20084 payload 824 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f631413d890 con 0x7f631413f120 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.44849.objecter ms_handle_connect 0x7f62f403e550 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.44849.objecter ms_handle_connect 0x7f631413f120 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6311d9b640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f62f403e550 0x7f62f4040a10 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7f63080ad000 tx=0x7f6308004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f6302ffd640 10 client.44849.objecter resend_mon_ops 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.245+0000 7f631259c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 0x7f631413bbe0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f630400e510 tx=0x7f630400e9d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6304017070 con 0x7f631413f120 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 10 monclient: dump: 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.275 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f630401c450 con 0x7f631413f120 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:27.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 msgr2=0x7f631413bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 0x7f631413bbe0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f630400e510 tx=0x7f630400e9d0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 msgr2=0x7f631413bbe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.249+0000 7f6302ffd640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413f120 0x7f631413bbe0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6318b1c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:27.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6318b1c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 0x7f631413c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6318b1c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6314051e80 con 0x7f631413c120 2024-09-17T16:37:27.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 0x7f631413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient: get_auth_request con 0x7f631413c120 auth_method 0 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 10 monclient(hunting): handle_auth_done global_id 20093 payload 824 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 1 -- 172.21.15.105:0/1256050615 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6314139ae0 con 0x7f631413c120 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f631259c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 0x7f631413c530 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6304008010 tx=0x7f6304004740 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6302ffd640 10 client.44849.objecter ms_handle_connect 0x7f631413c120 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.257+0000 7f6302ffd640 10 client.44849.objecter resend_mon_ops 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.261+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f630400f040 con 0x7f631413c120 2024-09-17T16:37:27.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.261+0000 7f6302ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.261+0000 7f6302ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.261+0000 7f6302ffd640 10 monclient: dump: 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.280 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.281 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.281 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 594872, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:27.285 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.286 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.287 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:27.288 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.289 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:27.290 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.291 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.293 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.296 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:27.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:27.296 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:27.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:27.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.302 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:27.304 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.306 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.307 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:27.308 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:27.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f6304013dc0 con 0x7f631413c120 2024-09-17T16:37:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 msgr2=0x7f631413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 0x7f631413c530 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6304008010 tx=0x7f6304004740 comp rx=0 tx=0).stop 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 msgr2=0x7f631413c530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6302ffd640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f631413c120 0x7f631413c530 secure :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6304008010 tx=0x7f6304004740 comp rx=0 tx=0).stop 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 10 librados: watch_flush enter 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 10 librados: watch_flush exit 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 20 client.44849.objecter shutdown clearing up homeless session... 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 10 client.44849.objecter successfully canceled tick 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f62f403e550 msgr2=0x7f62f4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f62f403e550 0x7f62f4040a10 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7f63080ad000 tx=0x7f6308004740 comp rx=0 tx=0).stop 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 10 monclient: shutdown 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 msgr2=0x7f631413b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 --2- 172.21.15.105:0/1256050615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63141396a0 0x7f631413b6a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fd0 tx=0x7f630c00e4b0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 reap_dead start 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6312d9d640 1 -- 172.21.15.105:0/1256050615 reap_dead start 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 shutdown_connections 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 >> 172.21.15.105:0/1256050615 conn(0x7f6314068ba0 msgr2=0x7f63140a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 shutdown_connections 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 -- 172.21.15.105:0/1256050615 wait complete. 2024-09-17T16:37:27.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.277+0000 7f6318b1c640 1 librados: shutdown 2024-09-17T16:37:27.313 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:27.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:37:27.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:37:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.417+0000 7f96570dd640 10 monclient: get_monmap_and_config 2024-09-17T16:37:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.417+0000 7f96570dd640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.417+0000 7f96570dd640 10 monclient: monmap: 2024-09-17T16:37:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:27.423277+0000 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:27.423277+0000 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:27.428 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 Processor -- start 2024-09-17T16:37:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 -- start start 2024-09-17T16:37:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient: init 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient: _add_conns ranks=[9,10,0,8,2,4,5,7,6,3,1] 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9650138760 0x7f96500a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-a con 0x7f9650138760 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9650138b40 0x7f9650139490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-f con 0x7f9650138b40 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9650138f60 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96500a3440 con 0x7f9650138f60 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f96500a3720 con 0x7f9650138760 2024-09-17T16:37:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f96500a3a00 con 0x7f9650138b40 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient(hunting): authenticate will time out at 1530.688940s 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9650138760 0x7f96500a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9650138760 0x7f96500a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46376/0 (socket says 172.21.15.105:46376) 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 -- 172.21.15.105:0/376919329 learned_addr learned my addr 172.21.15.105:0/376919329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): get_auth_request con 0x7f9650138760 auth_method 0 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient(hunting): handle_auth_done global_id 35757 payload 274 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 -- 172.21.15.105:0/376919329 >> v2:172.21.15.160:3300/0 conn(0x7f9650138b40 msgr2=0x7f9650139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 --2- 172.21.15.105:0/376919329 >> v2:172.21.15.160:3300/0 conn(0x7f9650138b40 0x7f9650139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 -- 172.21.15.105:0/376919329 >> v1:172.21.15.105:6789/0 conn(0x7f9650138f60 legacy=0x7f96501399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 monclient: found mon.noname-a 2024-09-17T16:37:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 -- 172.21.15.105:0/376919329 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96500a3ce0 con 0x7f9650138760 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f9654e79640 1 --2- 172.21.15.105:0/376919329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9650138760 0x7f96500a26b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9650074800 tx=0x7f964c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa3ae1a5576cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 1 -- 172.21.15.105:0/376919329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f964c0ab870 con 0x7f9650138760 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: dump: 2024-09-17T16:37:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 1 -- 172.21.15.105:0/376919329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c0b0070 con 0x7f9650138760 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 5 monclient: authenticate success, global_id 35757 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 1 -- 172.21.15.105:0/376919329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f964c0aa360 con 0x7f9650138760 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96570dd640 4 set_mon_vals no callback set 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.421+0000 7f96477fe640 10 monclient: dump: 2024-09-17T16:37:27.445 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.446 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: shutdown 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/376919329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9650138760 msgr2=0x7f96500a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- 172.21.15.105:0/376919329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9650138760 0x7f96500a26b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9650074800 tx=0x7f964c01c3e0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/376919329 shutdown_connections 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- 172.21.15.105:0/376919329 >> v2:172.21.15.160:3300/0 conn(0x7f9650138b40 0x7f9650139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- 172.21.15.105:0/376919329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9650138760 0x7f96500a26b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/376919329 >> 172.21.15.105:0/376919329 conn(0x7f9650068ba0 msgr2=0x7f9650068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/376919329 shutdown_connections 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/376919329 wait complete. 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: monmap: 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.448 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 librados: starting msgr at 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 librados: starting objecter 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 Processor -- start 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- start start 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 librados: setting wanted keys 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 librados: calling monclient init 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: init 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _add_conns ranks=[1,6,7,8,0,4,3,5,2] 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9650138760 0x7f965013be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-f con 0x7f9650138760 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 0x7f965013c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-d con 0x7f9650138b40 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650138f60 0x7f965013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): picked mon.noname-i con 0x7f9650138f60 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9650144170 con 0x7f9650138760 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9650143ff0 con 0x7f9650138b40 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f96501442f0 con 0x7f9650138f60 2024-09-17T16:37:27.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): authenticate will time out at 1530.693855s 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 0x7f965013c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 0x7f965013c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:59440/0 (socket says 172.21.15.105:59440) 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 -- 172.21.15.105:0/3096617938 learned_addr learned my addr 172.21.15.105:0/3096617938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): get_auth_request con 0x7f9650138b40 auth_method 0 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f965567a640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650138f60 0x7f965013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f965567a640 10 monclient(hunting): get_auth_request con 0x7f9650138f60 auth_method 0 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9650138760 0x7f965013be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f965567a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f965567a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f965567a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient(hunting): handle_auth_done global_id 24522 payload 824 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650138f60 msgr2=0x7f965013c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650138f60 0x7f965013c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9650138760 msgr2=0x7f965013be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9650138760 0x7f965013be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 monclient: found mon.noname-d 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965013ec70 con 0x7f9650138b40 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 10 monclient: get_auth_request con 0x7f9650138760 auth_method 0 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9650138760 0x7f965013be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9647fff640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 0x7f965013c390 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f964000b580 tx=0x7f964000ba40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 client.?.objecter ms_handle_connect 0x7f9650138b40 2024-09-17T16:37:27.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9640014070 con 0x7f9650138b40 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: dump: 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.455 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 5 monclient: authenticate success, global_id 24522 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _renew_subs 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9640010ae0 con 0x7f9650138b40 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965013d780 con 0x7f9650138b40 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9640010c90 con 0x7f9650138b40 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:27.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 client.24522.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 10 monclient: dump: 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.458 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96467fc640 4 set_mon_vals no callback set 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _renew_subs 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f965013f240 con 0x7f9650138b40 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 librados: init done 2024-09-17T16:37:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 0x7f96501359c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9650051e80 con 0x7f96501355e0 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9640017070 con 0x7f9650138b40 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 0x7f96501359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 10 monclient: get_auth_request con 0x7f96501355e0 auth_method 0 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f9654e79640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.425+0000 7f96457fa640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f963c03e550 0x7f963c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(382..382 src has 1..382) ==== 6202+0+0 (secure 0 0 0) 0x7f964004fbe0 con 0x7f9650138b40 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 client.24522.objecter ms_dispatch 0x7f9650000ba0 osd_map(382..382 src has 1..382) 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 3 client.24522.objecter handle_osd_map got epochs [382,382] > 0 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 3 client.24522.objecter handle_osd_map decoding full epoch 382 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 20 client.24522.objecter dump_active .. 0 homeless 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f965567a640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f963c03e550 0x7f963c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f965567a640 10 monclient: get_auth_request con 0x7f963c03e550 auth_method 0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 10 monclient(hunting): handle_auth_done global_id 64222 payload 824 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9650139b70 con 0x7f96501355e0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 client.24522.objecter ms_handle_connect 0x7f96501355e0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 client.24522.objecter resend_mon_ops 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 0x7f96501359c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96500a46c0 tx=0x7f9650069c20 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 client.24522.objecter ms_handle_connect 0x7f963c03e550 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f965567a640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f963c03e550 0x7f963c040a10 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7f96480012f0 tx=0x7f9648009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f964c0ab400 con 0x7f96501355e0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 monclient: dump: 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f964c0b0040 con 0x7f96501355e0 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 msgr2=0x7f96501359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 0x7f96501359c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96500a46c0 tx=0x7f9650069c20 comp rx=0 tx=0).stop 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 msgr2=0x7f96501359c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.429+0000 7f96457fa640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f96501355e0 0x7f96501359c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96500a46c0 tx=0x7f9650069c20 comp rx=0 tx=0).stop 2024-09-17T16:37:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.433+0000 7f96570dd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 0x7f96501db250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f965013f620 con 0x7f9650135f00 2024-09-17T16:37:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 0x7f96501db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient: get_auth_request con 0x7f9650135f00 auth_method 0 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 10 monclient(hunting): handle_auth_done global_id 64231 payload 824 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 1 -- 172.21.15.105:0/3096617938 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f965013ffe0 con 0x7f9650135f00 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f9654e79640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 0x7f96501db250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097c0 tx=0x7f964c009240 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 client.24522.objecter ms_handle_connect 0x7f9650135f00 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 client.24522.objecter resend_mon_ops 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f964c0ab400 con 0x7f9650135f00 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 monclient: dump: 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.469 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f964c0ab400 con 0x7f9650135f00 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 msgr2=0x7f96501db250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 0x7f96501db250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097c0 tx=0x7f964c009240 comp rx=0 tx=0).stop 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 msgr2=0x7f96501db250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96457fa640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9650135f00 0x7f96501db250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.470 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 62, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 20535, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:27.471 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.473 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.474 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:27.475 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:27.476 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:27.477 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:27.479 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:27.481 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.482 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.483 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.484 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:27.486 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.490 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.491 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:27.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:27.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.492 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:27.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:27.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.498 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:27.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:27.500 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:27.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:27.503 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 10 librados: watch_flush enter 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 10 librados: watch_flush exit 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 20 client.24522.objecter shutdown clearing up homeless session... 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 10 client.24522.objecter successfully canceled tick 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f963c03e550 msgr2=0x7f963c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f963c03e550 0x7f963c040a10 secure :-1 s=READY pgs=3536 cs=0 l=1 rev1=1 crypto rx=0x7f96480012f0 tx=0x7f9648009040 comp rx=0 tx=0).stop 2024-09-17T16:37:27.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 10 monclient: shutdown 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 msgr2=0x7f965013c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 --2- 172.21.15.105:0/3096617938 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9650138b40 0x7f965013c390 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f964000b580 tx=0x7f964000ba40 comp rx=0 tx=0).stop 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f965567a640 1 -- 172.21.15.105:0/3096617938 reap_dead start 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f965567a640 1 -- 172.21.15.105:0/3096617938 reap_dead start 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 shutdown_connections 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 >> 172.21.15.105:0/3096617938 conn(0x7f9650068ba0 msgr2=0x7f965009f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 shutdown_connections 2024-09-17T16:37:27.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 -- 172.21.15.105:0/3096617938 wait complete. 2024-09-17T16:37:27.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.437+0000 7f96570dd640 1 librados: shutdown 2024-09-17T16:37:27.508 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:37:27.508 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'c'] 2024-09-17T16:37:27.508 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-17T16:37:27.508 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-17T16:37:27.509 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-09-17T16:37:27.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 10 monclient: get_monmap_and_config 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 10 monclient: monmap: 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:27.622028+0000 2024-09-17T16:37:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:27.622028+0000 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:27.627 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 1 Processor -- start 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 1 -- start start 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.617+0000 7f2e73553640 10 monclient: init 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient: _add_conns ranks=[1,9,6,8,2,0,4,5,7,10,3] 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2e6c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-a con 0x7f2e6c0a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 0x7f2e6c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-d con 0x7f2e6c0a73d0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6c06ebc0 con 0x7f2e6c0a68e0 2024-09-17T16:37:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6c0a3fc0 con 0x7f2e6c0a4910 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2e6c0a63c0 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient(hunting): authenticate will time out at 1530.888293s 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 0x7f2e6c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46392/0 (socket says 172.21.15.105:46392) 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 -- 172.21.15.105:0/128742297 learned_addr learned my addr 172.21.15.105:0/128742297 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a4910 auth_method 0 2024-09-17T16:37:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a73d0 auth_method 0 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient(hunting): handle_auth_done global_id 35766 payload 274 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 -- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 msgr2=0x7f2e6c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 0x7f2e6c141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 -- 172.21.15.105:0/128742297 >> v1:172.21.15.160:6789/0 conn(0x7f2e6c0a68e0 legacy=0x7f2e6c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 monclient: found mon.noname-a 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 -- 172.21.15.105:0/128742297 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c0a3690 con 0x7f2e6c0a4910 2024-09-17T16:37:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e71af0640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 0x7f2e6c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e70aee640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0d255535e89e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 1 -- 172.21.15.105:0/128742297 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e5c0b2070 con 0x7f2e6c0a4910 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: dump: 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 5 monclient: authenticate success, global_id 35766 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 1 -- 172.21.15.105:0/128742297 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0ab7f0 con 0x7f2e6c0a4910 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 4 set_mon_vals no callback set 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 1 -- 172.21.15.105:0/128742297 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e5c0abe50 con 0x7f2e6c0a4910 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e63fff640 10 monclient: dump: 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 10 monclient: shutdown 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a4910 msgr2=0x7f2e6c13eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- 172.21.15.105:0/128742297 shutdown_connections 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c0a73d0 0x7f2e6c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 --2- 172.21.15.105:0/128742297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a4910 0x7f2e6c13eaf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- 172.21.15.105:0/128742297 >> 172.21.15.105:0/128742297 conn(0x7f2e6c068ba0 msgr2=0x7f2e6c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.621+0000 7f2e73553640 1 -- 172.21.15.105:0/128742297 shutdown_connections 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- 172.21.15.105:0/128742297 wait complete. 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: build_initial_monmap 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: monmap: 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 librados: starting msgr at 2024-09-17T16:37:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 librados: starting objecter 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 Processor -- start 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- start start 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 librados: setting wanted keys 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 librados: calling monclient init 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: init 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _add_conns ranks=[3,7,0,5,6,2,1,4,8] 2024-09-17T16:37:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4910 0x7f2e6c1d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-g con 0x7f2e6c0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c0a68e0 0x7f2e6c1d1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-i con 0x7f2e6c0a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 0x7f2e6c1d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): picked mon.noname-a con 0x7f2e6c0a73d0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c143f20 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c143da0 con 0x7f2e6c0a4910 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2e6c1440a0 con 0x7f2e6c0a68e0 2024-09-17T16:37:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): authenticate will time out at 1530.893529s 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 0x7f2e6c1d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 0x7f2e6c1d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46398/0 (socket says 172.21.15.105:46398) 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 learned_addr learned my addr 172.21.15.105:0/2253157758 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a73d0 auth_method 0 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e70aee640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c0a68e0 0x7f2e6c1d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient(hunting): handle_auth_done global_id 35775 payload 824 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4910 0x7f2e6c1d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c0a68e0 msgr2=0x7f2e6c1d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c0a68e0 0x7f2e6c1d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4910 msgr2=0x7f2e6c1d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4910 0x7f2e6c1d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 monclient: found mon.noname-a 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c129260 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient: get_auth_request con 0x7f2e6c0a4910 auth_method 0 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4910 0x7f2e6c1d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e71af0640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 0x7f2e6c1d1aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400e930 tx=0x7f2e6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 client.?.objecter ms_handle_connect 0x7f2e6c0a73d0 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e64017070 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: dump: 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.654 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.655 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e6400cdb0 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 5 monclient: authenticate success, global_id 35775 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _renew_subs 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c127d70 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e640138e0 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e637fe640 4 set_mon_vals no callback set 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 monclient: dump: 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.657 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 client.35775.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:27.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _renew_subs 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c129830 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 librados: init done 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 0x7f2e6c1267e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f2e6c051e80 con 0x7f2e6c126400 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2e64013470 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e550 0x7f2e50040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(382..382 src has 1..382) ==== 6202+0+0 (secure 0 0 0) 0x7f2e6400f040 con 0x7f2e6c0a73d0 2024-09-17T16:37:27.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 10 client.35775.objecter ms_dispatch 0x7f2e6c000ba0 osd_map(382..382 src has 1..382) 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 3 client.35775.objecter handle_osd_map got epochs [382,382] > 0 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 3 client.35775.objecter handle_osd_map decoding full epoch 382 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 0x7f2e6c1267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient: get_auth_request con 0x7f2e6c126400 auth_method 0 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e61ffb640 20 client.35775.objecter dump_active .. 0 homeless 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.625+0000 7f2e70aee640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e550 0x7f2e50040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e70aee640 10 monclient: get_auth_request con 0x7f2e5003e550 auth_method 0 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e712ef640 10 monclient(hunting): handle_auth_done global_id 20102 payload 824 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e712ef640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e6c0a2670 con 0x7f2e6c126400 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 0x7f2e6c1267e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800da60 tx=0x7f2e6800df20 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 client.35775.objecter ms_handle_connect 0x7f2e6c126400 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 client.35775.objecter resend_mon_ops 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e68003bb0 con 0x7f2e6c126400 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:27.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e70aee640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e550 0x7f2e50040a10 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c0a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 monclient: dump: 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.662 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 client.35775.objecter ms_handle_connect 0x7f2e5003e550 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2e68003bb0 con 0x7f2e6c126400 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 msgr2=0x7f2e6c1267e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 0x7f2e6c1267e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800da60 tx=0x7f2e6800df20 comp rx=0 tx=0).stop 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 msgr2=0x7f2e6c1267e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.629+0000 7f2e61ffb640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126400 0x7f2e6c1267e0 secure :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800da60 tx=0x7f2e6800df20 comp rx=0 tx=0).stop 2024-09-17T16:37:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:37:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 0x7f2e6c122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f2e6c0a6cc0 con 0x7f2e6c126d20 2024-09-17T16:37:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 0x7f2e6c122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient: get_auth_request con 0x7f2e6c126d20 auth_method 0 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 10 monclient(hunting): handle_auth_done global_id 20111 payload 824 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 1 -- 172.21.15.105:0/2253157758 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f2e6c0746d0 con 0x7f2e6c126d20 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e712ef640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 0x7f2e6c122530 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2e68015000 tx=0x7f2e680014b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 client.35775.objecter ms_handle_connect 0x7f2e6c126d20 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 client.35775.objecter resend_mon_ops 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e68005650 con 0x7f2e6c126d20 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 monclient: dump: 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:27.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f2e68005650 con 0x7f2e6c126d20 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 msgr2=0x7f2e6c122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 0x7f2e6c122530 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2e68015000 tx=0x7f2e680014b0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 msgr2=0x7f2e6c122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e61ffb640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2e6c126d20 0x7f2e6c122530 secure :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2e68015000 tx=0x7f2e680014b0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.670 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:37:27.670 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:37:27.670 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 librados: watch_flush enter 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 librados: watch_flush exit 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 20 client.35775.objecter shutdown clearing up homeless session... 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 client.35775.objecter successfully canceled tick 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e550 msgr2=0x7f2e50040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e550 0x7f2e50040a10 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c0a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 10 monclient: shutdown 2024-09-17T16:37:27.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 msgr2=0x7f2e6c1d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 --2- 172.21.15.105:0/2253157758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e6c0a73d0 0x7f2e6c1d1aa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400e930 tx=0x7f2e6400edf0 comp rx=0 tx=0).stop 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 reap_dead start 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e71af0640 1 -- 172.21.15.105:0/2253157758 reap_dead start 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 shutdown_connections 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 >> 172.21.15.105:0/2253157758 conn(0x7f2e6c068ba0 msgr2=0x7f2e6c0a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 shutdown_connections 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 -- 172.21.15.105:0/2253157758 wait complete. 2024-09-17T16:37:27.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:27.637+0000 7f2e73553640 1 librados: shutdown 2024-09-17T16:37:27.673 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-17T16:37:27.674 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-17T16:37:27.674 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:37:27.711 INFO:tasks.ceph.mon.h:Stopped 2024-09-17T16:37:27.712 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-17T16:37:27.712 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-17T16:37:27.712 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T16:37:27.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:37:27.726 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T16:37:27.727 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-17T16:37:27.727 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-17T16:37:27.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:37:42.728 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-17T16:37:42.729 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-17T16:37:42.729 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-17T16:37:42.729 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:37:42.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 10 monclient: get_monmap_and_config 2024-09-17T16:37:42.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 10 monclient: build_initial_monmap 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 10 monclient: monmap: 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:42.854310+0000 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:42.854310+0000 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:42.858 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:42.859 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:42.860 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 1 Processor -- start 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 1 -- start start 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.849+0000 7f262ef75640 10 monclient: init 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient: _add_conns ranks=[7,8,0,1,3,4,9,2,10,6,5] 2024-09-17T16:37:42.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-i con 0x7f26280a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-e con 0x7f26280a6cc0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f26281396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280a7b80 con 0x7f26281396a0 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f26280a2ea0 con 0x7f26280a68e0 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f2628069ab0 con 0x7f26280a6cc0 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262ef75640 10 monclient(hunting): authenticate will time out at 1546.120282s 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262d512640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f26281396a0 0x7f2628139b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53466/0 (socket says 172.21.15.105:53466) 2024-09-17T16:37:42.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 learned_addr learned my addr 172.21.15.105:0/2583184898 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f26280a6cc0 auth_method 0 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:42.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f26280a68e0 auth_method 0 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:42.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:42.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:43.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:43.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f26280a6cc0 auth_method 0 2024-09-17T16:37:43.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f26280a68e0 auth_method 0 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:37:43.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:43.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:43.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:43.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.057+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:37:43.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f26280a6cc0 auth_method 0 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f26280a68e0 auth_method 0 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:43.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:43.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:43.461+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:37:44.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f26280a6cc0 auth_method 0 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f26280a68e0 auth_method 0 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:44.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:37:44.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:44.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:44.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:44.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:44.265+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:37:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): tick 2024-09-17T16:37:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:37:45.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 msgr2=0x7f26280a70a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26280a6cc0 0x7f26280a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 msgr2=0x7f26280a48a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26280a68e0 0x7f26280a48a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> v1:172.21.15.105:6789/0 conn(0x7f26281396a0 legacy=0x7f2628139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient: _add_conns ranks=[4,10,3,2,6,9,1,5] 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-c con 0x7f2608003810 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-f con 0x7f2608006260 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:45.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-g con 0x7f2608008da0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f2628069ab0 con 0x7f2608006260 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f260800b9a0 con 0x7f2608008da0 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f260800bba0 con 0x7f2608003810 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f26267fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 msgr2=0x7f2608005c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f2608006260 auth_method 0 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 10 monclient(hunting): get_auth_request con 0x7f2608008da0 auth_method 0 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:45.853+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 msgr2=0x7f2608005c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:37:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f2608006260 auth_method 0 2024-09-17T16:37:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 10 monclient(hunting): get_auth_request con 0x7f2608008da0 auth_method 0 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:37:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.057+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:37:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 msgr2=0x7f2608005c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f2608006260 auth_method 0 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 10 monclient(hunting): get_auth_request con 0x7f2608008da0 auth_method 0 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:37:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:46.461+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:37:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 msgr2=0x7f2608005c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:37:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:47.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 10 monclient(hunting): get_auth_request con 0x7f2608008da0 auth_method 0 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f2608006260 auth_method 0 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:37:47.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:37:47.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:47.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:37:47.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f262d512640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:37:47.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:47.265+0000 7f2627fff640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:37:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): tick 2024-09-17T16:37:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:37:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 msgr2=0x7f2608005c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2608003810 0x7f2608005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 msgr2=0x7f260800b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2608008da0 0x7f260800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 msgr2=0x7f26080086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> v2:172.21.15.160:3300/0 conn(0x7f2608006260 0x7f26080086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient: _add_conns ranks=[2,6,1,5,9] 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 0x7f260800e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-b con 0x7f260800c1f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f260800ec20 0x7f2608011050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-d con 0x7f260800ec20 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2608011780 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280a3620 con 0x7f2608011780 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2628069ab0 con 0x7f260800c1f0 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 1 -- 172.21.15.105:0/2583184898 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f260800b9a0 con 0x7f260800ec20 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f26267fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 reap_dead start 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262d512640 1 -- 172.21.15.105:0/2583184898 reap_dead start 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 0x7f260800e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f260800c1f0 auth_method 0 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient(hunting): handle_auth_done global_id 44858 payload 274 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f260800ec20 msgr2=0x7f2608011050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f260800ec20 0x7f2608011050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 >> v1:172.21.15.160:6789/0 conn(0x7f2608011780 legacy=0x7f2608013c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient: _finish_hunting 0 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 monclient: found mon.noname-b 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.853+0000 7f262cd11640 1 -- 172.21.15.105:0/2583184898 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260801ee50 con 0x7f260800c1f0 2024-09-17T16:37:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262cd11640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 0x7f260800e640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f261800eb30 tx=0x7f261801d4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20ce1a5cf3493ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 1 -- 172.21.15.105:0/2583184898 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26180b3070 con 0x7f260800c1f0 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: dump: 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: _finish_auth 0 2024-09-17T16:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: _check_auth_tickets 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 5 monclient: authenticate success, global_id 44858 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 1 -- 172.21.15.105:0/2583184898 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26180ac780 con 0x7f260800c1f0 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 4 set_mon_vals no callback set 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 1 -- 172.21.15.105:0/2583184898 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f261800f040 con 0x7f260800c1f0 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f26277fe640 10 monclient: dump: 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 10 monclient: shutdown 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 -- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 msgr2=0x7f260800e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 0x7f260800e640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f261800eb30 tx=0x7f261801d4f0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 -- 172.21.15.105:0/2583184898 shutdown_connections 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f260800ec20 0x7f2608011050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 --2- 172.21.15.105:0/2583184898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f260800c1f0 0x7f260800e640 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 -- 172.21.15.105:0/2583184898 >> 172.21.15.105:0/2583184898 conn(0x7f26280a96e0 msgr2=0x7f2628068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 -- 172.21.15.105:0/2583184898 shutdown_connections 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.861+0000 7f262ef75640 1 -- 172.21.15.105:0/2583184898 wait complete. 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: build_initial_monmap 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: monmap: 2024-09-17T16:37:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 librados: starting msgr at 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 librados: starting objecter 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 Processor -- start 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- start start 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 librados: setting wanted keys 2024-09-17T16:37:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 librados: calling monclient init 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: init 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _add_conns ranks=[3,2,1,7,0,8,5,6,4] 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 0x7f26281ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-g con 0x7f260800ec20 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-b con 0x7f2608011780 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 0x7f26281c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): picked mon.noname-f con 0x7f2628138250 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2628139e10 con 0x7f2628138250 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2628139c90 con 0x7f2608011780 2024-09-17T16:37:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2628139f90 con 0x7f260800ec20 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient(hunting): authenticate will time out at 1552.133753s 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43328/0 (socket says 172.21.15.105:43328) 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 -- 172.21.15.105:0/1211523372 learned_addr learned my addr 172.21.15.105:0/1211523372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): get_auth_request con 0x7f2608011780 auth_method 0 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 0x7f26281ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 0x7f26281c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 10 monclient(hunting): get_auth_request con 0x7f260800ec20 auth_method 0 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 10 monclient(hunting): get_auth_request con 0x7f2628138250 auth_method 0 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient(hunting): handle_auth_done global_id 44867 payload 824 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 -- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 msgr2=0x7f26281ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 0x7f26281ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 -- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 msgr2=0x7f26281c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:48.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 0x7f26281c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 monclient: found mon.noname-b 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 -- 172.21.15.105:0/1211523372 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26281cd9b0 con 0x7f2608011780 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262cd11640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 0x7f26281ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262d512640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 0x7f26281c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2627fff640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2610019e80 tx=0x7f2610017780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 client.?.objecter ms_handle_connect 0x7f2608011780 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2610009930 con 0x7f2608011780 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:48.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: dump: 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:48.885 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: _finish_auth 0 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: _check_auth_tickets 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610016a20 con 0x7f2608011780 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2610016c10 con 0x7f2608011780 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f26257fa640 10 monclient: dump: 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:48.886 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 5 monclient: authenticate success, global_id 44867 2024-09-17T16:37:48.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f2626ffd640 4 set_mon_vals no callback set 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _renew_subs 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2628127430 con 0x7f2608011780 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 client.44867.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _renew_subs 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281253b0 con 0x7f2608011780 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.865+0000 7f262ef75640 1 librados: init done 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2610015040 con 0x7f2608011780 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262ef75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262ef75640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262ef75640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:48.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262813a320 con 0x7f2608011780 2024-09-17T16:37:48.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 0x7f260804b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:48.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(385..385 src has 1..385) ==== 5817+0+0 (secure 0 0 0) 0x7f2610012040 con 0x7f2608011780 2024-09-17T16:37:48.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 10 client.44867.objecter ms_dispatch 0x7f2628000ba0 osd_map(385..385 src has 1..385) 2024-09-17T16:37:48.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 3 client.44867.objecter handle_osd_map got epochs [385,385] > 0 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 3 client.44867.objecter handle_osd_map decoding full epoch 385 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 20 client.44867.objecter dump_active .. 0 homeless 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262cd11640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 0x7f260804b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262cd11640 10 monclient: get_auth_request con 0x7f2608049350 auth_method 0 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f26257fa640 10 client.44867.objecter ms_handle_connect 0x7f2608049350 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.869+0000 7f262cd11640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 0x7f260804b810 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f26180b0000 tx=0x7f26180a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.873+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26281cd9b0 con 0x7f2608011780 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.873+0000 7f26257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:48.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:48.873+0000 7f26257fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:49.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f262ef75640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:37:49.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f262ef75640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f262ef75640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f26280a8480 con 0x7f2608011780 2024-09-17T16:37:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f26257fa640 1 -- 172.21.15.105:0/1211523372 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3200 (secure 0 0 0) 0x7f261001e070 con 0x7f2608011780 2024-09-17T16:37:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f26257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:37:49.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.109+0000 7f26257fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:49.120 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":70,"quorum":[0,1,2,3,6,7,8],"quorum_names":["a","f","b","g","d","i","e"],"quorum_leader_name":"d","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:37:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 10 librados: watch_flush enter 2024-09-17T16:37:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 10 librados: watch_flush exit 2024-09-17T16:37:49.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 20 client.44867.objecter shutdown clearing up homeless session... 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 10 client.44867.objecter successfully canceled tick 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 msgr2=0x7f260804b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 0x7f260804b810 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f26180b0000 tx=0x7f26180a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 10 monclient: shutdown 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 msgr2=0x7f2628137d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2610019e80 tx=0x7f2610017780 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 shutdown_connections 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2608049350 0x7f260804b810 unknown :-1 s=CLOSED pgs=3538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2628138250 0x7f26281c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2608011780 0x7f2628137d10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 --2- 172.21.15.105:0/1211523372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f260800ec20 0x7f26281ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 >> 172.21.15.105:0/1211523372 conn(0x7f260800c1f0 msgr2=0x7f2628138b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 shutdown_connections 2024-09-17T16:37:49.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 -- 172.21.15.105:0/1211523372 wait complete. 2024-09-17T16:37:49.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.113+0000 7f262ef75640 1 librados: shutdown 2024-09-17T16:37:49.136 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-17T16:37:49.137 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-17T16:37:49.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: get_monmap_and_config 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: monmap: 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:49.252005+0000 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:49.252005+0000 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:49.256 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 Processor -- start 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 -- start start 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: init 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient: _add_conns ranks=[1,6,8,0,2,9,3,5,4,7,10] 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f26dc0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f26dc0a4910 0x7f26dc13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-d con 0x7f26dc0a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-e con 0x7f26dc0a73d0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:49.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26dc06ebc0 con 0x7f26dc0a68e0 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f26dc0a3fc0 con 0x7f26dc0a4910 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f26dc0a63c0 con 0x7f26dc0a73d0 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26e0fc0640 10 monclient(hunting): authenticate will time out at 1552.517854s 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49848/0 (socket says 172.21.15.105:49848) 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 -- 172.21.15.105:0/2452942125 learned_addr learned my addr 172.21.15.105:0/2452942125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): get_auth_request con 0x7f26dc0a73d0 auth_method 0 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f26dc0a4910 0x7f26dc13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): get_auth_request con 0x7f26dc0a4910 auth_method 0 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26d9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient(hunting): handle_auth_done global_id 54377 payload 274 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 -- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f26dc0a4910 msgr2=0x7f26dc13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f26dc0a4910 0x7f26dc13eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 -- 172.21.15.105:0/2452942125 >> v1:172.21.15.160:6789/0 conn(0x7f26dc0a68e0 legacy=0x7f26dc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 monclient: found mon.noname-e 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:49.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.249+0000 7f26dad9d640 1 -- 172.21.15.105:0/2452942125 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc0a3690 con 0x7f26dc0a73d0 2024-09-17T16:37:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009e80 tx=0x7f26d401c8e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=99fdaa95d1a07adc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 1 -- 172.21.15.105:0/2452942125 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26d40090d0 con 0x7f26dc0a73d0 2024-09-17T16:37:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: dump: 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.270 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 1 -- 172.21.15.105:0/2452942125 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d40a5b60 con 0x7f26dc0a73d0 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 1 -- 172.21.15.105:0/2452942125 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26d40a5d50 con 0x7f26dc0a73d0 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d959a640 10 monclient: dump: 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.271 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.272 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 5 monclient: authenticate success, global_id 54377 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 4 set_mon_vals no callback set 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: shutdown 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 msgr2=0x7f26dc141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009e80 tx=0x7f26d401c8e0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- 172.21.15.105:0/2452942125 shutdown_connections 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26dc0a73d0 0x7f26dc141040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/2452942125 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f26dc0a4910 0x7f26dc13eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- 172.21.15.105:0/2452942125 >> 172.21.15.105:0/2452942125 conn(0x7f26dc068ba0 msgr2=0x7f26dc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- 172.21.15.105:0/2452942125 shutdown_connections 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- 172.21.15.105:0/2452942125 wait complete. 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: monmap: 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.274 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 librados: starting msgr at 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 librados: starting objecter 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 Processor -- start 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- start start 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 librados: setting wanted keys 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 librados: calling monclient init 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: init 2024-09-17T16:37:49.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient: _add_conns ranks=[4,5,1,6,8,0,2,3,7] 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26dc0a4910 0x7f26dc126410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-c con 0x7f26dc0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26dc0a68e0 0x7f26dc126950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-h con 0x7f26dc0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): picked mon.noname-f con 0x7f26dc0a73d0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc143f20 con 0x7f26dc0a73d0 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f26dc143da0 con 0x7f26dc0a4910 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f26dc1440a0 con 0x7f26dc0a68e0 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26e0fc0640 10 monclient(hunting): authenticate will time out at 1552.523173s 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26da59c640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26dc0a4910 msgr2=0x7f26dc126410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26da59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26dc0a4910 0x7f26dc126410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d9d9b640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26dc0a68e0 msgr2=0x7f26dc126950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26d9d9b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26dc0a68e0 0x7f26dc126950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:36342/0 (socket says 172.21.15.105:36342) 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 1 -- 172.21.15.105:0/901037979 learned_addr learned my addr 172.21.15.105:0/901037979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): get_auth_request con 0x7f26dc0a73d0 auth_method 0 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.253+0000 7f26dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 10 monclient(hunting): handle_auth_done global_id 54478 payload 824 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26dc0a68e0 msgr2=0x7f26dc126950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26dc0a68e0 0x7f26dc126950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26dc0a4910 msgr2=0x7f26dc126410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26dc0a4910 0x7f26dc126410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 monclient: found mon.noname-f 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc06de80 con 0x7f26dc0a73d0 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26dad9d640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009040 tx=0x7f26d4019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.?.objecter ms_handle_connect 0x7f26dc0a73d0 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26d40bc020 con 0x7f26dc0a73d0 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: dump: 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.279 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d4003e10 con 0x7f26dc0a73d0 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 5 monclient: authenticate success, global_id 54478 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 monclient: _renew_subs 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc129e60 con 0x7f26dc0a73d0 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26d40044b0 con 0x7f26dc0a73d0 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: dump: 2024-09-17T16:37:49.281 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.282 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26d8d99640 4 set_mon_vals no callback set 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 client.54478.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 monclient: _renew_subs 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc12a140 con 0x7f26dc0a73d0 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 1 librados: init done 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 0x7f26dc1d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc051e80 con 0x7f26dc1d0fb0 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 0x7f26dc1d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient: get_auth_request con 0x7f26dc1d0fb0 auth_method 0 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 10 monclient(hunting): handle_auth_done global_id 35784 payload 824 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26dc0a20f0 con 0x7f26dc1d0fb0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26da59c640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 0x7f26dc1d1390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c450 tx=0x7f26cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.54478.objecter ms_handle_connect 0x7f26dc1d0fb0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.54478.objecter resend_mon_ops 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f26d40bf020 con 0x7f26dc0a73d0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26bc03e480 0x7f26bc040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26cc0057d0 con 0x7f26dc1d0fb0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: dump: 2024-09-17T16:37:49.284 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.285 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(385..385 src has 1..385) ==== 5817+0+0 (secure 0 0 0) 0x7f26d40b3040 con 0x7f26dc0a73d0 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.54478.objecter ms_dispatch 0x7f26dc000ba0 osd_map(385..385 src has 1..385) 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 3 client.54478.objecter handle_osd_map got epochs [385,385] > 0 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 3 client.54478.objecter handle_osd_map decoding full epoch 385 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 20 client.54478.objecter dump_active .. 0 homeless 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f26cc005970 con 0x7f26dc1d0fb0 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 msgr2=0x7f26dc1d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 0x7f26dc1d1390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c450 tx=0x7f26cc00c910 comp rx=0 tx=0).stop 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 msgr2=0x7f26dc1d1390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d0fb0 0x7f26dc1d1390 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c450 tx=0x7f26cc00c910 comp rx=0 tx=0).stop 2024-09-17T16:37:49.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26d9d9b640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26bc03e480 0x7f26bc040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26d9d9b640 10 monclient: get_auth_request con 0x7f26bc03e480 auth_method 0 2024-09-17T16:37:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26caffd640 10 client.54478.objecter ms_handle_connect 0x7f26bc03e480 2024-09-17T16:37:49.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.257+0000 7f26d9d9b640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26bc03e480 0x7f26bc040940 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f26dc128450 tx=0x7f26d0009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 0x7f26dc122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc0a6cc0 con 0x7f26dc1d18d0 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 0x7f26dc122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient: get_auth_request con 0x7f26dc1d18d0 auth_method 0 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 10 monclient(hunting): handle_auth_done global_id 35793 payload 824 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 1 -- 172.21.15.105:0/901037979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f26dc1292d0 con 0x7f26dc1d18d0 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26da59c640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 0x7f26dc122530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26cc000ee0 tx=0x7f26cc001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 client.54478.objecter ms_handle_connect 0x7f26dc1d18d0 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 client.54478.objecter resend_mon_ops 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26cc019cf0 con 0x7f26dc1d18d0 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 monclient: dump: 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9432 (secure 0 0 0) 0x7f26cc019880 con 0x7f26dc1d18d0 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 msgr2=0x7f26dc122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 0x7f26dc122530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26cc000ee0 tx=0x7f26cc001160 comp rx=0 tx=0).stop 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 msgr2=0x7f26dc122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26caffd640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26dc1d18d0 0x7f26dc122530 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26cc000ee0 tx=0x7f26cc001160 comp rx=0 tx=0).stop 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:49.292 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 460068, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.293 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.294 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:49.296 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.297 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:49.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:49.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.305 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.306 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.310 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.311 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:49.313 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.314 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:49.315 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.317 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:49.317 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:49.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 librados: watch_flush enter 2024-09-17T16:37:49.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 librados: watch_flush exit 2024-09-17T16:37:49.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 20 client.54478.objecter shutdown clearing up homeless session... 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 client.54478.objecter successfully canceled tick 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26bc03e480 msgr2=0x7f26bc040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26bc03e480 0x7f26bc040940 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7f26dc128450 tx=0x7f26d0009190 comp rx=0 tx=0).stop 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 10 monclient: shutdown 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 msgr2=0x7f26dc126e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009040 tx=0x7f26d4019680 comp rx=0 tx=0).stop 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26dad9d640 1 -- 172.21.15.105:0/901037979 reap_dead start 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 shutdown_connections 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 --2- 172.21.15.105:0/901037979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26dc0a73d0 0x7f26dc126e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 >> 172.21.15.105:0/901037979 conn(0x7f26dc068ba0 msgr2=0x7f26dc09fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 shutdown_connections 2024-09-17T16:37:49.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 -- 172.21.15.105:0/901037979 wait complete. 2024-09-17T16:37:49.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.265+0000 7f26e0fc0640 1 librados: shutdown 2024-09-17T16:37:49.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:49.320 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-17T16:37:49.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:37:49.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.433+0000 7f702c455640 10 monclient: get_monmap_and_config 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.433+0000 7f702c455640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.433+0000 7f702c455640 10 monclient: monmap: 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:49.439719+0000 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:49.439719+0000 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:49.443 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:49.444 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 Processor -- start 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 -- start start 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient: init 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient: _add_conns ranks=[3,9,0,4,1,7,5,10,2,6,8] 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70240a3720 0x7f70240a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): picked mon.noname-g con 0x7f70240a3720 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70240a3b00 0x7f70240a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): picked mon.noname-a con 0x7f70240a3b00 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70240a3f00 addr v1:172.21.15.105:6789/0 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702406db50 con 0x7f70240a3f00 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f70240a0e80 con 0x7f70240a3b00 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f70240a1490 con 0x7f70240a3720 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702c455640 10 monclient(hunting): authenticate will time out at 1552.705438s 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70240a3b00 0x7f70240a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a9f2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f70240a3f00 0x7f70240a5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44846/0 (socket says 172.21.15.105:44846) 2024-09-17T16:37:49.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a9f2640 1 -- 172.21.15.105:0/325682520 learned_addr learned my addr 172.21.15.105:0/325682520 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): get_auth_request con 0x7f70240a3b00 auth_method 0 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a1f1640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70240a3720 0x7f70240a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a1f1640 10 monclient(hunting): get_auth_request con 0x7f70240a3720 auth_method 0 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a1f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a1f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f702a1f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient(hunting): handle_auth_done global_id 35802 payload 274 2024-09-17T16:37:49.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 -- 172.21.15.105:0/325682520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70240a3720 msgr2=0x7f70240a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70240a3720 0x7f70240a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 -- 172.21.15.105:0/325682520 >> v1:172.21.15.105:6789/0 conn(0x7f70240a3f00 legacy=0x7f70240a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 monclient: found mon.noname-a 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.437+0000 7f70299f0640 1 -- 172.21.15.105:0/325682520 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70240a7240 con 0x7f70240a3b00 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70299f0640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3b00 0x7f70240a56c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f70140097c0 tx=0x7f701401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715324a7cdcc4d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 1 -- 172.21.15.105:0/325682520 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70140b0070 con 0x7f70240a3b00 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:49.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: dump: 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.459 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 1 -- 172.21.15.105:0/325682520 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70140aa810 con 0x7f70240a3b00 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 1 -- 172.21.15.105:0/325682520 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70140a9300 con 0x7f70240a3b00 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 5 monclient: authenticate success, global_id 35802 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:49.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 4 set_mon_vals no callback set 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70291ef640 10 monclient: dump: 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.461 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: shutdown 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/325682520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3b00 msgr2=0x7f70240a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3b00 0x7f70240a56c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f70140097c0 tx=0x7f701401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/325682520 shutdown_connections 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3b00 0x7f70240a56c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- 172.21.15.105:0/325682520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70240a3720 0x7f70240a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/325682520 >> 172.21.15.105:0/325682520 conn(0x7f70240aa390 msgr2=0x7f70240a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/325682520 shutdown_connections 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/325682520 wait complete. 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: monmap: 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:49.463 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 librados: starting msgr at 2024-09-17T16:37:49.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 librados: starting objecter 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 Processor -- start 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- start start 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 librados: setting wanted keys 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 librados: calling monclient init 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: init 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _add_conns ranks=[1,4,0,3,7,2,5,8,6] 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70240a3720 0x7f702413e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): picked mon.noname-f con 0x7f70240a3720 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70240a3b00 0x7f702413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): picked mon.noname-c con 0x7f70240a3b00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): picked mon.noname-a con 0x7f70240a3f00 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:49.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70241442e0 con 0x7f70240a3f00 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f70240a6520 con 0x7f70240a3720 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7024144460 con 0x7f70240a3b00 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient(hunting): authenticate will time out at 1552.710457s 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70299f0640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70240a3b00 msgr2=0x7f702413ed50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f70299f0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70240a3b00 0x7f702413ed50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43690/0 (socket says 172.21.15.105:43690) 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 -- 172.21.15.105:0/4284427939 learned_addr learned my addr 172.21.15.105:0/4284427939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): get_auth_request con 0x7f70240a3f00 auth_method 0 2024-09-17T16:37:49.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a1f1640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70240a3720 0x7f702413e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a1f1640 10 monclient(hunting): get_auth_request con 0x7f70240a3720 auth_method 0 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a1f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a1f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a1f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient(hunting): handle_auth_done global_id 35820 payload 824 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70240a3b00 msgr2=0x7f702413ed50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:49.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f70240a3b00 0x7f702413ed50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70240a3720 msgr2=0x7f702413e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70240a3720 0x7f702413e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 monclient: found mon.noname-a 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70241257c0 con 0x7f70240a3f00 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702a9f2640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f702000c3f0 tx=0x7f702000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 client.?.objecter ms_handle_connect 0x7f70240a3f00 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7020017070 con 0x7f70240a3f00 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: dump: 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.469 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70200070d0 con 0x7f70240a3f00 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 5 monclient: authenticate success, global_id 35820 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _renew_subs 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7024125aa0 con 0x7f70240a3f00 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70200076f0 con 0x7f70240a3f00 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 client.35820.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701affd640 10 monclient: dump: 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.471 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f701bfff640 4 set_mon_vals no callback set 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _renew_subs 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7024069c10 con 0x7f70240a3f00 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.441+0000 7f702c455640 1 librados: init done 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702c455640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702c455640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 0x7f7024071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7024051e80 con 0x7f7024071090 2024-09-17T16:37:49.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 0x7f7024071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient: get_auth_request con 0x7f7024071090 auth_method 0 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7020007280 con 0x7f70240a3f00 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f700803e580 0x7f7008040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 10 monclient(hunting): handle_auth_done global_id 44876 payload 824 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70241402d0 con 0x7f7024071090 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(385..385 src has 1..385) ==== 5817+0+0 (secure 0 0 0) 0x7f702004ee50 con 0x7f70240a3f00 2024-09-17T16:37:49.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 client.35820.objecter ms_dispatch 0x7f7024000ba0 osd_map(385..385 src has 1..385) 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 3 client.35820.objecter handle_osd_map got epochs [385,385] > 0 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 3 client.35820.objecter handle_osd_map decoding full epoch 385 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 20 client.35820.objecter dump_active .. 0 homeless 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 client.35820.objecter ms_handle_connect 0x7f7024071090 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 client.35820.objecter resend_mon_ops 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f702a1f1640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 0x7f7024071470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f701c0045c0 tx=0x7f701c007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f70299f0640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f700803e580 0x7f7008040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f70299f0640 10 monclient: get_auth_request con 0x7f700803e580 auth_method 0 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f701c014880 con 0x7f7024071090 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 monclient: dump: 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.474 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.475 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f701c019070 con 0x7f7024071090 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 msgr2=0x7f7024071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 0x7f7024071470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f701c0045c0 tx=0x7f701c007700 comp rx=0 tx=0).stop 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 msgr2=0x7f7024071470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7024071090 0x7f7024071470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f701affd640 10 client.35820.objecter ms_handle_connect 0x7f700803e580 2024-09-17T16:37:49.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.445+0000 7f70299f0640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f700803e580 0x7f7008040a40 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f70140ad000 tx=0x7f7014008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702c455640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702c455640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 0x7f70241db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702c455640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7024124930 con 0x7f70240719b0 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 0x7f70241db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient: get_auth_request con 0x7f70240719b0 auth_method 0 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.449+0000 7f702a1f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702a1f1640 10 monclient(hunting): handle_auth_done global_id 44885 payload 824 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702a1f1640 1 -- 172.21.15.105:0/4284427939 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70240a1e70 con 0x7f70240719b0 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 client.35820.objecter ms_handle_connect 0x7f70240719b0 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 client.35820.objecter resend_mon_ops 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702a1f1640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 0x7f70241db3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f701c0045c0 tx=0x7f701c004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f701c01e260 con 0x7f70240719b0 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 monclient: dump: 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.479 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9432 (secure 0 0 0) 0x7f701c01e260 con 0x7f70240719b0 2024-09-17T16:37:49.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 msgr2=0x7f70241db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 0x7f70241db3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f701c0045c0 tx=0x7f701c004a50 comp rx=0 tx=0).stop 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 msgr2=0x7f70241db3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f701affd640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70240719b0 0x7f70241db3c0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f701c0045c0 tx=0x7f701c004a50 comp rx=0 tx=0).stop 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:49.481 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 320105, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.483 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.484 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:49.485 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.486 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.491 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:49.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.496 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.499 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.501 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.502 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:49.503 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 10 librados: watch_flush enter 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 10 librados: watch_flush exit 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 20 client.35820.objecter shutdown clearing up homeless session... 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 10 client.35820.objecter successfully canceled tick 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f700803e580 msgr2=0x7f7008040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f700803e580 0x7f7008040a40 secure :-1 s=READY pgs=3540 cs=0 l=1 rev1=1 crypto rx=0x7f70140ad000 tx=0x7f7014008040 comp rx=0 tx=0).stop 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 10 monclient: shutdown 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 msgr2=0x7f702413f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702a9f2640 1 -- 172.21.15.105:0/4284427939 reap_dead start 2024-09-17T16:37:49.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f702000c3f0 tx=0x7f702000c7f0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 shutdown_connections 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 --2- 172.21.15.105:0/4284427939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70240a3f00 0x7f702413f290 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 >> 172.21.15.105:0/4284427939 conn(0x7f70240aa390 msgr2=0x7f70240a8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 shutdown_connections 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 -- 172.21.15.105:0/4284427939 wait complete. 2024-09-17T16:37:49.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.453+0000 7f702c455640 1 librados: shutdown 2024-09-17T16:37:49.506 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:49.506 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-17T16:37:49.506 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:37:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: get_monmap_and_config 2024-09-17T16:37:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: monmap: 2024-09-17T16:37:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.623 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:49.620315+0000 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:49.620315+0000 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.624 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:49.625 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 Processor -- start 2024-09-17T16:37:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 -- start start 2024-09-17T16:37:49.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: init 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient: _add_conns ranks=[3,2,10,8,7,0,1,9,4,5,6] 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac0a68e0 0x7f32ac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-g con 0x7f32ac0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-b con 0x7f32ac0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f32ac1396a0 0x7f32ac139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-f con 0x7f32ac1396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f32ac0a3fc0 con 0x7f32ac1396a0 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0a63c0 con 0x7f32ac0a4910 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0a7740 con 0x7f32ac0a68e0 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b38b1640 10 monclient(hunting): authenticate will time out at 1552.886036s 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43338/0 (socket says 172.21.15.105:43338) 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3655297185 learned_addr learned my addr 172.21.15.105:0/3655297185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): get_auth_request con 0x7f32ac0a4910 auth_method 0 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b164d640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac0a68e0 0x7f32ac0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b1e4e640 1 --2- 172.21.15.105:0/3655297185 >> v2:172.21.15.160:3300/0 conn(0x7f32ac1396a0 0x7f32ac139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b164d640 10 monclient(hunting): get_auth_request con 0x7f32ac0a68e0 auth_method 0 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b164d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b164d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b164d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b1e4e640 10 monclient(hunting): get_auth_request con 0x7f32ac1396a0 auth_method 0 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b1e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b1e4e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b1e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_done global_id 44894 payload 274 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3655297185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac0a68e0 msgr2=0x7f32ac0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac0a68e0 0x7f32ac0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3655297185 >> v2:172.21.15.160:3300/0 conn(0x7f32ac1396a0 msgr2=0x7f32ac139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3655297185 >> v2:172.21.15.160:3300/0 conn(0x7f32ac1396a0 0x7f32ac139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 monclient: found mon.noname-b 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.617+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3655297185 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac06ebc0 con 0x7f32ac0a4910 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f329c009870 tx=0x7f329c01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295c6486bc26901a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 1 -- 172.21.15.105:0/3655297185 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f329c0b0070 con 0x7f32ac0a4910 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: dump: 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 1 -- 172.21.15.105:0/3655297185 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c0aa810 con 0x7f32ac0a4910 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 5 monclient: authenticate success, global_id 44894 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 1 -- 172.21.15.105:0/3655297185 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f329c0a9300 con 0x7f32ac0a4910 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32a3fff640 10 monclient: dump: 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 4 set_mon_vals no callback set 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: shutdown 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- 172.21.15.105:0/3655297185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 msgr2=0x7f32ac0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f329c009870 tx=0x7f329c01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- 172.21.15.105:0/3655297185 shutdown_connections 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3655297185 >> v2:172.21.15.160:3300/0 conn(0x7f32ac1396a0 0x7f32ac139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32ac0a4910 0x7f32ac0a7200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3655297185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac0a68e0 0x7f32ac0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- 172.21.15.105:0/3655297185 >> 172.21.15.105:0/3655297185 conn(0x7f32ac068ba0 msgr2=0x7f32ac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- 172.21.15.105:0/3655297185 shutdown_connections 2024-09-17T16:37:49.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- 172.21.15.105:0/3655297185 wait complete. 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: monmap: 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.646 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 librados: starting msgr at 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 librados: starting objecter 2024-09-17T16:37:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 Processor -- start 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- start start 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 librados: setting wanted keys 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 librados: calling monclient init 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: init 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient: _add_conns ranks=[0,6,3,5,1,2,8,7,4] 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32ac0a4910 0x7f32ac137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-a con 0x7f32ac0a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 0x7f32ac138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-d con 0x7f32ac0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac1396a0 0x7f32ac13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): picked mon.noname-g con 0x7f32ac1396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac06e630 con 0x7f32ac0a4910 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac0a7af0 con 0x7f32ac1396a0 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f32ac143320 con 0x7f32ac0a68e0 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b38b1640 10 monclient(hunting): authenticate will time out at 1552.891251s 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 0x7f32ac138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 0x7f32ac138500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:49664/0 (socket says 172.21.15.105:49664) 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3509997084 learned_addr learned my addr 172.21.15.105:0/3509997084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): get_auth_request con 0x7f32ac0a68e0 auth_method 0 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32ac0a4910 0x7f32ac137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): get_auth_request con 0x7f32ac0a4910 auth_method 0 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b1e4e640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac1396a0 0x7f32ac13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b1e4e640 10 monclient(hunting): get_auth_request con 0x7f32ac1396a0 auth_method 0 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b1e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b1e4e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b1e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient(hunting): handle_auth_done global_id 24549 payload 824 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac1396a0 msgr2=0x7f32ac13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32ac1396a0 0x7f32ac13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32ac0a4910 msgr2=0x7f32ac137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32ac0a4910 0x7f32ac137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 monclient: found mon.noname-d 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.621+0000 7f32b0e4c640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac13fcd0 con 0x7f32ac0a68e0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b0e4c640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 0x7f32ac138500 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ae040 tx=0x7f329c019190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.?.objecter ms_handle_connect 0x7f32ac0a68e0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f329c0b9070 con 0x7f32ac0a68e0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: dump: 2024-09-17T16:37:49.652 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c0044b0 con 0x7f32ac0a68e0 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f329c0b0040 con 0x7f32ac0a68e0 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: dump: 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a2ffd640 4 set_mon_vals no callback set 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 5 monclient: authenticate success, global_id 24549 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 monclient: _renew_subs 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac123b70 con 0x7f32ac0a68e0 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 client.24549.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 monclient: _renew_subs 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac077bf0 con 0x7f32ac0a68e0 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 1 librados: init done 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 0x7f32ac13dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f32ac122650 con 0x7f32ac13dc10 2024-09-17T16:37:49.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 0x7f32ac13dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient: get_auth_request con 0x7f32ac13dc10 auth_method 0 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f329c004040 con 0x7f32ac0a68e0 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 10 monclient(hunting): handle_auth_done global_id 24558 payload 824 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32ac0aa070 con 0x7f32ac13dc10 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f329003e550 0x7f3290040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(385..385 src has 1..385) ==== 5817+0+0 (secure 0 0 0) 0x7f329c0e9340 con 0x7f32ac0a68e0 2024-09-17T16:37:49.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.24549.objecter ms_dispatch 0x7f32ac000ba0 osd_map(385..385 src has 1..385) 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 3 client.24549.objecter handle_osd_map got epochs [385,385] > 0 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 3 client.24549.objecter handle_osd_map decoding full epoch 385 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b164d640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 0x7f32ac13dff0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a800bc40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 20 client.24549.objecter dump_active .. 0 homeless 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.24549.objecter ms_handle_connect 0x7f32ac13dc10 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.24549.objecter resend_mon_ops 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32a8018070 con 0x7f32ac13dc10 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: dump: 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b1e4e640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f329003e550 0x7f3290040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b1e4e640 10 monclient: get_auth_request con 0x7f329003e550 auth_method 0 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f32a80108a0 con 0x7f32ac13dc10 2024-09-17T16:37:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 msgr2=0x7f32ac13dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 0x7f32ac13dff0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a800bc40 comp rx=0 tx=0).stop 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 msgr2=0x7f32ac13dff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13dc10 0x7f32ac13dff0 secure :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a800bc40 comp rx=0 tx=0).stop 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32a1ffb640 10 client.24549.objecter ms_handle_connect 0x7f329003e550 2024-09-17T16:37:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.625+0000 7f32b1e4e640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f329003e550 0x7f3290040a10 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7f32ac13ed40 tx=0x7f32a400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 0x7f32ac0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f32ac051e80 con 0x7f32ac13e530 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 0x7f32ac0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient: get_auth_request con 0x7f32ac13e530 auth_method 0 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 10 monclient(hunting): handle_auth_done global_id 24567 payload 824 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 1 -- 172.21.15.105:0/3509997084 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f32ac139ae0 con 0x7f32ac13e530 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 client.24549.objecter ms_handle_connect 0x7f32ac13e530 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 client.24549.objecter resend_mon_ops 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b164d640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 0x7f32ac0a0a20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a8004740 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32a801dd80 con 0x7f32ac13e530 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 monclient: dump: 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f32a8014430 con 0x7f32ac13e530 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 msgr2=0x7f32ac0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 0x7f32ac0a0a20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a8004740 comp rx=0 tx=0).stop 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 msgr2=0x7f32ac0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32a1ffb640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac13e530 0x7f32ac0a0a20 secure :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b780 tx=0x7f32a8004740 comp rx=0 tx=0).stop 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 617272, 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:49.666 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.667 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.668 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:49.669 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.670 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:49.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.672 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.673 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.674 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.675 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.676 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.677 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.678 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.680 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.681 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.682 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.683 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.684 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:49.685 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.686 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.687 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.688 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:37:49.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.689 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:49.689 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 librados: watch_flush enter 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 librados: watch_flush exit 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 20 client.24549.objecter shutdown clearing up homeless session... 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 client.24549.objecter successfully canceled tick 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f329003e550 msgr2=0x7f3290040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f329003e550 0x7f3290040a10 secure :-1 s=READY pgs=3541 cs=0 l=1 rev1=1 crypto rx=0x7f32ac13ed40 tx=0x7f32a400b040 comp rx=0 tx=0).stop 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 10 monclient: shutdown 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 msgr2=0x7f32ac138500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 --2- 172.21.15.105:0/3509997084 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f32ac0a68e0 0x7f32ac138500 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ae040 tx=0x7f329c019190 comp rx=0 tx=0).stop 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b1e4e640 1 -- 172.21.15.105:0/3509997084 reap_dead start 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b1e4e640 1 -- 172.21.15.105:0/3509997084 reap_dead start 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 shutdown_connections 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 >> 172.21.15.105:0/3509997084 conn(0x7f32ac068ba0 msgr2=0x7f32ac0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 shutdown_connections 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 -- 172.21.15.105:0/3509997084 wait complete. 2024-09-17T16:37:49.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.633+0000 7f32b38b1640 1 librados: shutdown 2024-09-17T16:37:49.692 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:49.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-17T16:37:49.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:37:49.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: get_monmap_and_config 2024-09-17T16:37:49.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: monmap: 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:49.809093+0000 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:49.809093+0000 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.813 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.814 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.814 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:49.814 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:49.814 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 Processor -- start 2024-09-17T16:37:49.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 -- start start 2024-09-17T16:37:49.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: init 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient: _add_conns ranks=[4,7,9,5,10,1,8,2,0,3,6] 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f900a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-c con 0x7f0f900a5860 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0f900a18f0 0x7f0f900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-i con 0x7f0f900a18f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:49.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f900a4070 0x7f0f90137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-a con 0x7f0f900a4070 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0f900a66c0 con 0x7f0f900a4070 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0f900a1d70 con 0x7f0f900a5860 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f8f7fe640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 msgr2=0x7f0f900a5c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0f900a33d0 con 0x7f0f900a18f0 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f95fbb640 10 monclient(hunting): authenticate will time out at 1553.075873s 2024-09-17T16:37:49.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.805+0000 7f0f8f7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f900a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f900a4070 0x7f0f90137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0f900a4070 0x7f0f90137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43700/0 (socket says 172.21.15.105:43700) 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 -- 172.21.15.105:0/747361065 learned_addr learned my addr 172.21.15.105:0/747361065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request con 0x7f0f900a4070 auth_method 0 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f87fff640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0f900a18f0 0x7f0f900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient(hunting): handle_auth_done global_id 35838 payload 274 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 -- 172.21.15.105:0/747361065 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0f900a18f0 msgr2=0x7f0f900a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0f900a18f0 0x7f0f900a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 -- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 msgr2=0x7f0f900a5c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f900a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 monclient: found mon.noname-a 2024-09-17T16:37:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:49.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 -- 172.21.15.105:0/747361065 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f900a36b0 con 0x7f0f900a4070 2024-09-17T16:37:49.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8ffff640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f900a4070 0x7f0f90137610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009e50 tx=0x7f0f8801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76daf1ec6518396 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 1 -- 172.21.15.105:0/747361065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f880ac020 con 0x7f0f900a4070 2024-09-17T16:37:49.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:49.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: dump: 2024-09-17T16:37:49.837 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.837 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.837 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.838 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.839 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 1 -- 172.21.15.105:0/747361065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f88009110 con 0x7f0f900a4070 2024-09-17T16:37:49.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 1 -- 172.21.15.105:0/747361065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f880a6d00 con 0x7f0f900a4070 2024-09-17T16:37:49.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 5 monclient: authenticate success, global_id 35838 2024-09-17T16:37:49.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 4 set_mon_vals no callback set 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f8effd640 10 monclient: dump: 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.843 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.843 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.843 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.843 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.844 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 10 monclient: shutdown 2024-09-17T16:37:49.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 -- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f900a4070 msgr2=0x7f0f90137610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f900a4070 0x7f0f90137610 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009e50 tx=0x7f0f8801c680 comp rx=0 tx=0).stop 2024-09-17T16:37:49.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 -- 172.21.15.105:0/747361065 shutdown_connections 2024-09-17T16:37:49.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f900a4070 0x7f0f90137610 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0f900a18f0 0x7f0f900a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/747361065 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f900a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 -- 172.21.15.105:0/747361065 >> 172.21.15.105:0/747361065 conn(0x7f0f900aa450 msgr2=0x7f0f900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 -- 172.21.15.105:0/747361065 shutdown_connections 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 1 -- 172.21.15.105:0/747361065 wait complete. 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.809+0000 7f0f95fbb640 10 monclient: build_initial_monmap 2024-09-17T16:37:49.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: monmap: 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:49.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:49.849 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:49.849 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:49.849 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:49.850 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 librados: starting msgr at 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 librados: starting objecter 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 Processor -- start 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- start start 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 librados: setting wanted keys 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 librados: calling monclient init 2024-09-17T16:37:49.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: init 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _add_conns ranks=[8,1,4,6,3,5,2,7,0] 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 0x7f0f90135cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-e con 0x7f0f900a18f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f900a4070 0x7f0f90136230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-f con 0x7f0f900a4070 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:37:49.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f90136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): picked mon.noname-c con 0x7f0f900a5860 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90141480 con 0x7f0f900a4070 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0f90141300 con 0x7f0f900a5860 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0f90141600 con 0x7f0f900a18f0 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): authenticate will time out at 1553.080882s 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8ffff640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 msgr2=0x7f0f90136770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f90136770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:49.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 0x7f0f90135cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 0x7f0f90135cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49850/0 (socket says 172.21.15.105:49850) 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 -- 172.21.15.105:0/2251705768 learned_addr learned my addr 172.21.15.105:0/2251705768 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): get_auth_request con 0x7f0f900a18f0 auth_method 0 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f900a4070 0x7f0f90136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient(hunting): handle_auth_done global_id 54386 payload 824 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 msgr2=0x7f0f90136770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0f900a5860 0x7f0f90136770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f900a4070 msgr2=0x7f0f90136230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f900a4070 0x7f0f90136230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 monclient: found mon.noname-e 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9013c310 con 0x7f0f900a18f0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient: get_auth_request con 0x7f0f900a4070 auth_method 0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f900a4070 0x7f0f90136230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8f7fe640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 0x7f0f90135cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800b550 tx=0x7f0f7800ba10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 client.?.objecter ms_handle_connect 0x7f0f900a18f0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f780092d0 con 0x7f0f900a18f0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:49.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: dump: 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: _finish_auth 0 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f78015070 con 0x7f0f900a18f0 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f78003c30 con 0x7f0f900a18f0 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 5 monclient: authenticate success, global_id 54386 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _renew_subs 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8e7fc640 4 set_mon_vals no callback set 2024-09-17T16:37:49.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9013ce90 con 0x7f0f900a18f0 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: dump: 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.858 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 client.54386.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _renew_subs 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f900a6190 con 0x7f0f900a18f0 2024-09-17T16:37:49.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 librados: init done 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 0x7f0f9006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0f90051e80 con 0x7f0f9006d220 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0f780050f0 con 0x7f0f900a18f0 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 0x7f0f9006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient: get_auth_request con 0x7f0f9006d220 auth_method 0 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f7003e550 0x7f0f70040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(385..385 src has 1..385) ==== 5817+0+0 (secure 0 0 0) 0x7f0f7804fac0 con 0x7f0f900a18f0 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 client.54386.objecter ms_dispatch 0x7f0f90000ba0 osd_map(385..385 src has 1..385) 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 3 client.54386.objecter handle_osd_map got epochs [385,385] > 0 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 3 client.54386.objecter handle_osd_map decoding full epoch 385 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 10 monclient(hunting): handle_auth_done global_id 54395 payload 824 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f9013c780 con 0x7f0f9006d220 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 20 client.54386.objecter dump_active .. 0 homeless 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 client.54386.objecter ms_handle_connect 0x7f0f9006d220 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 client.54386.objecter resend_mon_ops 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 0x7f0f9006d600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f9006e980 tx=0x7f0f8000dbf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8ffff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f7003e550 0x7f0f70040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f80016070 con 0x7f0f9006d220 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8ffff640 10 monclient: get_auth_request con 0x7f0f7003e550 auth_method 0 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: dump: 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0f80008700 con 0x7f0f9006d220 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 msgr2=0x7f0f9006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 0x7f0f9006d600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f9006e980 tx=0x7f0f8000dbf0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 msgr2=0x7f0f9006d600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.813+0000 7f0f8cff9640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006d220 0x7f0f9006d600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.817+0000 7f0f8cff9640 10 client.54386.objecter ms_handle_connect 0x7f0f7003e550 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.817+0000 7f0f8ffff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f7003e550 0x7f0f70040a10 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ad000 tx=0x7f0f880a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 0x7f0f9009e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0f90137610 con 0x7f0f9006db40 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 0x7f0f9009e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient: get_auth_request con 0x7f0f9006db40 auth_method 0 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:49.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 10 monclient(hunting): handle_auth_done global_id 54404 payload 824 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 1 -- 172.21.15.105:0/2251705768 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0f9006eb80 con 0x7f0f9006db40 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 client.54386.objecter ms_handle_connect 0x7f0f9006db40 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 client.54386.objecter resend_mon_ops 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f87fff640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 0x7f0f9009e280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f9006e980 tx=0x7f0f80004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0f8001b8c0 con 0x7f0f9006db40 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 monclient: dump: 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.865 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9599 (secure 0 0 0) 0x7f0f800113c0 con 0x7f0f9006db40 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:49.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 msgr2=0x7f0f9009e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 0x7f0f9009e280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f9006e980 tx=0x7f0f80004740 comp rx=0 tx=0).stop 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 msgr2=0x7f0f9009e280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8cff9640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f9006db40 0x7f0f9009e280 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f9006e980 tx=0x7f0f80004740 comp rx=0 tx=0).stop 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:49.867 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 176819, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:49.868 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.869 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:49.871 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:49.872 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:49.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.876 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:49.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:49.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.881 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.883 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.885 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:49.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.889 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:49.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:49.891 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:49.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 librados: watch_flush enter 2024-09-17T16:37:49.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 librados: watch_flush exit 2024-09-17T16:37:49.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 20 client.54386.objecter shutdown clearing up homeless session... 2024-09-17T16:37:49.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 client.54386.objecter successfully canceled tick 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f7003e550 msgr2=0x7f0f70040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0f7003e550 0x7f0f70040a10 secure :-1 s=READY pgs=3542 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ad000 tx=0x7f0f880a7000 comp rx=0 tx=0).stop 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 10 monclient: shutdown 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 msgr2=0x7f0f90135cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 --2- 172.21.15.105:0/2251705768 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0f900a18f0 0x7f0f90135cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800b550 tx=0x7f0f7800ba10 comp rx=0 tx=0).stop 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8ffff640 1 -- 172.21.15.105:0/2251705768 reap_dead start 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f8ffff640 1 -- 172.21.15.105:0/2251705768 reap_dead start 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 shutdown_connections 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 >> 172.21.15.105:0/2251705768 conn(0x7f0f900aa450 msgr2=0x7f0f900a7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 shutdown_connections 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 -- 172.21.15.105:0/2251705768 wait complete. 2024-09-17T16:37:49.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:49.821+0000 7f0f95fbb640 1 librados: shutdown 2024-09-17T16:37:49.895 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:49.895 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-17T16:37:49.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:37:50.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: get_monmap_and_config 2024-09-17T16:37:50.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: monmap: 2024-09-17T16:37:50.015 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.015 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:50.012374+0000 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:50.012374+0000 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.016 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.017 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:50.017 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:50.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 Processor -- start 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 -- start start 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: init 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient: _add_conns ranks=[3,4,8,10,2,9,7,1,0,5,6] 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8e00a68e0 0x7fa8e00a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-g con 0x7fa8e00a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:50.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 0x7fa8e00a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-c con 0x7fa8e00a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-e con 0x7fa8e01396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00a3fc0 con 0x7fa8e00a68e0 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa8e00a63c0 con 0x7fa8e00a4910 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa8e00a7740 con 0x7fa8e01396a0 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e72e3640 10 monclient(hunting): authenticate will time out at 1553.278152s 2024-09-17T16:37:50.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e487e640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 msgr2=0x7fa8e00a7200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e487e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 0x7fa8e00a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49878/0 (socket says 172.21.15.105:49878) 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 -- 172.21.15.105:0/1207550754 learned_addr learned my addr 172.21.15.105:0/1207550754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): get_auth_request con 0x7fa8e01396a0 auth_method 0 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e507f640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8e00a68e0 0x7fa8e00a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e507f640 10 monclient(hunting): get_auth_request con 0x7fa8e00a68e0 auth_method 0 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e507f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e507f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e507f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient(hunting): handle_auth_done global_id 54413 payload 274 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 -- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 msgr2=0x7fa8e00a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 0x7fa8e00a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 -- 172.21.15.105:0/1207550754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8e00a68e0 msgr2=0x7fa8e00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8e00a68e0 0x7fa8e00a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 monclient: found mon.noname-e 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.009+0000 7fa8e5880640 1 -- 172.21.15.105:0/1207550754 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e006ec10 con 0x7fa8e01396a0 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e5880640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098d0 tx=0x7fa8d801cec0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=99e026a82e95cf97 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 1 -- 172.21.15.105:0/1207550754 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d80b0070 con 0x7fa8e01396a0 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: dump: 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.032 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.033 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 1 -- 172.21.15.105:0/1207550754 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d80aa810 con 0x7fa8e01396a0 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 1 -- 172.21.15.105:0/1207550754 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d80a9300 con 0x7fa8e01396a0 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 5 monclient: authenticate success, global_id 54413 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 4 set_mon_vals no callback set 2024-09-17T16:37:50.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d7fff640 10 monclient: dump: 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.035 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: shutdown 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 msgr2=0x7fa8e0139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80098d0 tx=0x7fa8d801cec0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- 172.21.15.105:0/1207550754 shutdown_connections 2024-09-17T16:37:50.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa8e01396a0 0x7fa8e0139ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e00a4910 0x7fa8e00a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/1207550754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8e00a68e0 0x7fa8e00a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- 172.21.15.105:0/1207550754 >> 172.21.15.105:0/1207550754 conn(0x7fa8e0068ba0 msgr2=0x7fa8e0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- 172.21.15.105:0/1207550754 shutdown_connections 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- 172.21.15.105:0/1207550754 wait complete. 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: monmap: 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:50.037 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 librados: starting msgr at 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 librados: starting objecter 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 Processor -- start 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- start start 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 librados: setting wanted keys 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 librados: calling monclient init 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: init 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient: _add_conns ranks=[5,2,4,7,0,6,1,8,3] 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa8e00a4910 0x7fa8e0138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-h con 0x7fa8e00a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 0x7fa8e01387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-b con 0x7fa8e00a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:50.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e01396a0 0x7fa8e013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): picked mon.noname-c con 0x7fa8e01396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e0143320 con 0x7fa8e00a68e0 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa8e006e630 con 0x7fa8e01396a0 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa8e01434a0 con 0x7fa8e00a4910 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e72e3640 10 monclient(hunting): authenticate will time out at 1553.283284s 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e5880640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e01396a0 msgr2=0x7fa8e013d7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e5880640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e01396a0 0x7fa8e013d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:50.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 0x7fa8e01387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e507f640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa8e00a4910 msgr2=0x7fa8e0138280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e507f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa8e00a4910 0x7fa8e0138280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 0x7fa8e01387c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43342/0 (socket says 172.21.15.105:43342) 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 -- 172.21.15.105:0/3838455202 learned_addr learned my addr 172.21.15.105:0/3838455202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): get_auth_request con 0x7fa8e00a68e0 auth_method 0 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient(hunting): handle_auth_done global_id 44903 payload 824 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa8e00a4910 msgr2=0x7fa8e0138280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa8e00a4910 0x7fa8e0138280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e01396a0 msgr2=0x7fa8e013d7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa8e01396a0 0x7fa8e013d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 monclient: found mon.noname-b 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e013f290 con 0x7fa8e00a68e0 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8e487e640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 0x7fa8e01387c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000b9e0 tx=0x7fa8d000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d5ffb640 10 client.?.objecter ms_handle_connect 0x7fa8e00a68e0 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.013+0000 7fa8d5ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d000ce10 con 0x7fa8e00a68e0 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: dump: 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.043 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d0010070 con 0x7fa8e00a68e0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 5 monclient: authenticate success, global_id 44903 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d00158e0 con 0x7fa8e00a68e0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 monclient: _renew_subs 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e00aa0e0 con 0x7fa8e00a68e0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:50.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d77fe640 4 set_mon_vals no callback set 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: dump: 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.046 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 client.44903.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 monclient: _renew_subs 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e0138d00 con 0x7fa8e00a68e0 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 1 librados: init done 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 0x7fa8e013dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0051e80 con 0x7fa8e0122790 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa8d0003d10 con 0x7fa8e00a68e0 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 0x7fa8e013dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa8cc03e550 0x7fa8cc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient: get_auth_request con 0x7fa8e0122790 auth_method 0 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(386..386 src has 1..386) ==== 6189+0+0 (secure 0 0 0) 0x7fa8d004fcb0 con 0x7fa8e00a68e0 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 client.44903.objecter ms_dispatch 0x7fa8e0000ba0 osd_map(386..386 src has 1..386) 2024-09-17T16:37:50.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 3 client.44903.objecter handle_osd_map got epochs [386,386] > 0 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 3 client.44903.objecter handle_osd_map decoding full epoch 386 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 20 client.44903.objecter dump_active .. 0 homeless 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e5880640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa8cc03e550 0x7fa8cc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e5880640 10 monclient: get_auth_request con 0x7fa8cc03e550 auth_method 0 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 10 monclient(hunting): handle_auth_done global_id 54505 payload 824 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8e0077ca0 con 0x7fa8e0122790 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 client.44903.objecter ms_handle_connect 0x7fa8e0122790 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 client.44903.objecter resend_mon_ops 2024-09-17T16:37:50.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e507f640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 0x7fa8e013dd30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc00eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 client.44903.objecter ms_handle_connect 0x7fa8cc03e550 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8e5880640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa8cc03e550 0x7fa8cc040a10 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80ad000 tx=0x7fa8d8004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8dc017020 con 0x7fa8e0122790 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: dump: 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.050 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.051 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa8dc00ecd0 con 0x7fa8e0122790 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 msgr2=0x7fa8e013dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 0x7fa8e013dd30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc00eba0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 msgr2=0x7fa8e013dd30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.017+0000 7fa8d5ffb640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e0122790 0x7fa8e013dd30 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc00eba0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e72e3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 0x7fa8e013e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e72e3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e0139ae0 con 0x7fa8e013e270 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 0x7fa8e013e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient: get_auth_request con 0x7fa8e013e270 auth_method 0 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 10 monclient(hunting): handle_auth_done global_id 54514 payload 824 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 1 -- 172.21.15.105:0/3838455202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa8e013ebc0 con 0x7fa8e013e270 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 10 client.44903.objecter ms_handle_connect 0x7fa8e013e270 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 10 client.44903.objecter resend_mon_ops 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8e507f640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 0x7fa8e013e680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8dc017040 con 0x7fa8e013e270 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.025+0000 7fa8d5ffb640 10 monclient: dump: 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.055 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.056 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9265 (secure 0 0 0) 0x7fa8dc013430 con 0x7fa8e013e270 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 msgr2=0x7fa8e013e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 0x7fa8e013e680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc004740 comp rx=0 tx=0).stop 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 msgr2=0x7fa8e013e680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8d5ffb640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8e013e270 0x7fa8e013e680 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc000ee0 tx=0x7fa8dc004740 comp rx=0 tx=0).stop 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 177034, 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:50.058 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.059 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.060 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.061 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:50.062 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.063 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:50.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:50.066 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.067 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.069 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.072 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.073 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.074 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.076 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.077 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.078 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.079 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.080 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.082 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.084 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:50.084 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:50.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 10 librados: watch_flush enter 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 10 librados: watch_flush exit 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 20 client.44903.objecter shutdown clearing up homeless session... 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 10 client.44903.objecter successfully canceled tick 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa8cc03e550 msgr2=0x7fa8cc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa8cc03e550 0x7fa8cc040a10 secure :-1 s=READY pgs=3543 cs=0 l=1 rev1=1 crypto rx=0x7fa8d80ad000 tx=0x7fa8d8004740 comp rx=0 tx=0).stop 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 10 monclient: shutdown 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 msgr2=0x7fa8e01387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 --2- 172.21.15.105:0/3838455202 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8e00a68e0 0x7fa8e01387c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000b9e0 tx=0x7fa8d000bea0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e5880640 1 -- 172.21.15.105:0/3838455202 reap_dead start 2024-09-17T16:37:50.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e5880640 1 -- 172.21.15.105:0/3838455202 reap_dead start 2024-09-17T16:37:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 shutdown_connections 2024-09-17T16:37:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 >> 172.21.15.105:0/3838455202 conn(0x7fa8e0068ba0 msgr2=0x7fa8e00a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 shutdown_connections 2024-09-17T16:37:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 -- 172.21.15.105:0/3838455202 wait complete. 2024-09-17T16:37:50.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.029+0000 7fa8e72e3640 1 librados: shutdown 2024-09-17T16:37:50.087 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:50.087 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-17T16:37:50.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:37:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: get_monmap_and_config 2024-09-17T16:37:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: monmap: 2024-09-17T16:37:50.203 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:50.200416+0000 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:50.200416+0000 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.204 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:50.205 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 Processor -- start 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 -- start start 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: init 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient: _add_conns ranks=[3,2,5,8,4,6,10,0,1,7,9] 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 0x7fc5980a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-g con 0x7fc5980a3720 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-b con 0x7fc5980a3b00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 0x7fc5980a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-h con 0x7fc5980a3f00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc59806db50 con 0x7fc5980a3b00 2024-09-17T16:37:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980a0e80 con 0x7fc5980a3720 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc5980a1490 con 0x7fc5980a3f00 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc5a00e7640 10 monclient(hunting): authenticate will time out at 1553.466193s 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59e684640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 msgr2=0x7fc5980a5c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59e684640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 0x7fc5980a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:43354/0 (socket says 172.21.15.105:43354) 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 -- 172.21.15.105:0/3554454595 learned_addr learned my addr 172.21.15.105:0/3554454595 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): get_auth_request con 0x7fc5980a3b00 auth_method 0 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 0x7fc5980a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 10 monclient(hunting): get_auth_request con 0x7fc5980a3720 auth_method 0 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient(hunting): handle_auth_done global_id 44912 payload 274 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 -- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 msgr2=0x7fc5980a5c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 0x7fc5980a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 -- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 msgr2=0x7fc5980a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 0x7fc5980a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 monclient: found mon.noname-b 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59d682640 1 -- 172.21.15.105:0/3554454595 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5980a7240 con 0x7fc5980a3b00 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.197+0000 7fc59de83640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 0x7fc5980a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5880098a0 tx=0x7fc58801c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65797d428038ed0e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 1 -- 172.21.15.105:0/3554454595 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5880b0070 con 0x7fc5980a3b00 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: dump: 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 1 -- 172.21.15.105:0/3554454595 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5880aa810 con 0x7fc5980a3b00 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 5 monclient: authenticate success, global_id 44912 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 1 -- 172.21.15.105:0/3554454595 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5880a9300 con 0x7fc5980a3b00 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 4 set_mon_vals no callback set 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59ce81640 10 monclient: dump: 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.220 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: shutdown 2024-09-17T16:37:50.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- 172.21.15.105:0/3554454595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 msgr2=0x7fc5980a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc5880098a0 tx=0x7fc58801c460 comp rx=0 tx=0).stop 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- 172.21.15.105:0/3554454595 shutdown_connections 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc5980a3f00 0x7fc5980a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc5980a3b00 0x7fc5980a56c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/3554454595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3720 0x7fc5980a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- 172.21.15.105:0/3554454595 >> 172.21.15.105:0/3554454595 conn(0x7fc5980aa390 msgr2=0x7fc5980a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- 172.21.15.105:0/3554454595 shutdown_connections 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- 172.21.15.105:0/3554454595 wait complete. 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: monmap: 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 librados: starting msgr at 2024-09-17T16:37:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 librados: starting objecter 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 Processor -- start 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- start start 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 librados: setting wanted keys 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 librados: calling monclient init 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: init 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient: _add_conns ranks=[0,8,3,2,5,4,7,1,6] 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980a3720 0x7fc598139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-a con 0x7fc5980a3720 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 0x7fc59813a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-e con 0x7fc5980a3b00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3f00 0x7fc59813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): picked mon.noname-g con 0x7fc5980a3f00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5981432d0 con 0x7fc5980a3720 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980a6550 con 0x7fc5980a3f00 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc598143450 con 0x7fc5980a3b00 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc5a00e7640 10 monclient(hunting): authenticate will time out at 1553.471234s 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 0x7fc59813a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980a3720 0x7fc598139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 0x7fc59813a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49890/0 (socket says 172.21.15.105:49890) 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 -- 172.21.15.105:0/2027824577 learned_addr learned my addr 172.21.15.105:0/2027824577 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): get_auth_request con 0x7fc5980a3b00 auth_method 0 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): get_auth_request con 0x7fc5980a3720 auth_method 0 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient(hunting): handle_auth_done global_id 54422 payload 824 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59e684640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3f00 0x7fc59813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3f00 msgr2=0x7fc59813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5980a3f00 0x7fc59813a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980a3720 msgr2=0x7fc598139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980a3720 0x7fc598139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 monclient: found mon.noname-e 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5980a81c0 con 0x7fc5980a3b00 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59de83640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5980a3720 0x7fc598139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc59d682640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 0x7fc59813a260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5880ae040 tx=0x7fc588019200 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc58e7fc640 10 client.?.objecter ms_handle_connect 0x7fc5980a3b00 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.201+0000 7fc58e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5880b9070 con 0x7fc5980a3b00 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: dump: 2024-09-17T16:37:50.227 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 5 monclient: authenticate success, global_id 54422 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5880044b0 con 0x7fc5980a3b00 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 monclient: _renew_subs 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc598126060 con 0x7fc5980a3b00 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5880b0040 con 0x7fc5980a3b00 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58f7fe640 4 set_mon_vals no callback set 2024-09-17T16:37:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 client.54422.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: dump: 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 monclient: _renew_subs 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5981251f0 con 0x7fc5980a3b00 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 1 librados: init done 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc588004040 con 0x7fc5980a3b00 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 0x7fc59813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc598051e80 con 0x7fc59813cd30 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc57c03e580 0x7fc57c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(386..386 src has 1..386) ==== 6189+0+0 (secure 0 0 0) 0x7fc5880e9420 con 0x7fc5980a3b00 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 client.54422.objecter ms_dispatch 0x7fc598000ba0 osd_map(386..386 src has 1..386) 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 3 client.54422.objecter handle_osd_map got epochs [386,386] > 0 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 3 client.54422.objecter handle_osd_map decoding full epoch 386 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 0x7fc59813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 20 client.54422.objecter dump_active .. 0 homeless 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient: get_auth_request con 0x7fc59813cd30 auth_method 0 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59e684640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc57c03e580 0x7fc57c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59e684640 10 monclient: get_auth_request con 0x7fc57c03e580 auth_method 0 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 10 monclient(hunting): handle_auth_done global_id 54453 payload 824 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5980a0230 con 0x7fc59813cd30 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 client.54422.objecter ms_handle_connect 0x7fc59813cd30 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 client.54422.objecter resend_mon_ops 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59de83640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 0x7fc59813d110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc594011990 tx=0x7fc594011e50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 client.54422.objecter ms_handle_connect 0x7fc57c03e580 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc59e684640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc57c03e580 0x7fc57c040a40 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7fc5980a08a0 tx=0x7fc590008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc594019020 con 0x7fc59813cd30 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: dump: 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.233 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.234 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc594004300 con 0x7fc59813cd30 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 msgr2=0x7fc59813d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 0x7fc59813d110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc594011990 tx=0x7fc594011e50 comp rx=0 tx=0).stop 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 msgr2=0x7fc59813d110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.205+0000 7fc58e7fc640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813cd30 0x7fc59813d110 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc594011990 tx=0x7fc594011e50 comp rx=0 tx=0).stop 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc5a00e7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 0x7fc59809e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc5a00e7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980a1e70 con 0x7fc59813d650 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 0x7fc59809e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient: get_auth_request con 0x7fc59813d650 auth_method 0 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 10 monclient(hunting): handle_auth_done global_id 54462 payload 824 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 1 -- 172.21.15.105:0/2027824577 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc598125ce0 con 0x7fc59813d650 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 client.54422.objecter ms_handle_connect 0x7fc59813d650 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 client.54422.objecter resend_mon_ops 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc59de83640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 0x7fc59809e740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc594011e90 tx=0x7fc5940046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc594019030 con 0x7fc59813d650 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 monclient: dump: 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.238 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9265 (secure 0 0 0) 0x7fc594003830 con 0x7fc59813d650 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:50.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 msgr2=0x7fc59809e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 0x7fc59809e740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc594011e90 tx=0x7fc5940046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 msgr2=0x7fc59809e740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.213+0000 7fc58e7fc640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc59813d650 0x7fc59809e740 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc594011e90 tx=0x7fc5940046d0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:50.240 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 177222, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.241 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.242 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:50.243 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:50.244 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:50.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:50.247 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.253 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:50.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:50.260 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:50.261 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 10 librados: watch_flush enter 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 10 librados: watch_flush exit 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 20 client.54422.objecter shutdown clearing up homeless session... 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 10 client.54422.objecter successfully canceled tick 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc57c03e580 msgr2=0x7fc57c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc57c03e580 0x7fc57c040a40 secure :-1 s=READY pgs=3544 cs=0 l=1 rev1=1 crypto rx=0x7fc5980a08a0 tx=0x7fc590008040 comp rx=0 tx=0).stop 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 10 monclient: shutdown 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 msgr2=0x7fc59813a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 --2- 172.21.15.105:0/2027824577 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc5980a3b00 0x7fc59813a260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5880ae040 tx=0x7fc588019200 comp rx=0 tx=0).stop 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc59e684640 1 -- 172.21.15.105:0/2027824577 reap_dead start 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc59e684640 1 -- 172.21.15.105:0/2027824577 reap_dead start 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 shutdown_connections 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 >> 172.21.15.105:0/2027824577 conn(0x7fc5980aa390 msgr2=0x7fc5980a91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 shutdown_connections 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 -- 172.21.15.105:0/2027824577 wait complete. 2024-09-17T16:37:50.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.217+0000 7fc5a00e7640 1 librados: shutdown 2024-09-17T16:37:50.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:50.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-17T16:37:50.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:37:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: get_monmap_and_config 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: monmap: 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:37:50.380670+0000 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:37:50.380670+0000 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:37:50.385 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 Processor -- start 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 -- start start 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: init 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient: _add_conns ranks=[7,4,9,10,6,2,0,8,1,3,5] 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-i con 0x7f9b3c0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 0x7f9b3c1397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-c con 0x7f9b3c0a7350 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b3c0a7730 0x7f9b3c139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-a con 0x7f9b3c0a7730 addr v2:172.21.15.105:3300/0 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9b3c0a3ac0 con 0x7f9b3c0a7730 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9b3c0a4080 con 0x7f9b3c0a7350 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9b3c0a6480 con 0x7f9b3c0a6a30 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4316f640 10 monclient(hunting): authenticate will time out at 1553.646970s 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b3bfff640 1 -- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 msgr2=0x7f9b3c1397f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3302/0 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 0x7f9b3c1397f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b3c0a7730 0x7f9b3c139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b3c0a7730 0x7f9b3c139d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43714/0 (socket says 172.21.15.105:43714) 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 -- 172.21.15.105:0/1849597214 learned_addr learned my addr 172.21.15.105:0/1849597214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): get_auth_request con 0x7f9b3c0a7730 auth_method 0 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 10 monclient(hunting): get_auth_request con 0x7f9b3c0a6a30 auth_method 0 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient(hunting): handle_auth_done global_id 35856 payload 274 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 -- 172.21.15.105:0/1849597214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 msgr2=0x7f9b3c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 -- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 msgr2=0x7f9b3c1397f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:37:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 0x7f9b3c1397f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 monclient: found mon.noname-a 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b4170c640 1 -- 172.21.15.105:0/1849597214 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c0a37e0 con 0x7f9b3c0a7730 2024-09-17T16:37:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.377+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7730 0x7f9b3c139d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009870 tx=0x7f9b2c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa59c5212dfe1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 1 -- 172.21.15.105:0/1849597214 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b2c0b0070 con 0x7f9b3c0a7730 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: dump: 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.399 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 1 -- 172.21.15.105:0/1849597214 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2c0aa810 con 0x7f9b3c0a7730 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 5 monclient: authenticate success, global_id 35856 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 10 monclient: get_monmap_and_config success 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 1 -- 172.21.15.105:0/1849597214 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b2c0a9300 con 0x7f9b3c0a7730 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 4 set_mon_vals no callback set 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b3b7fe640 10 monclient: dump: 2024-09-17T16:37:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 10 monclient: shutdown 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 -- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7730 msgr2=0x7f9b3c139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7730 0x7f9b3c139d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009870 tx=0x7f9b2c01c440 comp rx=0 tx=0).stop 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 -- 172.21.15.105:0/1849597214 shutdown_connections 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7730 0x7f9b3c139d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b3c0a7350 0x7f9b3c1397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1849597214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 -- 172.21.15.105:0/1849597214 >> 172.21.15.105:0/1849597214 conn(0x7f9b3c0a8b30 msgr2=0x7f9b3c0a8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 -- 172.21.15.105:0/1849597214 shutdown_connections 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 -- 172.21.15.105:0/1849597214 wait complete. 2024-09-17T16:37:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 10 monclient: build_initial_monmap 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 10 monclient: monmap: 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:37:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 librados: starting msgr at 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.381+0000 7f9b4316f640 1 librados: starting objecter 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 Processor -- start 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- start start 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 librados: setting wanted keys 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 librados: calling monclient init 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: init 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _reopen_session rank -1 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _add_conns ranks=[7,0,2,1,8,4,6,5,3] 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-i con 0x7f9b3c0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:37:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 0x7f9b3c1392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-a con 0x7f9b3c0a7350 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b3c0a7730 0x7f9b3c13e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): picked mon.noname-b con 0x7f9b3c0a7730 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c0a4c70 con 0x7f9b3c0a7350 2024-09-17T16:37:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c13a680 con 0x7f9b3c0a7730 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9b3c143470 con 0x7f9b3c0a6a30 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): _renew_subs 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): authenticate will time out at 1553.652722s 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 0x7f9b3c1392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b3c0a7730 0x7f9b3c13e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 0x7f9b3c1392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43716/0 (socket says 172.21.15.105:43716) 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 1 -- 172.21.15.105:0/1178302492 learned_addr learned my addr 172.21.15.105:0/1178302492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): get_auth_request con 0x7f9b3c0a7350 auth_method 0 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): get_auth_request con 0x7f9b3c0a7730 auth_method 0 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient(hunting): handle_auth_done global_id 44921 payload 824 2024-09-17T16:37:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 msgr2=0x7f9b3c138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 msgr2=0x7f9b3c1392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 0x7f9b3c1392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient: _finish_hunting 0 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 monclient: found mon.noname-b 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c122a50 con 0x7f9b3c0a7730 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient: get_auth_request con 0x7f9b3c0a6a30 auth_method 0 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c0a6a30 0x7f9b3c138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b3c0a7350 0x7f9b3c1392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4170c640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b3c0a7730 0x7f9b3c13e0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0ab000 tx=0x7f9b2c0191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 client.?.objecter ms_handle_connect 0x7f9b3c0a7730 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b2c0b0040 con 0x7f9b3c0a7730 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: dump: 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: _finish_auth 0 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: _check_auth_tickets 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2c0044b0 con 0x7f9b3c0a7730 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 5 monclient: authenticate success, global_id 44921 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _renew_subs 2024-09-17T16:37:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c06def0 con 0x7f9b3c0a7730 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b2c0aac70 con 0x7f9b3c0a7730 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 client.44921.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 monclient: dump: 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3affd640 4 set_mon_vals no callback set 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _renew_subs 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c123110 con 0x7f9b3c0a7730 2024-09-17T16:37:50.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 librados: init done 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 0x7f9b3c13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9b3c051e80 con 0x7f9b3c124590 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9b2c004040 con 0x7f9b3c0a7730 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 0x7f9b3c13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient: get_auth_request con 0x7f9b3c124590 auth_method 0 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b2803e580 0x7f9b28040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(386..386 src has 1..386) ==== 6189+0+0 (secure 0 0 0) 0x7f9b2c0e8fc0 con 0x7f9b3c0a7730 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 client.44921.objecter ms_dispatch 0x7f9b3c000ba0 osd_map(386..386 src has 1..386) 2024-09-17T16:37:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 3 client.44921.objecter handle_osd_map got epochs [386,386] > 0 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 3 client.44921.objecter handle_osd_map decoding full epoch 386 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 20 client.44921.objecter dump_active .. 0 homeless 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b2803e580 0x7f9b28040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b3bfff640 10 monclient: get_auth_request con 0x7f9b2803e580 auth_method 0 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_done global_id 64249 payload 824 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b3c0a2010 con 0x7f9b3c124590 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 client.44921.objecter ms_handle_connect 0x7f9b3c124590 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b397fa640 10 client.44921.objecter resend_mon_ops 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.385+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 0x7f9b3c13e610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009f10 tx=0x7f9b3400e510 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 client.44921.objecter ms_handle_connect 0x7f9b2803e580 2024-09-17T16:37:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b3bfff640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b2803e580 0x7f9b28040a40 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c13f830 tx=0x7f9b300091c0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b34015880 con 0x7f9b3c124590 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 monclient: dump: 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.415 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9b3401a070 con 0x7f9b3c124590 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 msgr2=0x7f9b3c13e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 0x7f9b3c13e610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009f10 tx=0x7f9b3400e510 comp rx=0 tx=0).stop 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 msgr2=0x7f9b3c13e610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.389+0000 7f9b397fa640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c124590 0x7f9b3c13e610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b4316f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 0x7f9b3c13ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b4316f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9b3c06a200 con 0x7f9b3c13eb50 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 0x7f9b3c13ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient: get_auth_request con 0x7f9b3c13eb50 auth_method 0 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:37:50.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 10 monclient(hunting): handle_auth_done global_id 64258 payload 824 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.393+0000 7f9b40f0b640 1 -- 172.21.15.105:0/1178302492 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9b3c0a6e10 con 0x7f9b3c13eb50 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 client.44921.objecter ms_handle_connect 0x7f9b3c13eb50 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b40f0b640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 0x7f9b3c13ef30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b34019000 tx=0x7f9b34004a50 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 client.44921.objecter resend_mon_ops 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b34015c00 con 0x7f9b3c13eb50 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 monclient: dump: 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:37:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:37:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9264 (secure 0 0 0) 0x7f9b3401f030 con 0x7f9b3c13eb50 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 msgr2=0x7f9b3c13ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 0x7f9b3c13ef30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b34019000 tx=0x7f9b34004a50 comp rx=0 tx=0).stop 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 msgr2=0x7f9b3c13ef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b397fa640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b3c13eb50 0x7f9b3c13ef30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 70, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 43495, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:37:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:37:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.428 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.429 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.431 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:37:50.433 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.434 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.436 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.437 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.439 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:37:50.441 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:37:50.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:37:50.443 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:37:50.443 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:37:50.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 10 librados: watch_flush enter 2024-09-17T16:37:50.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 10 librados: watch_flush exit 2024-09-17T16:37:50.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 20 client.44921.objecter shutdown clearing up homeless session... 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 10 client.44921.objecter successfully canceled tick 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b2803e580 msgr2=0x7f9b28040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b2803e580 0x7f9b28040a40 secure :-1 s=READY pgs=3545 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c13f830 tx=0x7f9b300091c0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 10 monclient: shutdown 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b3c0a7730 msgr2=0x7f9b3c13e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 --2- 172.21.15.105:0/1178302492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b3c0a7730 0x7f9b3c13e0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0ab000 tx=0x7f9b2c0191e0 comp rx=0 tx=0).stop 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4170c640 1 -- 172.21.15.105:0/1178302492 reap_dead start 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4170c640 1 -- 172.21.15.105:0/1178302492 reap_dead start 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 shutdown_connections 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 >> 172.21.15.105:0/1178302492 conn(0x7f9b3c0a8b30 msgr2=0x7f9b3c140340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 shutdown_connections 2024-09-17T16:37:50.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 -- 172.21.15.105:0/1178302492 wait complete. 2024-09-17T16:37:50.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:37:50.397+0000 7f9b4316f640 1 librados: shutdown 2024-09-17T16:37:50.446 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:37:50.446 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:38:12.988 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:38:12.981+0000 7f51dd0f4640 -1 snap_mapper.add_oid found existing snaps mapped on 300:7177081d:test-rados-api-smithi160-29504-80::foo:2, removing 2024-09-17T16:38:12.989 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-17T16:38:12.981+0000 7f5738e23640 -1 snap_mapper.add_oid found existing snaps mapped on 300:7177081d:test-rados-api-smithi160-29504-80::foo:2, removing 2024-09-17T16:38:44.000 INFO:tasks.workunit.client.0.smithi160.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18026 ms) 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (27188 ms) 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3030 ms) 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3020 ms) 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3060 ms) 2024-09-17T16:38:44.001 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3049 ms) 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (22583 ms) 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38284 ms) 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16751 ms) 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.002 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (3982 ms) 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3018 ms) 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25041 ms) 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14419 ms) 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (8547 ms) 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-17T16:38:44.003 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (23591 ms) 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrubs... 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25027 ms) 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10109 ms) 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9004 ms) 2024-09-17T16:38:44.004 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-17T16:38:44.005 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:38:44.005 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (8951 ms) 2024-09-17T16:38:44.005 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-17T16:39:20.446 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-17T16:39:20.447 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-17T16:39:20.447 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-17T16:39:20.451 INFO:tasks.ceph.mon.h:Started 2024-09-17T16:39:20.451 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-17T16:39:20.451 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T16:39:20.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T16:39:20.454 INFO:tasks.ceph.mon.c:Started 2024-09-17T16:39:20.454 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-17T16:39:20.454 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-17T16:39:20.454 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:39:20.559 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:39:20.553+0000 7ff568f96e00 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 401 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:39:35.455 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-17T16:39:35.456 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-17T16:39:35.456 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:39:35.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:35.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.601+0000 7f6f5c1c5640 10 monclient: get_monmap_and_config 2024-09-17T16:39:35.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.601+0000 7f6f5c1c5640 10 monclient: build_initial_monmap 2024-09-17T16:39:35.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.601+0000 7f6f5c1c5640 10 monclient: monmap: 2024-09-17T16:39:35.611 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:35.611 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.611 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:35.607456+0000 2024-09-17T16:39:35.612 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:35.607456+0000 2024-09-17T16:39:35.612 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:35.612 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.612 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:35.612 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:35.613 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 Processor -- start 2024-09-17T16:39:35.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 -- start start 2024-09-17T16:39:35.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient: init 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient: _add_conns ranks=[6,9,3,1,2,5,8,0,7,10,4] 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-d con 0x7f6f540a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 0x7f6f540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-a con 0x7f6f540a18f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a4070 0x7f6f54137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-g con 0x7f6f540a4070 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6f540a66c0 con 0x7f6f540a18f0 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6f540a1d70 con 0x7f6f540a4070 2024-09-17T16:39:35.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6f540a33d0 con 0x7f6f540a5860 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient(hunting): authenticate will time out at 1658.873088s 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 0x7f6f540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33718/0 (socket says 172.21.15.105:33718) 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 -- 172.21.15.105:0/3869161059 learned_addr learned my addr 172.21.15.105:0/3869161059 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): get_auth_request con 0x7f6f540a5860 auth_method 0 2024-09-17T16:39:35.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 0x7f6f540a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39518/0 (socket says 172.21.15.105:39518) 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): get_auth_request con 0x7f6f540a18f0 auth_method 0 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:35.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59760640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient(hunting): handle_auth_done global_id 24576 payload 274 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 -- 172.21.15.105:0/3869161059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a4070 msgr2=0x7f6f54137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 --2- 172.21.15.105:0/3869161059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a4070 0x7f6f54137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 -- 172.21.15.105:0/3869161059 >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 msgr2=0x7f6f540a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 --2- 172.21.15.105:0/3869161059 >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 0x7f6f540a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient: _finish_hunting 0 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 monclient: found mon.noname-d 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 -- 172.21.15.105:0/3869161059 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f540a36b0 con 0x7f6f540a5860 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f59f61640 1 --2- 172.21.15.105:0/3869161059 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009870 tx=0x7f6f4401c160 comp rx=0 tx=0).ready entity=mon.6 client_cookie=29e296980e8ffe33 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 1 -- 172.21.15.105:0/3869161059 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f440b0070 con 0x7f6f540a5860 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: dump: 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:35.629 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:35.630 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:35.631 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:35.631 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:35.631 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:35.631 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:35.631 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: _finish_auth 0 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: _check_auth_tickets 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 1 -- 172.21.15.105:0/3869161059 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f440aa810 con 0x7f6f540a5860 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 5 monclient: authenticate success, global_id 24576 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:35.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 1 -- 172.21.15.105:0/3869161059 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f440a9300 con 0x7f6f540a5860 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f5c1c5640 4 set_mon_vals no callback set 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.605+0000 7f6f58f5f640 10 monclient: dump: 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:35.633 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: shutdown 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/3869161059 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 msgr2=0x7f6f540a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/3869161059 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009870 tx=0x7f6f4401c160 comp rx=0 tx=0).stop 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/3869161059 shutdown_connections 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/3869161059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a4070 0x7f6f54137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/3869161059 >> v2:172.21.15.105:3300/0 conn(0x7f6f540a18f0 0x7f6f540a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/3869161059 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6f540a5860 0x7f6f540a5c40 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/3869161059 >> 172.21.15.105:0/3869161059 conn(0x7f6f540aa450 msgr2=0x7f6f540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/3869161059 shutdown_connections 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/3869161059 wait complete. 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: build_initial_monmap 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: monmap: 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:35.635 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 librados: starting msgr at 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 librados: starting objecter 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 Processor -- start 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- start start 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 librados: setting wanted keys 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 librados: calling monclient init 2024-09-17T16:39:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: init 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _add_conns ranks=[3,5,4,2,1,6,0,7,8] 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a18f0 0x7f6f54136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-g con 0x7f6f540a18f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f540a4070 0x7f6f541365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-h con 0x7f6f540a4070 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): picked mon.noname-c con 0x7f6f540a5860 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6f54141480 con 0x7f6f540a18f0 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6f54141300 con 0x7f6f540a5860 2024-09-17T16:39:35.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6f54141600 con 0x7f6f540a4070 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient(hunting): authenticate will time out at 1658.878047s 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40238/0 (socket says 172.21.15.105:40238) 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 -- 172.21.15.105:0/641050546 learned_addr learned my addr 172.21.15.105:0/641050546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): get_auth_request con 0x7f6f540a5860 auth_method 0 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:35.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59760640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f540a4070 0x7f6f541365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59f61640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a18f0 0x7f6f54136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59f61640 10 monclient(hunting): get_auth_request con 0x7f6f540a18f0 auth_method 0 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59f61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59f61640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59f61640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59760640 10 monclient(hunting): get_auth_request con 0x7f6f540a4070 auth_method 0 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59760640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59760640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f59760640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient(hunting): handle_auth_done global_id 74101 payload 824 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 -- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f540a4070 msgr2=0x7f6f541365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f540a4070 0x7f6f541365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 -- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a18f0 msgr2=0x7f6f54136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a18f0 0x7f6f54136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient: _finish_hunting 0 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 monclient: found mon.noname-c 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 -- 172.21.15.105:0/641050546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f540a05e0 con 0x7f6f540a5860 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5a762640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00b9a0 tx=0x7f6f4c00be60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:35.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 client.?.objecter ms_handle_connect 0x7f6f540a5860 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f4c019070 con 0x7f6f540a5860 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: dump: 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.641 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4c00ce10 con 0x7f6f540a5860 2024-09-17T16:39:35.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f4c0077e0 con 0x7f6f540a5860 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 5 monclient: authenticate success, global_id 74101 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _renew_subs 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5413cab0 con 0x7f6f540a5860 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4bfff640 4 set_mon_vals no callback set 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 10 monclient: dump: 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:35.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 client.74101.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _renew_subs 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5413d980 con 0x7f6f540a5860 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 librados: init done 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5413d0f0 con 0x7f6f540a5860 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.609+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6f4c007370 con 0x7f6f540a5860 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 0x7f6f38040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(445..445 src has 1..445) ==== 6230+0+0 (secure 0 0 0) 0x7f6f4c04fae0 con 0x7f6f540a5860 2024-09-17T16:39:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 10 client.74101.objecter ms_dispatch 0x7f6f54000ba0 osd_map(445..445 src has 1..445) 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 3 client.74101.objecter handle_osd_map got epochs [445,445] > 0 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 3 client.74101.objecter handle_osd_map decoding full epoch 445 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 20 client.74101.objecter dump_active .. 0 homeless 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f59f61640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 0x7f6f38040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f59f61640 10 monclient: get_auth_request con 0x7f6f3803e550 auth_method 0 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f4a7fc640 10 client.74101.objecter ms_handle_connect 0x7f6f3803e550 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.613+0000 7f6f59f61640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 0x7f6f38040a10 secure :-1 s=READY pgs=3548 cs=0 l=1 rev1=1 crypto rx=0x7f6f440ad000 tx=0x7f6f440a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.617+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f540a05e0 con 0x7f6f540a5860 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.617+0000 7f6f4a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.617+0000 7f6f4a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:35.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:39:35.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:35.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:35.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6f54137610 con 0x7f6f540a5860 2024-09-17T16:39:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f4a7fc640 1 -- 172.21.15.105:0/641050546 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f6f4c01c360 con 0x7f6f540a5860 2024-09-17T16:39:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f4a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:35.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f4a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:35.861 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 librados: watch_flush enter 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 librados: watch_flush exit 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 20 client.74101.objecter shutdown clearing up homeless session... 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 client.74101.objecter successfully canceled tick 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 msgr2=0x7f6f38040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 0x7f6f38040a10 secure :-1 s=READY pgs=3548 cs=0 l=1 rev1=1 crypto rx=0x7f6f440ad000 tx=0x7f6f440a6000 comp rx=0 tx=0).stop 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 10 monclient: shutdown 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 msgr2=0x7f6f5406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00b9a0 tx=0x7f6f4c00be60 comp rx=0 tx=0).stop 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 shutdown_connections 2024-09-17T16:39:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6f3803e550 0x7f6f38040a10 unknown :-1 s=CLOSED pgs=3548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6f540a5860 0x7f6f5406d220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6f540a4070 0x7f6f541365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 --2- 172.21.15.105:0/641050546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f540a18f0 0x7f6f54136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.852+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 >> 172.21.15.105:0/641050546 conn(0x7f6f540aa450 msgr2=0x7f6f540a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.856+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 shutdown_connections 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.856+0000 7f6f5c1c5640 1 -- 172.21.15.105:0/641050546 wait complete. 2024-09-17T16:39:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.856+0000 7f6f5c1c5640 1 librados: shutdown 2024-09-17T16:39:35.879 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:39:35.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:39:35.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:39:35.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: get_monmap_and_config 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: build_initial_monmap 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: monmap: 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:35.992468+0000 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:35.992468+0000 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:35.996 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:35.997 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:35.998 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:35.998 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 Processor -- start 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 -- start start 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: init 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient: _add_conns ranks=[2,0,10,4,1,3,5,7,8,9,6] 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 0x7f78240a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-b con 0x7f78240a2620 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78240a2f40 addr v1:172.21.15.105:6789/0 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 0x7f782413f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-f con 0x7f7824137b10 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782406f2e0 con 0x7f78240a2f40 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f782406e580 con 0x7f7824137b10 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f78240a15e0 con 0x7f78240a2620 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7829d0a640 10 monclient(hunting): authenticate will time out at 1659.258161s 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7822ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f78240a2f40 0x7f7824137400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54054/0 (socket says 172.21.15.105:54054) 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7822ffd640 1 -- 172.21.15.105:0/3357077630 learned_addr learned my addr 172.21.15.105:0/3357077630 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 --2- 172.21.15.105:0/3357077630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 0x7f78240a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): get_auth_request con 0x7f78240a2620 auth_method 0 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 1 --2- 172.21.15.105:0/3357077630 >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 0x7f782413f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 10 monclient(hunting): get_auth_request con 0x7f7824137b10 auth_method 0 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 1 -- 172.21.15.105:0/3357077630 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674832579 0 0) 0x7f782406f2e0 con 0x7f78240a2f40 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 10 monclient(hunting): _negotiate 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 10 monclient(hunting): my global_id is 35883 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78227fc640 1 -- 172.21.15.105:0/3357077630 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f780c003530 con 0x7f78240a2f40 2024-09-17T16:39:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient(hunting): handle_auth_done global_id 44930 payload 274 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 -- 172.21.15.105:0/3357077630 >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 msgr2=0x7f782413f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 --2- 172.21.15.105:0/3357077630 >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 0x7f782413f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 -- 172.21.15.105:0/3357077630 >> v1:172.21.15.105:6789/0 conn(0x7f78240a2f40 legacy=0x7f7824137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 monclient: found mon.noname-b 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:36.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f78237fe640 1 -- 172.21.15.105:0/3357077630 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78240a3c30 con 0x7f78240a2620 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.988+0000 7f7823fff640 1 --2- 172.21.15.105:0/3357077630 >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 0x7f782413f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 1 --2- 172.21.15.105:0/3357077630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 0x7f78240a2a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78140097c0 tx=0x7f781401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dbada8075d702c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 1 -- 172.21.15.105:0/3357077630 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78140b0070 con 0x7f78240a2620 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: dump: 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.015 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 5 monclient: authenticate success, global_id 44930 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 1 -- 172.21.15.105:0/3357077630 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78140aa3a0 con 0x7f78240a2620 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 4 set_mon_vals no callback set 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 1 -- 172.21.15.105:0/3357077630 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78140aa9c0 con 0x7f78240a2620 2024-09-17T16:39:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78227fc640 10 monclient: dump: 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.020 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: shutdown 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- 172.21.15.105:0/3357077630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 msgr2=0x7f78240a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- 172.21.15.105:0/3357077630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 0x7f78240a2a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78140097c0 tx=0x7f781401c360 comp rx=0 tx=0).stop 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- 172.21.15.105:0/3357077630 shutdown_connections 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- 172.21.15.105:0/3357077630 >> v2:172.21.15.160:3300/0 conn(0x7f7824137b10 0x7f782413f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- 172.21.15.105:0/3357077630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78240a2620 0x7f78240a2a00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- 172.21.15.105:0/3357077630 >> 172.21.15.105:0/3357077630 conn(0x7f7824068ba0 msgr2=0x7f7824068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- 172.21.15.105:0/3357077630 shutdown_connections 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- 172.21.15.105:0/3357077630 wait complete. 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: monmap: 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 librados: starting msgr at 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 librados: starting objecter 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 Processor -- start 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- start start 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 librados: setting wanted keys 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 librados: calling monclient init 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: init 2024-09-17T16:39:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient: _add_conns ranks=[3,4,6,7,5,2,1,8,0] 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78240a2620 0x7f782411e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-g con 0x7f78240a2620 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78240a2f40 0x7f782411e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-c con 0x7f78240a2f40 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): picked mon.noname-d con 0x7f7824137b10 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f78240a1f50 con 0x7f78240a2620 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f78240a1dd0 con 0x7f78240a2f40 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f7824142640 con 0x7f7824137b10 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7829d0a640 10 monclient(hunting): authenticate will time out at 1659.263223s 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78240a2f40 0x7f782411e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33722/0 (socket says 172.21.15.105:33722) 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78240a2f40 0x7f782411e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40248/0 (socket says 172.21.15.105:40248) 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 -- 172.21.15.105:0/2975596307 learned_addr learned my addr 172.21.15.105:0/2975596307 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): get_auth_request con 0x7f7824137b10 auth_method 0 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): get_auth_request con 0x7f78240a2f40 auth_method 0 2024-09-17T16:39:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7822ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78240a2620 0x7f782411e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 10 monclient(hunting): get_auth_request con 0x7f78240a2620 auth_method 0 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient(hunting): handle_auth_done global_id 24585 payload 824 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78240a2f40 msgr2=0x7f782411e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f78240a2f40 0x7f782411e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78240a2620 msgr2=0x7f782411e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f78240a2620 0x7f782411e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 monclient: found mon.noname-d 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7824120c60 con 0x7f7824137b10 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f7823fff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f781c007ed0 tx=0x7f781c00d4a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f780bfff640 10 client.?.objecter ms_handle_connect 0x7f7824137b10 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.992+0000 7f780bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f781c01a070 con 0x7f7824137b10 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: dump: 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.028 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 5 monclient: authenticate success, global_id 24585 2024-09-17T16:39:36.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781c003e10 con 0x7f7824137b10 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f781c0044a0 con 0x7f7824137b10 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 monclient: _renew_subs 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241213d0 con 0x7f7824137b10 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7821ffb640 4 set_mon_vals no callback set 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: dump: 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.030 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.031 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 client.24585.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 monclient: _renew_subs 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f782409e150 con 0x7f7824137b10 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 1 librados: init done 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 0x7f782413ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7824051e80 con 0x7f782413a8b0 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 0x7f782413ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient: get_auth_request con 0x7f782413a8b0 auth_method 0 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f781c004640 con 0x7f7824137b10 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 10 monclient(hunting): handle_auth_done global_id 35892 payload 824 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78241200d0 con 0x7f782413a8b0 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f781003e550 0x7f7810040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(445..445 src has 1..445) ==== 6230+0+0 (secure 0 0 0) 0x7f781c050100 con 0x7f7824137b10 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 client.24585.objecter ms_dispatch 0x7f7824000ba0 osd_map(445..445 src has 1..445) 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 3 client.24585.objecter handle_osd_map got epochs [445,445] > 0 2024-09-17T16:39:36.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 3 client.24585.objecter handle_osd_map decoding full epoch 445 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f78237fe640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 0x7f782413ac90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78240698c0 tx=0x7f7814019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 20 client.24585.objecter dump_active .. 0 homeless 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 client.24585.objecter ms_handle_connect 0x7f782413a8b0 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 client.24585.objecter resend_mon_ops 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78140b9070 con 0x7f782413a8b0 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: dump: 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.034 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7822ffd640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f781003e550 0x7f7810040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7822ffd640 10 monclient: get_auth_request con 0x7f781003e550 auth_method 0 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7814003620 con 0x7f782413a8b0 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 msgr2=0x7f782413ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 0x7f782413ac90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78240698c0 tx=0x7f7814019100 comp rx=0 tx=0).stop 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 msgr2=0x7f782413ac90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413a8b0 0x7f782413ac90 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78240698c0 tx=0x7f7814019100 comp rx=0 tx=0).stop 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f780bfff640 10 client.24585.objecter ms_handle_connect 0x7f781003e550 2024-09-17T16:39:36.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:35.996+0000 7f7822ffd640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f781003e550 0x7f7810040a10 secure :-1 s=READY pgs=3549 cs=0 l=1 rev1=1 crypto rx=0x7f7818007a70 tx=0x7f78180079c0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 0x7f78241d95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f782413f260 con 0x7f782413b1d0 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 0x7f78241d95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient: get_auth_request con 0x7f782413b1d0 auth_method 0 2024-09-17T16:39:36.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 10 monclient(hunting): handle_auth_done global_id 35901 payload 824 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 1 -- 172.21.15.105:0/2975596307 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7824072670 con 0x7f782413b1d0 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 client.24585.objecter ms_handle_connect 0x7f782413b1d0 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 client.24585.objecter resend_mon_ops 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f78237fe640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 0x7f78241d95a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f78140097c0 tx=0x7f78140a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78140b9050 con 0x7f782413b1d0 2024-09-17T16:39:36.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 monclient: dump: 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.040 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f78140b59b0 con 0x7f782413b1d0 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 msgr2=0x7f78241d95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 0x7f78241d95a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f78140097c0 tx=0x7f78140a8000 comp rx=0 tx=0).stop 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 msgr2=0x7f78241d95a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f780bfff640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f782413b1d0 0x7f78241d95a0 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f78140097c0 tx=0x7f78140a8000 comp rx=0 tx=0).stop 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 566808, 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:36.042 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.043 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.044 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:36.045 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:36.046 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.047 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:36.049 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.052 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.054 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.059 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.060 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.063 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.064 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.066 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:36.066 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 librados: watch_flush enter 2024-09-17T16:39:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 librados: watch_flush exit 2024-09-17T16:39:36.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 20 client.24585.objecter shutdown clearing up homeless session... 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 client.24585.objecter successfully canceled tick 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f781003e550 msgr2=0x7f7810040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f781003e550 0x7f7810040a10 secure :-1 s=READY pgs=3549 cs=0 l=1 rev1=1 crypto rx=0x7f7818007a70 tx=0x7f78180079c0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 10 monclient: shutdown 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 msgr2=0x7f782411ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f781c007ed0 tx=0x7f781c00d4a0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7823fff640 1 -- 172.21.15.105:0/2975596307 reap_dead start 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 shutdown_connections 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 --2- 172.21.15.105:0/2975596307 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7824137b10 0x7f782411ecf0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 >> 172.21.15.105:0/2975596307 conn(0x7f7824068ba0 msgr2=0x7f78240a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 shutdown_connections 2024-09-17T16:39:36.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 -- 172.21.15.105:0/2975596307 wait complete. 2024-09-17T16:39:36.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.004+0000 7f7829d0a640 1 librados: shutdown 2024-09-17T16:39:36.069 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:36.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:39:36.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:39:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: get_monmap_and_config 2024-09-17T16:39:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: monmap: 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:36.180365+0000 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:36.180365+0000 2024-09-17T16:39:36.185 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.186 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.187 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.187 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.187 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.187 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.187 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:36.188 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:36.188 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 Processor -- start 2024-09-17T16:39:36.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 -- start start 2024-09-17T16:39:36.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: init 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient: _add_conns ranks=[0,3,8,10,7,1,5,9,4,6,2] 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f49340a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f493409e740 0x7f493413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-g con 0x7f493409e740 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 0x7f493413dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-e con 0x7f493409e0b0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49340a0a80 con 0x7f49340a4750 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f493409ef60 con 0x7f493409e740 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f49340a31c0 con 0x7f493409e0b0 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4938ab2640 10 monclient(hunting): authenticate will time out at 1659.446243s 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 0x7f49340a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54060/0 (socket says 172.21.15.105:54060) 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 -- 172.21.15.105:0/3218534676 learned_addr learned my addr 172.21.15.105:0/3218534676 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 0x7f493413dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): get_auth_request con 0x7f493409e0b0 auth_method 0 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 -- 172.21.15.105:0/3218534676 >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 legacy=0x7f49340a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 -- 172.21.15.105:0/3218534676 >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 legacy=0x7f49340a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f493409e740 0x7f493413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 --1- 172.21.15.105:0/3218534676 >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 0x7f49340a4b30 :-1 s=OPENED pgs=367 cs=1 l=1).handle_message read tag failed 2024-09-17T16:39:36.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493259c640 1 --1- 172.21.15.105:0/3218534676 >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 0x7f49340a4b30 :-1 s=OPENED pgs=367 cs=1 l=1).fault on lossy channel, failing 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f493159a640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.105:6789/0 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): get_auth_request con 0x7f493409e740 auth_method 0 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient(hunting): handle_auth_done global_id 54431 payload 274 2024-09-17T16:39:36.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 -- 172.21.15.105:0/3218534676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f493409e740 msgr2=0x7f493413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f493409e740 0x7f493413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 -- 172.21.15.105:0/3218534676 >> v1:172.21.15.105:6789/0 conn(0x7f49340a4750 legacy=0x7f49340a4b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 monclient: found mon.noname-e 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:39:36.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.176+0000 7f4932d9d640 1 -- 172.21.15.105:0/3218534676 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49340a6710 con 0x7f493409e0b0 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 0x7f493413dc90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f492c009d90 tx=0x7f492c01c830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4397ad8527a48360 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 1 -- 172.21.15.105:0/3218534676 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f492c0ad070 con 0x7f493409e0b0 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: dump: 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.216 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.217 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 1 -- 172.21.15.105:0/3218534676 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c0a98b0 con 0x7f493409e0b0 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 5 monclient: authenticate success, global_id 54431 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:36.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 1 -- 172.21.15.105:0/3218534676 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f492c0a83a0 con 0x7f493409e0b0 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 4 set_mon_vals no callback set 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493159a640 10 monclient: dump: 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.219 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.220 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: shutdown 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- 172.21.15.105:0/3218534676 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 msgr2=0x7f493413dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 0x7f493413dc90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f492c009d90 tx=0x7f492c01c830 comp rx=0 tx=0).stop 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- 172.21.15.105:0/3218534676 shutdown_connections 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f493409e0b0 0x7f493413dc90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3218534676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f493409e740 0x7f493413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- 172.21.15.105:0/3218534676 >> 172.21.15.105:0/3218534676 conn(0x7f49340a5840 msgr2=0x7f49340a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- 172.21.15.105:0/3218534676 shutdown_connections 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- 172.21.15.105:0/3218534676 wait complete. 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: monmap: 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:36.223 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 librados: starting msgr at 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 librados: starting objecter 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 Processor -- start 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- start start 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 librados: setting wanted keys 2024-09-17T16:39:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 librados: calling monclient init 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: init 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient: _add_conns ranks=[6,1,2,7,8,0,4,5,3] 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 0x7f49341d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-d con 0x7f493409e0b0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f493409e740 0x7f493413c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-f con 0x7f493409e740 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49340a4750 0x7f493413cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): picked mon.noname-b con 0x7f49340a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f493406ccb0 con 0x7f493409e740 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f493406cb30 con 0x7f49340a4750 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f493406ce30 con 0x7f493409e0b0 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4938ab2640 10 monclient(hunting): authenticate will time out at 1659.451324s 2024-09-17T16:39:36.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 0x7f49341d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49340a4750 0x7f493413cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 0x7f49341d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33724/0 (socket says 172.21.15.105:33724) 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 -- 172.21.15.105:0/3265865317 learned_addr learned my addr 172.21.15.105:0/3265865317 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): get_auth_request con 0x7f493409e0b0 auth_method 0 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): get_auth_request con 0x7f49340a4750 auth_method 0 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4932d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f493409e740 0x7f493413c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4931d9b640 10 monclient(hunting): get_auth_request con 0x7f493409e740 auth_method 0 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4931d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4931d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f4931d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient(hunting): handle_auth_done global_id 24594 payload 824 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49340a4750 msgr2=0x7f493413cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49340a4750 0x7f493413cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f493409e740 msgr2=0x7f493413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f493409e740 0x7f493413c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 monclient: found mon.noname-d 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493413d840 con 0x7f493409e0b0 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f493259c640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 0x7f49341d1020 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f492400c4b0 tx=0x7f492400c970 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.180+0000 7f492affd640 10 client.?.objecter ms_handle_connect 0x7f493409e0b0 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4924017070 con 0x7f493409e0b0 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: dump: 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 5 monclient: authenticate success, global_id 24594 2024-09-17T16:39:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4924007070 con 0x7f493409e0b0 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 monclient: _renew_subs 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493413d2e0 con 0x7f493409e0b0 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4924007710 con 0x7f493409e0b0 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4930d99640 4 set_mon_vals no callback set 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 client.24594.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: dump: 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 monclient: _renew_subs 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f493412a990 con 0x7f493409e0b0 2024-09-17T16:39:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 1 librados: init done 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 0x7f49341277c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4934051e80 con 0x7f49341273e0 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 0x7f49341277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient: get_auth_request con 0x7f49341273e0 auth_method 0 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f49240072a0 con 0x7f493409e0b0 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 10 monclient(hunting): handle_auth_done global_id 44948 payload 824 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49341d1560 con 0x7f49341273e0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f491403e550 0x7f4914040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(445..445 src has 1..445) ==== 6230+0+0 (secure 0 0 0) 0x7f492404f060 con 0x7f493409e0b0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 client.24594.objecter ms_dispatch 0x7f4934000ba0 osd_map(445..445 src has 1..445) 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 3 client.24594.objecter handle_osd_map got epochs [445,445] > 0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 3 client.24594.objecter handle_osd_map decoding full epoch 445 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 20 client.24594.objecter dump_active .. 0 homeless 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 client.24594.objecter ms_handle_connect 0x7f49341273e0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 client.24594.objecter resend_mon_ops 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 0x7f49341277c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4934129540 tx=0x7f491c010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f491c0044a0 con 0x7f49341273e0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4932d9d640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f491403e550 0x7f4914040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: dump: 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4932d9d640 10 monclient: get_auth_request con 0x7f491403e550 auth_method 0 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f491c00ed40 con 0x7f49341273e0 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 msgr2=0x7f49341277c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 0x7f49341277c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4934129540 tx=0x7f491c010f50 comp rx=0 tx=0).stop 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 msgr2=0x7f49341277c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49341273e0 0x7f49341277c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f492affd640 10 client.24594.objecter ms_handle_connect 0x7f491403e550 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.184+0000 7f4932d9d640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f491403e550 0x7f4914040a10 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7f492c0af000 tx=0x7f492c0b4040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 0x7f493411f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f493412b150 con 0x7f4934127d00 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 0x7f493411f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient: get_auth_request con 0x7f4934127d00 auth_method 0 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 10 monclient(hunting): handle_auth_done global_id 44957 payload 824 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 1 -- 172.21.15.105:0/3265865317 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f49340a4b30 con 0x7f4934127d00 2024-09-17T16:39:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 client.24594.objecter ms_handle_connect 0x7f4934127d00 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 client.24594.objecter resend_mon_ops 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4931d9b640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 0x7f493411f390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4934129540 tx=0x7f491c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f491c01f2a0 con 0x7f4934127d00 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 monclient: dump: 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f491c01f2a0 con 0x7f4934127d00 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 msgr2=0x7f493411f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 0x7f493411f390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4934129540 tx=0x7f491c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 msgr2=0x7f493411f390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f492affd640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4934127d00 0x7f493411f390 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4934129540 tx=0x7f491c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 426845, 2024-09-17T16:39:36.243 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.244 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.245 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.246 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:36.247 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.248 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.250 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.252 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.253 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:36.256 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.260 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.261 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.262 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.263 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.264 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.265 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:36.267 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.268 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.269 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.269 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.269 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:36.269 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:36.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 librados: watch_flush enter 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 librados: watch_flush exit 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 20 client.24594.objecter shutdown clearing up homeless session... 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 client.24594.objecter successfully canceled tick 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f491403e550 msgr2=0x7f4914040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f491403e550 0x7f4914040a10 secure :-1 s=READY pgs=3550 cs=0 l=1 rev1=1 crypto rx=0x7f492c0af000 tx=0x7f492c0b4040 comp rx=0 tx=0).stop 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 10 monclient: shutdown 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 msgr2=0x7f49341d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 --2- 172.21.15.105:0/3265865317 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f493409e0b0 0x7f49341d1020 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f492400c4b0 tx=0x7f492400c970 comp rx=0 tx=0).stop 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4932d9d640 1 -- 172.21.15.105:0/3265865317 reap_dead start 2024-09-17T16:39:36.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4932d9d640 1 -- 172.21.15.105:0/3265865317 reap_dead start 2024-09-17T16:39:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 shutdown_connections 2024-09-17T16:39:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 >> 172.21.15.105:0/3265865317 conn(0x7f49340a5840 msgr2=0x7f49340a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 shutdown_connections 2024-09-17T16:39:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 -- 172.21.15.105:0/3265865317 wait complete. 2024-09-17T16:39:36.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.192+0000 7f4938ab2640 1 librados: shutdown 2024-09-17T16:39:36.272 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:36.272 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:39:36.272 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:39:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: get_monmap_and_config 2024-09-17T16:39:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: monmap: 2024-09-17T16:39:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.396 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:36.393047+0000 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:36.393047+0000 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.397 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:36.398 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:36.398 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 Processor -- start 2024-09-17T16:39:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 -- start start 2024-09-17T16:39:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: init 2024-09-17T16:39:36.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient: _add_conns ranks=[3,7,4,0,5,2,10,9,6,1,8] 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f100a68e0 0x7f1f100a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-g con 0x7f1f100a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1f100a4910 0x7f1f100a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-i con 0x7f1f100a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-c con 0x7f1f101396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1f100a3fc0 con 0x7f1f100a68e0 2024-09-17T16:39:36.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1f100a63c0 con 0x7f1f101396a0 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f1f100a7740 con 0x7f1f100a4910 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f15546640 10 monclient(hunting): authenticate will time out at 1659.659621s 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40256/0 (socket says 172.21.15.105:40256) 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2523380386 learned_addr learned my addr 172.21.15.105:0/2523380386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): get_auth_request con 0x7f1f101396a0 auth_method 0 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.388+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f07fff640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1f100a4910 0x7f1f100a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f100a68e0 0x7f1f100a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f07fff640 10 monclient(hunting): get_auth_request con 0x7f1f100a4910 auth_method 0 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0effd640 10 monclient(hunting): get_auth_request con 0x7f1f100a68e0 auth_method 0 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_done global_id 74119 payload 274 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1f100a4910 msgr2=0x7f1f100a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1f100a4910 0x7f1f100a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f100a68e0 msgr2=0x7f1f100a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f100a68e0 0x7f1f100a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 monclient: found mon.noname-c 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:36.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2523380386 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1006ec10 con 0x7f1f101396a0 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009e80 tx=0x7f1f0801d370 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3ae5fde7b769dcc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 1 -- 172.21.15.105:0/2523380386 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f080090d0 con 0x7f1f101396a0 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: dump: 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.413 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.414 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 1 -- 172.21.15.105:0/2523380386 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f080a5a70 con 0x7f1f101396a0 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 5 monclient: authenticate success, global_id 74119 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:36.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 1 -- 172.21.15.105:0/2523380386 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f080a5c20 con 0x7f1f101396a0 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f0e7fc640 10 monclient: dump: 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.416 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 4 set_mon_vals no callback set 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 10 monclient: shutdown 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 -- 172.21.15.105:0/2523380386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 msgr2=0x7f1f10139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009e80 tx=0x7f1f0801d370 comp rx=0 tx=0).stop 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 -- 172.21.15.105:0/2523380386 shutdown_connections 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f10139ae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1f100a4910 0x7f1f100a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 --2- 172.21.15.105:0/2523380386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f100a68e0 0x7f1f100a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 -- 172.21.15.105:0/2523380386 >> 172.21.15.105:0/2523380386 conn(0x7f1f10068ba0 msgr2=0x7f1f10068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 -- 172.21.15.105:0/2523380386 shutdown_connections 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.392+0000 7f1f15546640 1 -- 172.21.15.105:0/2523380386 wait complete. 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: monmap: 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.418 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.419 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 librados: starting msgr at 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 librados: starting objecter 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 Processor -- start 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- start start 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 librados: setting wanted keys 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 librados: calling monclient init 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: init 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _add_conns ranks=[5,8,4,1,6,3,2,0,7] 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1f100a4910 0x7f1f101d1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-h con 0x7f1f100a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1f100a68e0 0x7f1f101d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-e con 0x7f1f100a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f1013d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): picked mon.noname-c con 0x7f1f101396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1f10143320 con 0x7f1f101396a0 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1f1006e630 con 0x7f1f100a4910 2024-09-17T16:39:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1f101434a0 con 0x7f1f100a68e0 2024-09-17T16:39:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): authenticate will time out at 1659.665212s 2024-09-17T16:39:36.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f1013d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1f100a68e0 0x7f1f101d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f1013d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40258/0 (socket says 172.21.15.105:40258) 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 learned_addr learned my addr 172.21.15.105:0/2207836728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): get_auth_request con 0x7f1f101396a0 auth_method 0 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): get_auth_request con 0x7f1f100a68e0 auth_method 0 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1f100a4910 0x7f1f101d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): get_auth_request con 0x7f1f100a4910 auth_method 0 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient(hunting): handle_auth_done global_id 74128 payload 824 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1f100a68e0 msgr2=0x7f1f101d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1f100a68e0 0x7f1f101d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1f100a4910 msgr2=0x7f1f101d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1f100a4910 0x7f1f101d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 monclient: found mon.noname-c 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f10123650 con 0x7f1f101396a0 2024-09-17T16:39:36.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0f7fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f1013d380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f080af040 tx=0x7f1f080196b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 client.?.objecter ms_handle_connect 0x7f1f101396a0 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f080bc020 con 0x7f1f101396a0 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: dump: 2024-09-17T16:39:36.428 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.429 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.430 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08003bb0 con 0x7f1f101396a0 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 5 monclient: authenticate success, global_id 74128 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _renew_subs 2024-09-17T16:39:36.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f10123930 con 0x7f1f101396a0 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f08003da0 con 0x7f1f101396a0 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:36.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: dump: 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.433 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.434 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.434 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.434 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.434 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.434 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0dffb640 4 set_mon_vals no callback set 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 client.74128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _renew_subs 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f100ab950 con 0x7f1f101396a0 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 librados: init done 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 0x7f1f1002d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1f10051e80 con 0x7f1f1013e080 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 0x7f1f1002d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient: get_auth_request con 0x7f1f1013e080 auth_method 0 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1f080bf020 con 0x7f1f101396a0 2024-09-17T16:39:36.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1ef003e550 0x7f1ef0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 10 monclient(hunting): handle_auth_done global_id 74137 payload 824 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f100a9e00 con 0x7f1f1013e080 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(445..445 src has 1..445) ==== 6230+0+0 (secure 0 0 0) 0x7f1f080eb040 con 0x7f1f101396a0 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 client.74128.objecter ms_dispatch 0x7f1f10000ba0 osd_map(445..445 src has 1..445) 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 3 client.74128.objecter handle_osd_map got epochs [445,445] > 0 2024-09-17T16:39:36.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 3 client.74128.objecter handle_osd_map decoding full epoch 445 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 20 client.74128.objecter dump_active .. 0 homeless 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 client.74128.objecter ms_handle_connect 0x7f1f1013e080 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 client.74128.objecter resend_mon_ops 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 0x7f1f1002d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f0000f970 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1ef003e550 0x7f1ef0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f00019020 con 0x7f1f1013e080 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:36.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f077fe640 10 monclient: dump: 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.440 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.441 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.396+0000 7f1f07fff640 10 monclient: get_auth_request con 0x7f1ef003e550 auth_method 0 2024-09-17T16:39:36.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1f00014670 con 0x7f1f1013e080 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 msgr2=0x7f1f1002d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 0x7f1f1002d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f0000f970 comp rx=0 tx=0).stop 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 msgr2=0x7f1f1002d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e080 0x7f1f1002d450 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f0000f970 comp rx=0 tx=0).stop 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f07fff640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1ef003e550 0x7f1ef0040a10 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f1f1013eae0 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.400+0000 7f1f077fe640 10 client.74128.objecter ms_handle_connect 0x7f1ef003e550 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 0x7f1f1002d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1f10139ae0 con 0x7f1f1013e460 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 0x7f1f1002d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient: get_auth_request con 0x7f1f1013e460 auth_method 0 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 10 monclient(hunting): handle_auth_done global_id 74146 payload 824 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 1 -- 172.21.15.105:0/2207836728 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1f10123b70 con 0x7f1f1013e460 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0effd640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 0x7f1f1002d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f000046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 client.74128.objecter ms_handle_connect 0x7f1f1013e460 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 client.74128.objecter resend_mon_ops 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f00019040 con 0x7f1f1013e460 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 monclient: dump: 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.446 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f1f00004020 con 0x7f1f1013e460 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:36.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 msgr2=0x7f1f1002d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 0x7f1f1002d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f000046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 msgr2=0x7f1f1002d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f077fe640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f1013e460 0x7f1f1002d990 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000f4b0 tx=0x7f1f000046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:36.448 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 15867, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:39:36.449 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.450 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.451 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:36.452 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.453 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.454 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.455 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.459 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.460 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.462 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:36.463 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.464 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.466 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.467 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.470 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:36.471 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 librados: watch_flush enter 2024-09-17T16:39:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 librados: watch_flush exit 2024-09-17T16:39:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 20 client.74128.objecter shutdown clearing up homeless session... 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 client.74128.objecter successfully canceled tick 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1ef003e550 msgr2=0x7f1ef0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1ef003e550 0x7f1ef0040a10 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f1f1013eae0 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 10 monclient: shutdown 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 msgr2=0x7f1f1013d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 --2- 172.21.15.105:0/2207836728 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1f101396a0 0x7f1f1013d380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f080af040 tx=0x7f1f080196b0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 reap_dead start 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f0f7fe640 1 -- 172.21.15.105:0/2207836728 reap_dead start 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 shutdown_connections 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 >> 172.21.15.105:0/2207836728 conn(0x7f1f10068ba0 msgr2=0x7f1f101403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 shutdown_connections 2024-09-17T16:39:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.404+0000 7f1f15546640 1 -- 172.21.15.105:0/2207836728 wait complete. 2024-09-17T16:39:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.408+0000 7f1f15546640 1 librados: shutdown 2024-09-17T16:39:36.476 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:36.477 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:39:36.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:39:36.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: get_monmap_and_config 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: monmap: 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:36.589474+0000 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:36.589474+0000 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:36.593 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:36.594 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 Processor -- start 2024-09-17T16:39:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 -- start start 2024-09-17T16:39:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: init 2024-09-17T16:39:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient: _add_conns ranks=[9,10,6,3,5,7,2,4,0,8,1] 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f146c0a6a30 0x7f146c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-a con 0x7f146c0a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f146c0a4ab0 0x7f146c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-f con 0x7f146c0a4ab0 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-d con 0x7f146c1397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f146c0a7c40 con 0x7f146c0a6a30 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f146c0a7890 con 0x7f146c0a4ab0 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f146c0a4080 con 0x7f146c1397f0 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.584+0000 7f14741e1640 10 monclient(hunting): authenticate will time out at 1659.855911s 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:33740/0 (socket says 172.21.15.105:33740) 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 -- 172.21.15.105:0/93260834 learned_addr learned my addr 172.21.15.105:0/93260834 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): get_auth_request con 0x7f146c1397f0 auth_method 0 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.105:3300/0 conn(0x7f146c0a6a30 0x7f146c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): get_auth_request con 0x7f146c0a6a30 auth_method 0 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147177c640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.160:3300/0 conn(0x7f146c0a4ab0 0x7f146c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147177c640 10 monclient(hunting): get_auth_request con 0x7f146c0a4ab0 auth_method 0 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147177c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147177c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147177c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient(hunting): handle_auth_done global_id 24603 payload 274 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 -- 172.21.15.105:0/93260834 >> v2:172.21.15.160:3300/0 conn(0x7f146c0a4ab0 msgr2=0x7f146c0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.160:3300/0 conn(0x7f146c0a4ab0 0x7f146c0a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 -- 172.21.15.105:0/93260834 >> v2:172.21.15.105:3300/0 conn(0x7f146c0a6a30 msgr2=0x7f146c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.105:3300/0 conn(0x7f146c0a6a30 0x7f146c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 monclient: found mon.noname-d 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 -- 172.21.15.105:0/93260834 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c0a37e0 con 0x7f146c1397f0 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f147277e640 1 --2- 172.21.15.105:0/93260834 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f146c0a7e50 tx=0x7f145c01c3e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=17b7771f3cfde796 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 1 -- 172.21.15.105:0/93260834 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f145c0b0070 con 0x7f146c1397f0 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: dump: 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.614 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 1 -- 172.21.15.105:0/93260834 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145c0aa810 con 0x7f146c1397f0 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 1 -- 172.21.15.105:0/93260834 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f145c0a9300 con 0x7f146c1397f0 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 5 monclient: authenticate success, global_id 24603 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 4 set_mon_vals no callback set 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f1470f7b640 10 monclient: dump: 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.616 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.617 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 10 monclient: shutdown 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 -- 172.21.15.105:0/93260834 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 msgr2=0x7f146c139c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 --2- 172.21.15.105:0/93260834 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f146c0a7e50 tx=0x7f145c01c3e0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 -- 172.21.15.105:0/93260834 shutdown_connections 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 --2- 172.21.15.105:0/93260834 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c1397f0 0x7f146c139c80 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.160:3300/0 conn(0x7f146c0a4ab0 0x7f146c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 --2- 172.21.15.105:0/93260834 >> v2:172.21.15.105:3300/0 conn(0x7f146c0a6a30 0x7f146c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 -- 172.21.15.105:0/93260834 >> 172.21.15.105:0/93260834 conn(0x7f146c0a8d00 msgr2=0x7f146c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 -- 172.21.15.105:0/93260834 shutdown_connections 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.588+0000 7f14741e1640 1 -- 172.21.15.105:0/93260834 wait complete. 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient: monmap: 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.618 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 librados: starting msgr at 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 librados: starting objecter 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 Processor -- start 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 -- start start 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 librados: setting wanted keys 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 librados: calling monclient init 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient: init 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient: _add_conns ranks=[6,8,4,0,3,5,1,2,7] 2024-09-17T16:39:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c0a4ab0 0x7f146c1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-d con 0x7f146c0a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 0x7f146c138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-e con 0x7f146c0a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f146c1397f0 0x7f146c13d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): picked mon.noname-c con 0x7f146c1397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f146c06e770 con 0x7f146c1397f0 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f146c0a2fc0 con 0x7f146c0a4ab0 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f146c143260 con 0x7f146c0a6a30 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f14741e1640 10 monclient(hunting): authenticate will time out at 1659.861547s 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 0x7f146c138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f146c1397f0 0x7f146c13d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c0a4ab0 0x7f146c1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 0x7f146c138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:53266/0 (socket says 172.21.15.105:53266) 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f146c1397f0 0x7f146c13d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40292/0 (socket says 172.21.15.105:40292) 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 1 -- 172.21.15.105:0/4223443825 learned_addr learned my addr 172.21.15.105:0/4223443825 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): get_auth_request con 0x7f146c0a4ab0 auth_method 0 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): get_auth_request con 0x7f146c0a6a30 auth_method 0 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): get_auth_request con 0x7f146c1397f0 auth_method 0 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147277e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient(hunting): handle_auth_done global_id 24612 payload 824 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 msgr2=0x7f146c138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 0x7f146c138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f146c1397f0 msgr2=0x7f146c13d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f146c1397f0 0x7f146c13d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 monclient: found mon.noname-d 2024-09-17T16:39:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c123da0 con 0x7f146c0a4ab0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f147177c640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f146c0a6a30 0x7f146c138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f1471f7d640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c0a4ab0 0x7f146c1382d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f146400efc0 tx=0x7f146400c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f146a7fc640 10 client.?.objecter ms_handle_connect 0x7f146c0a4ab0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.592+0000 7f146a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1464018020 con 0x7f146c0a4ab0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 10 monclient: dump: 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f146a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f14741e1640 5 monclient: authenticate success, global_id 24612 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f14741e1640 10 monclient: _renew_subs 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f14741e1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.596+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c13f5c0 con 0x7f146c0a4ab0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 10 client.24612.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 10 monclient: _renew_subs 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c0aa090 con 0x7f146c0a4ab0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 1 librados: init done 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146400cdb0 con 0x7f146c0a4ab0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1464014d60 con 0x7f146c0a4ab0 2024-09-17T16:39:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: dump: 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146bfff640 4 set_mon_vals no callback set 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 0x7f144c004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f144c007cc0 con 0x7f144c002890 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f146401b020 con 0x7f146c0a4ab0 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 0x7f144c004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient: get_auth_request con 0x7f144c002890 auth_method 0 2024-09-17T16:39:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f145803e550 0x7f1458040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(446..446 src has 1..446) ==== 5845+0+0 (secure 0 0 0) 0x7f14640506a0 con 0x7f146c0a4ab0 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 client.24612.objecter ms_dispatch 0x7f146c000ba0 osd_map(446..446 src has 1..446) 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 3 client.24612.objecter handle_osd_map got epochs [446,446] > 0 2024-09-17T16:39:36.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 3 client.24612.objecter handle_osd_map decoding full epoch 446 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 10 monclient(hunting): handle_auth_done global_id 24621 payload 824 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f144c007ec0 con 0x7f144c002890 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 20 client.24612.objecter dump_active .. 0 homeless 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 client.24612.objecter ms_handle_connect 0x7f144c002890 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 client.24612.objecter resend_mon_ops 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147177c640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 0x7f144c004d30 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f146c13d6d0 tx=0x7f146000e5a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1460018070 con 0x7f144c002890 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: dump: 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.631 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f146000ee80 con 0x7f144c002890 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 msgr2=0x7f144c004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 0x7f144c004d30 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f146c13d6d0 tx=0x7f146000e5a0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147277e640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f145803e550 0x7f1458040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 msgr2=0x7f144c004d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f146a7fc640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c002890 0x7f144c004d30 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.600+0000 7f147277e640 10 monclient: get_auth_request con 0x7f145803e550 auth_method 0 2024-09-17T16:39:36.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f146a7fc640 10 client.24612.objecter ms_handle_connect 0x7f145803e550 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147277e640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f145803e550 0x7f1458040a10 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7f146c0a7e50 tx=0x7f145c0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f14741e1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f14741e1640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 0x7f144c007170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f14741e1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f144c00a200 con 0x7f144c006ce0 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 0x7f144c007170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient: get_auth_request con 0x7f144c006ce0 auth_method 0 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 10 monclient(hunting): handle_auth_done global_id 24630 payload 824 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 1 -- 172.21.15.105:0/4223443825 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f144c0076b0 con 0x7f144c006ce0 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f147177c640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 0x7f144c007170 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f146c13d6d0 tx=0x7f14600046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f146a7fc640 10 client.24612.objecter ms_handle_connect 0x7f144c006ce0 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.608+0000 7f146a7fc640 10 client.24612.objecter resend_mon_ops 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f146001dad0 con 0x7f144c006ce0 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:36.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 10 monclient: dump: 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f146001d660 con 0x7f144c006ce0 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 msgr2=0x7f144c007170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 0x7f144c007170 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f146c13d6d0 tx=0x7f14600046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 msgr2=0x7f144c007170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f146a7fc640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f144c006ce0 0x7f144c007170 secure :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f146c13d6d0 tx=0x7f14600046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 724252, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.646 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.649 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.650 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.651 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:36.653 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.657 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.661 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.662 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:36.662 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 10 librados: watch_flush enter 2024-09-17T16:39:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 10 librados: watch_flush exit 2024-09-17T16:39:36.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 20 client.24612.objecter shutdown clearing up homeless session... 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 10 client.24612.objecter successfully canceled tick 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f145803e550 msgr2=0x7f1458040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f145803e550 0x7f1458040a10 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7f146c0a7e50 tx=0x7f145c0a6000 comp rx=0 tx=0).stop 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 10 monclient: shutdown 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c0a4ab0 msgr2=0x7f146c1382d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 --2- 172.21.15.105:0/4223443825 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f146c0a4ab0 0x7f146c1382d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f146400efc0 tx=0x7f146400c490 comp rx=0 tx=0).stop 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f147277e640 1 -- 172.21.15.105:0/4223443825 reap_dead start 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f147277e640 1 -- 172.21.15.105:0/4223443825 reap_dead start 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 shutdown_connections 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 >> 172.21.15.105:0/4223443825 conn(0x7f146c0a8d00 msgr2=0x7f146c13fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 shutdown_connections 2024-09-17T16:39:36.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 -- 172.21.15.105:0/4223443825 wait complete. 2024-09-17T16:39:36.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.612+0000 7f14741e1640 1 librados: shutdown 2024-09-17T16:39:36.665 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:36.665 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:39:36.665 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:39:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: get_monmap_and_config 2024-09-17T16:39:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: monmap: 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:36.776165+0000 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:36.776165+0000 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.780 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:36.781 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 Processor -- start 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 -- start start 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: init 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient: _add_conns ranks=[4,2,9,7,6,3,0,5,1,8,10] 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 0x7f6ccc09b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-c con 0x7f6ccc09b510 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 0x7f6ccc0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-b con 0x7f6ccc0a25a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:36.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f6ccc0a42c0 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc09f3b0 con 0x7f6ccc0a42c0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc0a18e0 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6ccc0a1bc0 con 0x7f6ccc09b510 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6cd1ea1640 10 monclient(hunting): authenticate will time out at 1660.042013s 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 0x7f6ccc0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 0x7f6ccc09b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 0x7f6ccc09b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40300/0 (socket says 172.21.15.105:40300) 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39562/0 (socket says 172.21.15.105:39562) 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 1 -- 172.21.15.105:0/1750045828 learned_addr learned my addr 172.21.15.105:0/1750045828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): get_auth_request con 0x7f6ccc0a25a0 auth_method 0 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): get_auth_request con 0x7f6ccc09b510 auth_method 0 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): get_auth_request con 0x7f6ccc0a42c0 auth_method 0 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_done global_id 44966 payload 274 2024-09-17T16:39:36.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 -- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 msgr2=0x7f6ccc09b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 0x7f6ccc09b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 -- 172.21.15.105:0/1750045828 >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 msgr2=0x7f6ccc0a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/1750045828 >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 monclient: found mon.noname-b 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccaffd640 1 -- 172.21.15.105:0/1750045828 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0a5c20 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.772+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/1750045828 >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 0x7f6ccc0a3d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ba90 tx=0x7f6cc001ea30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd3a1cb56f84585 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 1 -- 172.21.15.105:0/1750045828 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cc00b4070 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: dump: 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.790 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 5 monclient: authenticate success, global_id 44966 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 1 -- 172.21.15.105:0/1750045828 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc00a79f0 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 4 set_mon_vals no callback set 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 1 -- 172.21.15.105:0/1750045828 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cc00a7c20 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cca7fc640 10 monclient: dump: 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.792 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: shutdown 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 msgr2=0x7f6ccc0a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 0x7f6ccc0a3d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000ba90 tx=0x7f6cc001ea30 comp rx=0 tx=0).stop 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/1750045828 shutdown_connections 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/1750045828 >> v2:172.21.15.105:3300/0 conn(0x7f6ccc0a42c0 0x7f6ccc0a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ccc0a25a0 0x7f6ccc0a3d80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/1750045828 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6ccc09b510 0x7f6ccc09b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/1750045828 >> 172.21.15.105:0/1750045828 conn(0x7f6ccc0a4e70 msgr2=0x7f6ccc0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/1750045828 shutdown_connections 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/1750045828 wait complete. 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: monmap: 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:36.794 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 librados: starting msgr at 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 librados: starting objecter 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 Processor -- start 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- start start 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 librados: setting wanted keys 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 librados: calling monclient init 2024-09-17T16:39:36.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: init 2024-09-17T16:39:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _add_conns ranks=[7,0,6,3,5,1,8,2,4] 2024-09-17T16:39:36.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6ccc09b510 0x7f6ccc1d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-i con 0x7f6ccc09b510 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 0x7f6ccc1d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f6ccc0a25a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6ccc0a42c0 0x7f6ccc1d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): picked mon.noname-d con 0x7f6ccc0a42c0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0a0410 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6ccc0a0290 con 0x7f6ccc0a42c0 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6ccc0a0590 con 0x7f6ccc09b510 2024-09-17T16:39:36.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient(hunting): authenticate will time out at 1660.046925s 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 0x7f6ccc1d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6ccc0a42c0 0x7f6ccc1d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 0x7f6ccc1d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39578/0 (socket says 172.21.15.105:39578) 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 -- 172.21.15.105:0/3028714647 learned_addr learned my addr 172.21.15.105:0/3028714647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:36.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): get_auth_request con 0x7f6ccc0a25a0 auth_method 0 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): get_auth_request con 0x7f6ccc0a42c0 auth_method 0 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient(hunting): handle_auth_done global_id 35928 payload 824 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6ccc09b510 msgr2=0x7f6ccc1d1500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6ccc09b510 0x7f6ccc1d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6ccc0a42c0 msgr2=0x7f6ccc1d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6ccc0a42c0 0x7f6ccc1d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 monclient: found mon.noname-a 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:36.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0a7ff0 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6ccaffd640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 0x7f6ccc1d1a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6cc001f2d0 tx=0x7f6cc001b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 client.?.objecter ms_handle_connect 0x7f6ccc0a25a0 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cc00b4040 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: dump: 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.807 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: _finish_auth 0 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc00bd070 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cc00044b0 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 5 monclient: authenticate success, global_id 35928 2024-09-17T16:39:36.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _renew_subs 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc097a40 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cc97fa640 4 set_mon_vals no callback set 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cb3fff640 10 monclient: dump: 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.810 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 client.35928.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:36.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _renew_subs 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc0a36c0 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.776+0000 7f6cd1ea1640 1 librados: init done 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cd1ea1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 0x7f6ccc094140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6ccc051e80 con 0x7f6ccc093d60 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 0x7f6ccc094140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient: get_auth_request con 0x7f6ccc093d60 auth_method 0 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6cc0004040 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_done global_id 54458 payload 824 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6cb803e550 0x7f6cb8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ccc09b8f0 con 0x7f6ccc093d60 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(446..446 src has 1..446) ==== 5845+0+0 (secure 0 0 0) 0x7f6cc00ea000 con 0x7f6ccc0a25a0 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 client.35928.objecter ms_dispatch 0x7f6ccc000ba0 osd_map(446..446 src has 1..446) 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 3 client.35928.objecter handle_osd_map got epochs [446,446] > 0 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 3 client.35928.objecter handle_osd_map decoding full epoch 446 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 20 client.35928.objecter dump_active .. 0 homeless 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccb7fe640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 0x7f6ccc094140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc00ec30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 client.35928.objecter ms_handle_connect 0x7f6ccc093d60 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 client.35928.objecter resend_mon_ops 2024-09-17T16:39:36.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cbc017070 con 0x7f6ccc093d60 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 monclient: dump: 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6cbc00ee80 con 0x7f6ccc093d60 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 msgr2=0x7f6ccc094140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 0x7f6ccc094140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc00ec30 comp rx=0 tx=0).stop 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 msgr2=0x7f6ccc094140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc093d60 0x7f6ccc094140 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc00ec30 comp rx=0 tx=0).stop 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6cb803e550 0x7f6cb8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccbfff640 10 monclient: get_auth_request con 0x7f6cb803e550 auth_method 0 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6cb3fff640 10 client.35928.objecter ms_handle_connect 0x7f6cb803e550 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.780+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6cb803e550 0x7f6cb8040a10 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009c10 tx=0x7f6cc400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.784+0000 7f6cd1ea1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.784+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 0x7f6ccc071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.784+0000 7f6cd1ea1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.784+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6ccc06a1c0 con 0x7f6ccc094680 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 0x7f6ccc071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient: get_auth_request con 0x7f6ccc094680 auth_method 0 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 10 monclient(hunting): handle_auth_done global_id 54467 payload 824 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 1 -- 172.21.15.105:0/3028714647 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6ccc0956e0 con 0x7f6ccc094680 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccb7fe640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 0x7f6ccc071ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 client.35928.objecter ms_handle_connect 0x7f6ccc094680 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 client.35928.objecter resend_mon_ops 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6cbc017070 con 0x7f6ccc094680 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:36.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 monclient: dump: 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:36.818 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f6cbc0062d0 con 0x7f6ccc094680 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 msgr2=0x7f6ccc071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 0x7f6ccc071ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc004740 comp rx=0 tx=0).stop 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 msgr2=0x7f6ccc071ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:36.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cb3fff640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6ccc094680 0x7f6ccc071ff0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e770 tx=0x7f6cbc004740 comp rx=0 tx=0).stop 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 283786, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:36.820 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.821 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:36.822 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:36.823 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:36.825 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:36.826 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.828 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.830 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.831 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:36.832 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.834 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:36.836 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.837 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.838 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:36.839 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.841 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:36.842 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:36.843 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 10 librados: watch_flush enter 2024-09-17T16:39:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 10 librados: watch_flush exit 2024-09-17T16:39:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 20 client.35928.objecter shutdown clearing up homeless session... 2024-09-17T16:39:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 10 client.35928.objecter successfully canceled tick 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6cb803e550 msgr2=0x7f6cb8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6cb803e550 0x7f6cb8040a10 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009c10 tx=0x7f6cc400b040 comp rx=0 tx=0).stop 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 10 monclient: shutdown 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 msgr2=0x7f6ccc1d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 --2- 172.21.15.105:0/3028714647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ccc0a25a0 0x7f6ccc1d1a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6cc001f2d0 tx=0x7f6cc001b7d0 comp rx=0 tx=0).stop 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccbfff640 1 -- 172.21.15.105:0/3028714647 reap_dead start 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6ccbfff640 1 -- 172.21.15.105:0/3028714647 reap_dead start 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 shutdown_connections 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 >> 172.21.15.105:0/3028714647 conn(0x7f6ccc0a4e70 msgr2=0x7f6ccc09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 shutdown_connections 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 -- 172.21.15.105:0/3028714647 wait complete. 2024-09-17T16:39:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.788+0000 7f6cd1ea1640 1 librados: shutdown 2024-09-17T16:39:36.846 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:36.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:39:36.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:39:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: get_monmap_and_config 2024-09-17T16:39:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: build_initial_monmap 2024-09-17T16:39:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: monmap: 2024-09-17T16:39:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:36.983993+0000 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:36.983993+0000 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:36.989 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 Processor -- start 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 -- start start 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: init 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient: _add_conns ranks=[9,7,10,6,3,8,0,2,4,5,1] 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80780a56e0 0x7f807809dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-a con 0x7f80780a56e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80780a3b00 0x7f80780a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-i con 0x7f80780a3b00 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 0x7f80781442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-f con 0x7f80780a4480 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f807809f0d0 con 0x7f80780a56e0 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f80780a8f40 con 0x7f80780a4480 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f80780a5ac0 con 0x7f80780a3b00 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807d0c4640 10 monclient(hunting): authenticate will time out at 1660.249559s 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80780a56e0 0x7f807809dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:36.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80780a56e0 0x7f807809dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39590/0 (socket says 172.21.15.105:39590) 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 -- 172.21.15.105:0/2497590738 learned_addr learned my addr 172.21.15.105:0/2497590738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): get_auth_request con 0x7f80780a56e0 auth_method 0 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807759e640 1 --2- 172.21.15.105:0/2497590738 >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 0x7f80781442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient(hunting): handle_auth_done global_id 35937 payload 274 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 -- 172.21.15.105:0/2497590738 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80780a3b00 msgr2=0x7f80780a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 --2- 172.21.15.105:0/2497590738 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80780a3b00 0x7f80780a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 -- 172.21.15.105:0/2497590738 >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 msgr2=0x7f80781442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 --2- 172.21.15.105:0/2497590738 >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 0x7f80781442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 monclient: found mon.noname-a 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f8076d9d640 1 -- 172.21.15.105:0/2497590738 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780774a0 con 0x7f80780a56e0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807759e640 10 monclient: get_auth_request con 0x7f80780a4480 auth_method 0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.980+0000 7f807759e640 1 --2- 172.21.15.105:0/2497590738 >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 0x7f80781442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 1 --2- 172.21.15.105:0/2497590738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80780a56e0 0x7f807809dc80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80780a2740 tx=0x7f806801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b77d17080cfdc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 1 -- 172.21.15.105:0/2497590738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80680b0070 con 0x7f80780a56e0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: dump: 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 1 -- 172.21.15.105:0/2497590738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80680aa810 con 0x7f80780a56e0 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 1 -- 172.21.15.105:0/2497590738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80680a9300 con 0x7f80780a56e0 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8075d9b640 10 monclient: dump: 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 5 monclient: authenticate success, global_id 35937 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 4 set_mon_vals no callback set 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: shutdown 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/2497590738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80780a56e0 msgr2=0x7f807809dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- 172.21.15.105:0/2497590738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80780a56e0 0x7f807809dc80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80780a2740 tx=0x7f806801c380 comp rx=0 tx=0).stop 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/2497590738 shutdown_connections 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- 172.21.15.105:0/2497590738 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f80780a3b00 0x7f80780a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- 172.21.15.105:0/2497590738 >> v2:172.21.15.160:3300/0 conn(0x7f80780a4480 0x7f80781442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- 172.21.15.105:0/2497590738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80780a56e0 0x7f807809dc80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/2497590738 >> 172.21.15.105:0/2497590738 conn(0x7f80780a4bf0 msgr2=0x7f80780a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/2497590738 shutdown_connections 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/2497590738 wait complete. 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: monmap: 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:37.006 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 librados: starting msgr at 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 librados: starting objecter 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 Processor -- start 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- start start 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 librados: setting wanted keys 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 librados: calling monclient init 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: init 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _add_conns ranks=[3,2,8,4,7,6,5,0,1] 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80780a3b00 0x7f80781d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-g con 0x7f80780a3b00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 0x7f8078073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-b con 0x7f80780a4480 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f80780a56e0 0x7f80780743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): picked mon.noname-e con 0x7f80780a56e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f80781475e0 con 0x7f80780a4480 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8078147460 con 0x7f80780a3b00 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8078147760 con 0x7f80780a56e0 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient(hunting): authenticate will time out at 1660.254625s 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 0x7f8078073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f80780a56e0 0x7f80780743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 0x7f8078073e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59246/0 (socket says 172.21.15.105:59246) 2024-09-17T16:39:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 -- 172.21.15.105:0/3887034144 learned_addr learned my addr 172.21.15.105:0/3887034144 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): get_auth_request con 0x7f80780a4480 auth_method 0 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): get_auth_request con 0x7f80780a56e0 auth_method 0 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80780a3b00 0x7f80781d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 10 monclient(hunting): get_auth_request con 0x7f80780a3b00 auth_method 0 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f8076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient(hunting): handle_auth_done global_id 44975 payload 824 2024-09-17T16:39:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f80780a56e0 msgr2=0x7f80780743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f80780a56e0 0x7f80780743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80780a3b00 msgr2=0x7f80781d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80780a3b00 0x7f80781d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 monclient: found mon.noname-b 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8078074900 con 0x7f80780a4480 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807659c640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 0x7f8078073e80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b480 tx=0x7f806c00b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 client.?.objecter ms_handle_connect 0x7f80780a4480 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f806c014070 con 0x7f80780a4480 2024-09-17T16:39:37.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: dump: 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.013 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c010ae0 con 0x7f80780a4480 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f806c010cd0 con 0x7f80780a4480 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f805f7fe640 10 monclient: dump: 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.015 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807559a640 4 set_mon_vals no callback set 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 5 monclient: authenticate success, global_id 44975 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _renew_subs 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781d5e10 con 0x7f80780a4480 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 client.44975.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _renew_subs 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80780710a0 con 0x7f80780a4480 2024-09-17T16:39:37.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.984+0000 7f807d0c4640 1 librados: init done 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f806c017070 con 0x7f80780a4480 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807d0c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807d0c4640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 0x7f8078099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f80780718a0 con 0x7f8078099700 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f806403e550 0x7f8064040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(447..447 src has 1..447) ==== 6217+0+0 (secure 0 0 0) 0x7f806c04fc40 con 0x7f80780a4480 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 client.44975.objecter ms_dispatch 0x7f8078000ba0 osd_map(447..447 src has 1..447) 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 3 client.44975.objecter handle_osd_map got epochs [447,447] > 0 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 3 client.44975.objecter handle_osd_map decoding full epoch 447 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 20 client.44975.objecter dump_active .. 0 homeless 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 0x7f8078099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807759e640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f806403e550 0x7f8064040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient: get_auth_request con 0x7f8078099700 auth_method 0 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807759e640 10 monclient: get_auth_request con 0x7f806403e550 auth_method 0 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 10 monclient(hunting): handle_auth_done global_id 54550 payload 824 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80780a3f40 con 0x7f8078099700 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 client.44975.objecter ms_handle_connect 0x7f806403e550 2024-09-17T16:39:37.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f807759e640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f806403e550 0x7f8064040a10 secure :-1 s=READY pgs=3554 cs=0 l=1 rev1=1 crypto rx=0x7f80781d6840 tx=0x7f8070007780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 client.44975.objecter ms_handle_connect 0x7f8078099700 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 client.44975.objecter resend_mon_ops 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f8076d9d640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 0x7f8078099ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80780a9110 tx=0x7f8068019180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80680b0050 con 0x7f8078099700 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 monclient: dump: 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.019 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f80680b1040 con 0x7f8078099700 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 msgr2=0x7f8078099ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 0x7f8078099ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80780a9110 tx=0x7f8068019180 comp rx=0 tx=0).stop 2024-09-17T16:39:37.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 msgr2=0x7f8078099ae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.988+0000 7f805f7fe640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8078099700 0x7f8078099ae0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80780a9110 tx=0x7f8068019180 comp rx=0 tx=0).stop 2024-09-17T16:39:37.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 0x7f80781427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8078051e80 con 0x7f807809a020 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 0x7f80781427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient: get_auth_request con 0x7f807809a020 auth_method 0 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 10 monclient(hunting): handle_auth_done global_id 54559 payload 824 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 1 -- 172.21.15.105:0/3887034144 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8078071440 con 0x7f807809a020 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f8076d9d640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 0x7f80781427a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8068000c10 tx=0x7f80680a8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 client.44975.objecter ms_handle_connect 0x7f807809a020 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 client.44975.objecter resend_mon_ops 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80680abc40 con 0x7f807809a020 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 monclient: dump: 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.024 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f80680c1070 con 0x7f807809a020 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 msgr2=0x7f80781427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 0x7f80781427a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8068000c10 tx=0x7f80680a8000 comp rx=0 tx=0).stop 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 msgr2=0x7f80781427a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f805f7fe640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f807809a020 0x7f80781427a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 284005, 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:37.026 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:37.027 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.028 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.029 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:37.030 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.031 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.032 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:37.033 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.034 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:37.036 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.041 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.044 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.046 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.047 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:37.048 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 librados: watch_flush enter 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 librados: watch_flush exit 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 20 client.44975.objecter shutdown clearing up homeless session... 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 client.44975.objecter successfully canceled tick 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f806403e550 msgr2=0x7f8064040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f806403e550 0x7f8064040a10 secure :-1 s=READY pgs=3554 cs=0 l=1 rev1=1 crypto rx=0x7f80781d6840 tx=0x7f8070007780 comp rx=0 tx=0).stop 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 10 monclient: shutdown 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 msgr2=0x7f8078073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 --2- 172.21.15.105:0/3887034144 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80780a4480 0x7f8078073e80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b480 tx=0x7f806c00b940 comp rx=0 tx=0).stop 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807759e640 1 -- 172.21.15.105:0/3887034144 reap_dead start 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807759e640 1 -- 172.21.15.105:0/3887034144 reap_dead start 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 shutdown_connections 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 >> 172.21.15.105:0/3887034144 conn(0x7f80780a4bf0 msgr2=0x7f8078141460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 shutdown_connections 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 -- 172.21.15.105:0/3887034144 wait complete. 2024-09-17T16:39:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:36.996+0000 7f807d0c4640 1 librados: shutdown 2024-09-17T16:39:37.051 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:37.051 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:39:37.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:39:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: get_monmap_and_config 2024-09-17T16:39:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: monmap: 2024-09-17T16:39:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.163 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:37.160006+0000 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:37.160006+0000 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.164 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.165 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.165 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:37.165 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:37.165 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 Processor -- start 2024-09-17T16:39:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 -- start start 2024-09-17T16:39:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: init 2024-09-17T16:39:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient: _add_conns ranks=[3,7,5,10,9,8,6,1,4,2,0] 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 0x7fd23c0a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-g con 0x7fd23c0a3720 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 0x7fd23c0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-i con 0x7fd23c0a3b00 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 0x7fd23c0a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-h con 0x7fd23c0a3f00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd23c06db50 con 0x7fd23c0a3720 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd23c0a0e80 con 0x7fd23c0a3f00 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fd23c0a1490 con 0x7fd23c0a3b00 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd243c06640 10 monclient(hunting): authenticate will time out at 1660.425659s 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 0x7fd23c0a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 0x7fd23c0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 0x7fd23c0a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 0x7fd23c0a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:56926/0 (socket says 172.21.15.105:56926) 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3b00 auth_method 0 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 1 -- 172.21.15.105:0/86445637 learned_addr learned my addr 172.21.15.105:0/86445637 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3f00 auth_method 0 2024-09-17T16:39:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3720 auth_method 0 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient(hunting): handle_auth_done global_id 74120 payload 274 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 -- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 msgr2=0x7fd23c0a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 0x7fd23c0a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 -- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 msgr2=0x7fd23c0a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 0x7fd23c0a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 monclient: found mon.noname-h 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2421a3640 1 -- 172.21.15.105:0/86445637 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c0a7240 con 0x7fd23c0a3f00 2024-09-17T16:39:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.156+0000 7fd2411a1640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 0x7fd23c0a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 0x7fd23c0a5c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009dc0 tx=0x7fd22c01c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=12c7a8bc990f90e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 1 -- 172.21.15.105:0/86445637 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd22c0b2070 con 0x7fd23c0a3f00 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: dump: 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 1 -- 172.21.15.105:0/86445637 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c0a59b0 con 0x7fd23c0a3f00 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 5 monclient: authenticate success, global_id 74120 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 1 -- 172.21.15.105:0/86445637 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd22c0a5be0 con 0x7fd23c0a3f00 2024-09-17T16:39:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 4 set_mon_vals no callback set 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2409a0640 10 monclient: dump: 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: shutdown 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 msgr2=0x7fd23c0a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 0x7fd23c0a5c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009dc0 tx=0x7fd22c01c850 comp rx=0 tx=0).stop 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- 172.21.15.105:0/86445637 shutdown_connections 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3f00 0x7fd23c0a5c00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd23c0a3b00 0x7fd23c0a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- 172.21.15.105:0/86445637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3720 0x7fd23c0a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- 172.21.15.105:0/86445637 >> 172.21.15.105:0/86445637 conn(0x7fd23c0aa390 msgr2=0x7fd23c0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- 172.21.15.105:0/86445637 shutdown_connections 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- 172.21.15.105:0/86445637 wait complete. 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: monmap: 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 librados: starting msgr at 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 librados: starting objecter 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 Processor -- start 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- start start 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 librados: setting wanted keys 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 librados: calling monclient init 2024-09-17T16:39:37.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: init 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient: _add_conns ranks=[5,1,3,0,4,6,2,7,8] 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3720 0x7fd23c139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-h con 0x7fd23c0a3720 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd23c0a3b00 0x7fd23c13a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-f con 0x7fd23c0a3b00 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 0x7fd23c13a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): picked mon.noname-g con 0x7fd23c0a3f00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd23c1432d0 con 0x7fd23c0a3b00 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd23c0a6550 con 0x7fd23c0a3f00 2024-09-17T16:39:37.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd23c143450 con 0x7fd23c0a3720 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd243c06640 10 monclient(hunting): authenticate will time out at 1660.431567s 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3720 0x7fd23c139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2411a1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd23c0a3b00 0x7fd23c13a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 0x7fd23c13a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3720 0x7fd23c139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:33820/0 (socket says 172.21.15.105:33820) 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 1 -- 172.21.15.105:0/3430158426 learned_addr learned my addr 172.21.15.105:0/3430158426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3720 auth_method 0 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2419a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3f00 auth_method 0 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2421a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2411a1640 10 monclient(hunting): get_auth_request con 0x7fd23c0a3b00 auth_method 0 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2411a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2411a1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.160+0000 7fd2411a1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2411a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient(hunting): handle_auth_done global_id 54525 payload 824 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3720 msgr2=0x7fd23c139ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd23c0a3720 0x7fd23c139ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd23c0a3b00 msgr2=0x7fd23c13a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd23c0a3b00 0x7fd23c13a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 monclient: found mon.noname-g 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:39:37.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23c125dc0 con 0x7fd23c0a3f00 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2421a3640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 0x7fd23c13a950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0ac000 tx=0x7fd22c0195f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 client.?.objecter ms_handle_connect 0x7fd23c0a3f00 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd22c0b2050 con 0x7fd23c0a3f00 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: dump: 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c0b3040 con 0x7fd23c0a3f00 2024-09-17T16:39:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 5 monclient: authenticate success, global_id 54525 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 monclient: _renew_subs 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23c13ca00 con 0x7fd23c0a3f00 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd22c003bb0 con 0x7fd23c0a3f00 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: dump: 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 client.54525.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23bfff640 4 set_mon_vals no callback set 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 monclient: _renew_subs 2024-09-17T16:39:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23c125ae0 con 0x7fd23c0a3f00 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 1 librados: init done 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 0x7fd23c071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd23c0a02d0 con 0x7fd23c071090 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 0x7fd23c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient: get_auth_request con 0x7fd23c071090 auth_method 0 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 10 monclient(hunting): handle_auth_done global_id 54534 payload 824 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd23c0a1e70 con 0x7fd23c071090 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd22c0c0030 con 0x7fd23c0a3f00 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd22803e580 0x7fd228040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(447..447 src has 1..447) ==== 6217+0+0 (secure 0 0 0) 0x7fd22c0be020 con 0x7fd23c0a3f00 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 client.54525.objecter ms_dispatch 0x7fd23c000ba0 osd_map(447..447 src has 1..447) 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 3 client.54525.objecter handle_osd_map got epochs [447,447] > 0 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 3 client.54525.objecter handle_osd_map decoding full epoch 447 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2419a2640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 0x7fd23c071470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd234011e30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 20 client.54525.objecter dump_active .. 0 homeless 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 client.54525.objecter ms_handle_connect 0x7fd23c071090 2024-09-17T16:39:37.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 client.54525.objecter resend_mon_ops 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2411a1640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd22803e580 0x7fd228040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd2411a1640 10 monclient: get_auth_request con 0x7fd22803e580 auth_method 0 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd234019070 con 0x7fd23c071090 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.164+0000 7fd23a7fc640 10 monclient: dump: 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.195 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 10 client.54525.objecter ms_handle_connect 0x7fd22803e580 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd2411a1640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd22803e580 0x7fd228040a40 secure :-1 s=READY pgs=3555 cs=0 l=1 rev1=1 crypto rx=0x7fd23c074290 tx=0x7fd230009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd234004300 con 0x7fd23c071090 2024-09-17T16:39:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 msgr2=0x7fd23c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 0x7fd23c071470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd234011e30 comp rx=0 tx=0).stop 2024-09-17T16:39:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 msgr2=0x7fd23c071470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.168+0000 7fd23a7fc640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c071090 0x7fd23c071470 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd234011e30 comp rx=0 tx=0).stop 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd243c06640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd243c06640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 0x7fd23c09e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd243c06640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd23c1248a0 con 0x7fd23c0a8f80 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 0x7fd23c09e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient: get_auth_request con 0x7fd23c0a8f80 auth_method 0 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 10 monclient(hunting): handle_auth_done global_id 54543 payload 824 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 1 -- 172.21.15.105:0/3430158426 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd23c0a0950 con 0x7fd23c0a8f80 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd2419a2640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 0x7fd23c09e950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd2340046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd23a7fc640 10 client.54525.objecter ms_handle_connect 0x7fd23c0a8f80 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.172+0000 7fd23a7fc640 10 client.54525.objecter resend_mon_ops 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd234019040 con 0x7fd23c0a8f80 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 10 monclient: dump: 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fd23401e450 con 0x7fd23c0a8f80 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 msgr2=0x7fd23c09e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 0x7fd23c09e950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd2340046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 msgr2=0x7fd23c09e950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd23a7fc640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a8f80 0x7fd23c09e950 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd234009fd0 tx=0x7fd2340046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 284181, 2024-09-17T16:39:37.202 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.203 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.204 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.205 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:37.206 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.207 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.208 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.209 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.210 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:37.212 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.213 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.214 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.215 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.216 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.217 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.219 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.221 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.223 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.224 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:37.225 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 10 librados: watch_flush enter 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 10 librados: watch_flush exit 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 20 client.54525.objecter shutdown clearing up homeless session... 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 10 client.54525.objecter successfully canceled tick 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd22803e580 msgr2=0x7fd228040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd22803e580 0x7fd228040a40 secure :-1 s=READY pgs=3555 cs=0 l=1 rev1=1 crypto rx=0x7fd23c074290 tx=0x7fd230009190 comp rx=0 tx=0).stop 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 10 monclient: shutdown 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 msgr2=0x7fd23c13a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd2421a3640 1 -- 172.21.15.105:0/3430158426 reap_dead start 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 0x7fd23c13a950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0ac000 tx=0x7fd22c0195f0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 shutdown_connections 2024-09-17T16:39:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 --2- 172.21.15.105:0/3430158426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd23c0a3f00 0x7fd23c13a950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 >> 172.21.15.105:0/3430158426 conn(0x7fd23c0aa390 msgr2=0x7fd23c0a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 shutdown_connections 2024-09-17T16:39:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 -- 172.21.15.105:0/3430158426 wait complete. 2024-09-17T16:39:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.176+0000 7fd243c06640 1 librados: shutdown 2024-09-17T16:39:37.228 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:37.228 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:39:37.228 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:39:37.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: get_monmap_and_config 2024-09-17T16:39:37.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: monmap: 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:37.341021+0000 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:37.341021+0000 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:37.346 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 Processor -- start 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 -- start start 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: init 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient: _add_conns ranks=[3,7,1,10,0,9,4,8,2,5,6] 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-g con 0x7f5f280a6a30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f280a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-i con 0x7f5f280a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5f281397f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:39:37.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f280a4080 con 0x7f5f281397f0 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280a7c40 con 0x7f5f280a6a30 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5f280a7890 con 0x7f5f280a4ab0 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.336+0000 7f5f30264640 10 monclient(hunting): authenticate will time out at 1660.607670s 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:56970/0 (socket says 172.21.15.105:56970) 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 -- 172.21.15.105:0/4288910347 learned_addr learned my addr 172.21.15.105:0/4288910347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): get_auth_request con 0x7f5f280a6a30 auth_method 0 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f280a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): get_auth_request con 0x7f5f280a4ab0 auth_method 0 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2d7ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient(hunting): handle_auth_done global_id 54552 payload 274 2024-09-17T16:39:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 -- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 msgr2=0x7f5f280a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f280a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 -- 172.21.15.105:0/4288910347 >> v1:172.21.15.160:6789/0 conn(0x7f5f281397f0 legacy=0x7f5f28139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 monclient: found mon.noname-g 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:39:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 -- 172.21.15.105:0/4288910347 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280a37e0 con 0x7f5f280a6a30 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2e000640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f240098a0 tx=0x7f5f2401c4a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=74aad6cba54df93d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 1 -- 172.21.15.105:0/4288910347 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f240b1070 con 0x7f5f280a6a30 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: dump: 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.359 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 5 monclient: authenticate success, global_id 54552 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 1 -- 172.21.15.105:0/4288910347 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240ab810 con 0x7f5f280a6a30 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 4 set_mon_vals no callback set 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 1 -- 172.21.15.105:0/4288910347 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f240ab9c0 con 0x7f5f280a6a30 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f2cffe640 10 monclient: dump: 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.361 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.362 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 10 monclient: shutdown 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 -- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 msgr2=0x7f5f280a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f240098a0 tx=0x7f5f2401c4a0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 -- 172.21.15.105:0/4288910347 shutdown_connections 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f280a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 --2- 172.21.15.105:0/4288910347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5f280a6a30 0x7f5f280a6e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.340+0000 7f5f30264640 1 -- 172.21.15.105:0/4288910347 >> 172.21.15.105:0/4288910347 conn(0x7f5f280a8d00 msgr2=0x7f5f280a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- 172.21.15.105:0/4288910347 shutdown_connections 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- 172.21.15.105:0/4288910347 wait complete. 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: monmap: 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.364 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 librados: starting msgr at 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 librados: starting objecter 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 Processor -- start 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- start start 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 librados: setting wanted keys 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 librados: calling monclient init 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: init 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _add_conns ranks=[7,5,0,6,2,3,8,1,4] 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f281d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-i con 0x7f5f280a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:37.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f280a6a30 0x7f5f281d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-h con 0x7f5f280a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): picked mon.noname-a con 0x7f5f281397f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28144270 con 0x7f5f281397f0 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5f2806e770 con 0x7f5f280a6a30 2024-09-17T16:39:37.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f5f281443f0 con 0x7f5f280a4ab0 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): authenticate will time out at 1660.613851s 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39600/0 (socket says 172.21.15.105:39600) 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 -- 172.21.15.105:0/1262989846 learned_addr learned my addr 172.21.15.105:0/1262989846 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): get_auth_request con 0x7f5f281397f0 auth_method 0 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f281d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f280a6a30 0x7f5f281d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): get_auth_request con 0x7f5f280a4ab0 auth_method 0 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 10 monclient(hunting): get_auth_request con 0x7f5f280a6a30 auth_method 0 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient(hunting): handle_auth_done global_id 35946 payload 824 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 msgr2=0x7f5f281d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f5f280a4ab0 0x7f5f281d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f280a6a30 msgr2=0x7f5f281d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f280a6a30 0x7f5f281d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 monclient: found mon.noname-a 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2813eea0 con 0x7f5f281397f0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2d7ff640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f280a6a30 0x7f5f281d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f2e801640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b9e0 tx=0x7f5f2000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 client.?.objecter ms_handle_connect 0x7f5f281397f0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f2000cd10 con 0x7f5f281397f0 2024-09-17T16:39:37.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: dump: 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.372 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f20014440 con 0x7f5f281397f0 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 5 monclient: authenticate success, global_id 35946 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _renew_subs 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f280a9f00 con 0x7f5f281397f0 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f20014ae0 con 0x7f5f281397f0 2024-09-17T16:39:37.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1e7fc640 10 monclient: dump: 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.374 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f1f7fe640 4 set_mon_vals no callback set 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 client.35946.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _renew_subs 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f28122e20 con 0x7f5f281397f0 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 librados: init done 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 0x7f5f2813d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.344+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5f28051e80 con 0x7f5f2813d3b0 2024-09-17T16:39:37.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5f2001b020 con 0x7f5f281397f0 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 0x7f5f2813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient: get_auth_request con 0x7f5f2813d3b0 auth_method 0 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5f0c03e550 0x7f5f0c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(447..447 src has 1..447) ==== 6217+0+0 (secure 0 0 0) 0x7f5f2004f980 con 0x7f5f281397f0 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 client.35946.objecter ms_dispatch 0x7f5f28000ba0 osd_map(447..447 src has 1..447) 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 3 client.35946.objecter handle_osd_map got epochs [447,447] > 0 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 3 client.35946.objecter handle_osd_map decoding full epoch 447 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 20 client.35946.objecter dump_active .. 0 homeless 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2d7ff640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5f0c03e550 0x7f5f0c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2d7ff640 10 monclient: get_auth_request con 0x7f5f0c03e550 auth_method 0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 10 monclient(hunting): handle_auth_done global_id 74147 payload 824 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f280008e0 con 0x7f5f2813d3b0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 client.35946.objecter ms_handle_connect 0x7f5f2813d3b0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 client.35946.objecter resend_mon_ops 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2e000640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 0x7f5f2813d790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f2806eab0 tx=0x7f5f2401ace0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 client.35946.objecter ms_handle_connect 0x7f5f0c03e550 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f2d7ff640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5f0c03e550 0x7f5f0c040a10 secure :-1 s=READY pgs=3556 cs=0 l=1 rev1=1 crypto rx=0x7f5f2813eb10 tx=0x7f5f18008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f240bb070 con 0x7f5f2813d3b0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 monclient: dump: 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.378 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5f24003760 con 0x7f5f2813d3b0 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 msgr2=0x7f5f2813d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 0x7f5f2813d790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f2806eab0 tx=0x7f5f2401ace0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 msgr2=0x7f5f2813d790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.348+0000 7f5f1e7fc640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813d3b0 0x7f5f2813d790 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f2806eab0 tx=0x7f5f2401ace0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 0x7f5f280a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f5f2806a200 con 0x7f5f2813dcd0 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 0x7f5f280a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient: get_auth_request con 0x7f5f2813dcd0 auth_method 0 2024-09-17T16:39:37.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 10 monclient(hunting): handle_auth_done global_id 74156 payload 824 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 1 -- 172.21.15.105:0/1262989846 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5f28140960 con 0x7f5f2813dcd0 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e000640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 0x7f5f280a0a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f2401c2d0 tx=0x7f5f24019240 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 client.35946.objecter ms_handle_connect 0x7f5f2813dcd0 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 client.35946.objecter resend_mon_ops 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5f240bb040 con 0x7f5f2813dcd0 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:39:37.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 monclient: dump: 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.383 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f5f240b6ae0 con 0x7f5f2813dcd0 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 msgr2=0x7f5f280a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 0x7f5f280a0a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f2401c2d0 tx=0x7f5f24019240 comp rx=0 tx=0).stop 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 msgr2=0x7f5f280a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f1e7fc640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5f2813dcd0 0x7f5f280a0a20 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f2401c2d0 tx=0x7f5f24019240 comp rx=0 tx=0).stop 2024-09-17T16:39:37.384 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16827, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:37.385 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.387 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.388 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.389 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.391 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.392 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.393 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.394 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.395 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.396 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:37.397 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.399 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:37.400 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.401 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:37.403 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.405 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:37.406 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 librados: watch_flush enter 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 librados: watch_flush exit 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 20 client.35946.objecter shutdown clearing up homeless session... 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 client.35946.objecter successfully canceled tick 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5f0c03e550 msgr2=0x7f5f0c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f5f0c03e550 0x7f5f0c040a10 secure :-1 s=READY pgs=3556 cs=0 l=1 rev1=1 crypto rx=0x7f5f2813eb10 tx=0x7f5f18008040 comp rx=0 tx=0).stop 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 10 monclient: shutdown 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 msgr2=0x7f5f281d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f2e801640 1 -- 172.21.15.105:0/1262989846 reap_dead start 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b9e0 tx=0x7f5f2000bea0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 shutdown_connections 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 --2- 172.21.15.105:0/1262989846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f281397f0 0x7f5f281d1d80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 >> 172.21.15.105:0/1262989846 conn(0x7f5f280a8d00 msgr2=0x7f5f280a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 shutdown_connections 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 -- 172.21.15.105:0/1262989846 wait complete. 2024-09-17T16:39:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.356+0000 7f5f30264640 1 librados: shutdown 2024-09-17T16:39:37.410 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:37.410 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:39:37.410 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: get_monmap_and_config 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: monmap: 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:37.523893+0000 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:37.523893+0000 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:37.529 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 Processor -- start 2024-09-17T16:39:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 -- start start 2024-09-17T16:39:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: init 2024-09-17T16:39:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient: _add_conns ranks=[0,2,1,10,7,8,4,9,5,3,6] 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fccf00a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccf013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-b con 0x7fccf00a5240 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fccf009e660 addr v1:172.21.15.160:6789/0 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf00a61d0 con 0x7fccf00a4750 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf00a6410 con 0x7fccf009e660 2024-09-17T16:39:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fccf00a66c0 con 0x7fccf00a5240 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 10 monclient(hunting): authenticate will time out at 1660.789652s 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fccf00a4750 0x7fccf00a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54064/0 (socket says 172.21.15.105:54064) 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 -- 172.21.15.105:0/10836807 learned_addr learned my addr 172.21.15.105:0/10836807 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccf013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): get_auth_request con 0x7fccf00a5240 auth_method 0 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881640431 0 0) 0x7fccf00a61d0 con 0x7fccf00a4750 2024-09-17T16:39:37.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): _negotiate 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): my global_id is 35955 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd8003530 con 0x7fccf00a4750 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 927587614 0 0) 0x7fccd8003530 con 0x7fccf00a4750 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 msgr2=0x7fccf013e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccf013e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 >> v1:172.21.15.160:6789/0 conn(0x7fccf009e660 legacy=0x7fccf0140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf009dc10 con 0x7fccf00a4750 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7fcce8001cf0 con 0x7fccf00a4750 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 5 monclient: authenticate success, global_id 35955 2024-09-17T16:39:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: dump: 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.545 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 >> v1:172.21.15.105:6789/0 conn(0x7fccf00a4750 legacy=0x7fccf00a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient: _add_conns ranks=[2,5,0,1,7,3,8,6,4] 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccd8004240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): picked mon.b con 0x7fccf00a5240 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccd8004860 0x7fccd8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): picked mon.h con 0x7fccd8004860 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd80099c0 0x7fccd800bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): picked mon.a con 0x7fccd80099c0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fccd800c480 con 0x7fccd80099c0 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fccd800c600 con 0x7fccf00a5240 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fccd800c7b0 con 0x7fccd8004860 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf4c5b640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf7ec1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccd8004240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd80099c0 0x7fccd800bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request con 0x7fccf00a5240 auth_method 0 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): get_auth_request con 0x7fccd80099c0 auth_method 0 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccd8004860 0x7fccd8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): get_auth_request con 0x7fccd8004860 auth_method 0 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf545c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_done global_id 35955 payload 293 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 -- 172.21.15.105:0/10836807 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccd8004860 msgr2=0x7fccd8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccd8004860 0x7fccd8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 -- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd80099c0 msgr2=0x7fccd800bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd80099c0 0x7fccd800bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 monclient: found mon.b 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 1 -- 172.21.15.105:0/10836807 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd8003530 con 0x7fccf00a5240 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf5c5d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.520+0000 7fccf645e640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd80099c0 0x7fccd800bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccd8004240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcce800a670 tx=0x7fcce801d0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f731eb30021032 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce800a260 con 0x7fccf00a5240 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: dump: 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.551 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce80a9030 con 0x7fccf00a5240 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 1 -- 172.21.15.105:0/10836807 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce80079b0 con 0x7fccf00a5240 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 4 set_mon_vals no callback set 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf4c5b640 10 monclient: dump: 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: shutdown 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 msgr2=0x7fccd8004240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/10836807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccf00a5240 0x7fccd8004240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcce800a670 tx=0x7fcce801d0a0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 1 -- 172.21.15.105:0/10836807 reap_dead start 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- 172.21.15.105:0/10836807 shutdown_connections 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- 172.21.15.105:0/10836807 >> 172.21.15.105:0/10836807 conn(0x7fccf00a93f0 msgr2=0x7fccf00a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- 172.21.15.105:0/10836807 shutdown_connections 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- 172.21.15.105:0/10836807 wait complete. 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: monmap: 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 librados: starting msgr at 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 librados: starting objecter 2024-09-17T16:39:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 Processor -- start 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- start start 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 librados: setting wanted keys 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 librados: calling monclient init 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: init 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient: _add_conns ranks=[5,8,6,4,3,1,0,2,7] 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf00a5240 0x7fccf0122330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-h con 0x7fccf00a5240 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 0x7fccf0122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-e con 0x7fccf0122870 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 0x7fccf01d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): picked mon.noname-d con 0x7fccf01d1120 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fccf0143080 con 0x7fccf00a5240 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fccf006b5b0 con 0x7fccf01d1120 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fccf0143200 con 0x7fccf0122870 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf7ec1640 10 monclient(hunting): authenticate will time out at 1660.795549s 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 0x7fccf0122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 0x7fccf01d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 0x7fccf0122c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:53296/0 (socket says 172.21.15.105:53296) 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 1 -- 172.21.15.105:0/3197060217 learned_addr learned my addr 172.21.15.105:0/3197060217 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): get_auth_request con 0x7fccf0122870 auth_method 0 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): get_auth_request con 0x7fccf01d1120 auth_method 0 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf645e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf00a5240 0x7fccf0122330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf545c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request con 0x7fccf00a5240 auth_method 0 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.524+0000 7fccf5c5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 10 monclient(hunting): handle_auth_done global_id 24648 payload 824 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 msgr2=0x7fccf0122c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 0x7fccf0122c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf00a5240 msgr2=0x7fccf0122330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf00a5240 0x7fccf0122330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 monclient: found mon.noname-d 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0125dd0 con 0x7fccf01d1120 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fccf00a5240 0x7fccf0122330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf545c640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fccf0122870 0x7fccf0122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf645e640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 0x7fccf01d1580 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fccf009db20 tx=0x7fcce000fb90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.?.objecter ms_handle_connect 0x7fccf01d1120 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce00013e0 con 0x7fccf01d1120 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: dump: 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.560 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce00067e0 con 0x7fccf01d1120 2024-09-17T16:39:37.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 5 monclient: authenticate success, global_id 24648 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 monclient: _renew_subs 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01260b0 con 0x7fccf01d1120 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce0006e00 con 0x7fccf01d1120 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: dump: 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.562 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.563 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce7fff640 4 set_mon_vals no callback set 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 client.24648.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 monclient: _renew_subs 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf0125af0 con 0x7fccf01d1120 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 1 librados: init done 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 0x7fccf01d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fccf0051e80 con 0x7fccf002e5d0 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fcce001b020 con 0x7fccf01d1120 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fccdc03e580 0x7fccdc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 0x7fccf01d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(447..447 src has 1..447) ==== 6217+0+0 (secure 0 0 0) 0x7fcce0019020 con 0x7fccf01d1120 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.24648.objecter ms_dispatch 0x7fccf0000ba0 osd_map(447..447 src has 1..447) 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 3 client.24648.objecter handle_osd_map got epochs [447,447] > 0 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 3 client.24648.objecter handle_osd_map decoding full epoch 447 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient: get_auth_request con 0x7fccf002e5d0 auth_method 0 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 20 client.24648.objecter dump_active .. 0 homeless 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf545c640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fccdc03e580 0x7fccdc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf545c640 10 monclient: get_auth_request con 0x7fccdc03e580 auth_method 0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_done global_id 64303 payload 824 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccf01265b0 con 0x7fccf002e5d0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.24648.objecter ms_handle_connect 0x7fccdc03e580 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf545c640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fccdc03e580 0x7fccdc040a40 secure :-1 s=READY pgs=3557 cs=0 l=1 rev1=1 crypto rx=0x7fccf01243c0 tx=0x7fccec008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.24648.objecter ms_handle_connect 0x7fccf002e5d0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 client.24648.objecter resend_mon_ops 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 0x7fccf01d1ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcce800a640 tx=0x7fccd8005840 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce80013d0 con 0x7fccf002e5d0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.528+0000 7fcce67fc640 10 monclient: dump: 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcce80a9020 con 0x7fccf002e5d0 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 msgr2=0x7fccf01d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 0x7fccf01d1ac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcce800a640 tx=0x7fccd8005840 comp rx=0 tx=0).stop 2024-09-17T16:39:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 msgr2=0x7fccf01d1ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.532+0000 7fcce67fc640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf002e5d0 0x7fccf01d1ac0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcce800a640 tx=0x7fccd8005840 comp rx=0 tx=0).stop 2024-09-17T16:39:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf7ec1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 0x7fccf0064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf7ec1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fccf0123db0 con 0x7fccf0063e00 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 0x7fccf0064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf5c5d640 10 monclient: get_auth_request con 0x7fccf0063e00 auth_method 0 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf5c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf5c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.536+0000 7fccf5c5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 10 monclient(hunting): handle_auth_done global_id 64312 payload 824 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 1 -- 172.21.15.105:0/3197060217 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fccf002e9b0 con 0x7fccf0063e00 2024-09-17T16:39:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf5c5d640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 0x7fccf0064210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fccf00a4f40 tx=0x7fcce80a7070 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 client.24648.objecter ms_handle_connect 0x7fccf0063e00 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 client.24648.objecter resend_mon_ops 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcce800a330 con 0x7fccf0063e00 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 monclient: dump: 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fcce80c1020 con 0x7fccf0063e00 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 msgr2=0x7fccf0064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 0x7fccf0064210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fccf00a4f40 tx=0x7fcce80a7070 comp rx=0 tx=0).stop 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 msgr2=0x7fccf0064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fcce67fc640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fccf0063e00 0x7fccf0064210 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fccf00a4f40 tx=0x7fcce80a7070 comp rx=0 tx=0).stop 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:37.573 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150642, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:37.574 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:37.575 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.576 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:37.577 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:37.578 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.579 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.580 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:37.582 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.583 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.585 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:37.586 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.587 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:37.588 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.590 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.592 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:37.597 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:37.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:37.598 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:37.598 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:37.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 10 librados: watch_flush enter 2024-09-17T16:39:37.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 10 librados: watch_flush exit 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 20 client.24648.objecter shutdown clearing up homeless session... 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 10 client.24648.objecter successfully canceled tick 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fccdc03e580 msgr2=0x7fccdc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fccdc03e580 0x7fccdc040a40 secure :-1 s=READY pgs=3557 cs=0 l=1 rev1=1 crypto rx=0x7fccf01243c0 tx=0x7fccec008040 comp rx=0 tx=0).stop 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 10 monclient: shutdown 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 msgr2=0x7fccf01d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf645e640 1 -- 172.21.15.105:0/3197060217 reap_dead start 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 0x7fccf01d1580 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fccf009db20 tx=0x7fcce000fb90 comp rx=0 tx=0).stop 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 shutdown_connections 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 --2- 172.21.15.105:0/3197060217 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fccf01d1120 0x7fccf01d1580 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 >> 172.21.15.105:0/3197060217 conn(0x7fccf00a93f0 msgr2=0x7fccf00a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 shutdown_connections 2024-09-17T16:39:37.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 -- 172.21.15.105:0/3197060217 wait complete. 2024-09-17T16:39:37.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.540+0000 7fccf7ec1640 1 librados: shutdown 2024-09-17T16:39:37.601 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:37.601 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:39:37.601 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:39:37.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.704+0000 7f1197ac1640 10 monclient: get_monmap_and_config 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.704+0000 7f1197ac1640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.704+0000 7f1197ac1640 10 monclient: monmap: 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:37.711055+0000 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:37.711055+0000 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.715 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:37.716 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:37.717 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 Processor -- start 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 -- start start 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient: init 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient: _add_conns ranks=[2,4,8,9,0,5,3,7,6,10,1] 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-b con 0x7f119006d220 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-c con 0x7f119006db40 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 0x7f1190139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-e con 0x7f119006df20 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:37.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f119006e800 con 0x7f119006d220 2024-09-17T16:39:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f11900a2200 con 0x7f119006db40 2024-09-17T16:39:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f11900a63b0 con 0x7f119006df20 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient(hunting): authenticate will time out at 1660.977012s 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 0x7f1190139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40304/0 (socket says 172.21.15.105:40304) 2024-09-17T16:39:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:59264/0 (socket says 172.21.15.105:59264) 2024-09-17T16:39:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 -- 172.21.15.105:0/1498069109 learned_addr learned my addr 172.21.15.105:0/1498069109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): get_auth_request con 0x7f119006db40 auth_method 0 2024-09-17T16:39:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): get_auth_request con 0x7f119006d220 auth_method 0 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): get_auth_request con 0x7f119006df20 auth_method 0 2024-09-17T16:39:37.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient(hunting): handle_auth_done global_id 74173 payload 274 2024-09-17T16:39:37.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 -- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 msgr2=0x7f1190139fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 0x7f1190139fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 -- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 msgr2=0x7f119006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 monclient: found mon.noname-c 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 -- 172.21.15.105:0/1498069109 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119006efb0 con 0x7f119006db40 2024-09-17T16:39:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119585d640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119605e640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 0x7f1190139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119505c640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f118c0098d0 tx=0x7f118c01c4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=547cf782dec8f221 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 1 -- 172.21.15.105:0/1498069109 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f118c0b0070 con 0x7f119006db40 2024-09-17T16:39:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: dump: 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.747 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 1 -- 172.21.15.105:0/1498069109 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c0aa810 con 0x7f119006db40 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 1 -- 172.21.15.105:0/1498069109 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f118c0a9300 con 0x7f119006db40 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 5 monclient: authenticate success, global_id 74173 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f1197ac1640 4 set_mon_vals no callback set 2024-09-17T16:39:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.708+0000 7f119485b640 10 monclient: dump: 2024-09-17T16:39:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: shutdown 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 msgr2=0x7f1190139a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f118c0098d0 tx=0x7f118c01c4d0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/1498069109 shutdown_connections 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f119006df20 0x7f1190139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006db40 0x7f1190139a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- 172.21.15.105:0/1498069109 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f119006d220 0x7f119006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/1498069109 >> 172.21.15.105:0/1498069109 conn(0x7f11900a69c0 msgr2=0x7f11900a6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/1498069109 shutdown_connections 2024-09-17T16:39:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/1498069109 wait complete. 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: build_initial_monmap 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: monmap: 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:37.753 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:37.755 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:37.755 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 librados: starting msgr at 2024-09-17T16:39:37.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 librados: starting objecter 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 Processor -- start 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- start start 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 librados: setting wanted keys 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 librados: calling monclient init 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: init 2024-09-17T16:39:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _add_conns ranks=[4,0,1,3,8,6,5,2,7] 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-c con 0x7f119006d220 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f119006db40 0x7f11901390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-a con 0x7f119006db40 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f119006df20 0x7f11901395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): picked mon.noname-f con 0x7f119006df20 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11901434e0 con 0x7f119006db40 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f119013a900 con 0x7f119006df20 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1190143660 con 0x7f119006d220 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient(hunting): authenticate will time out at 1660.982067s 2024-09-17T16:39:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:40314/0 (socket says 172.21.15.105:40314) 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 -- 172.21.15.105:0/701727735 learned_addr learned my addr 172.21.15.105:0/701727735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): get_auth_request con 0x7f119006d220 auth_method 0 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f119006db40 0x7f11901390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): get_auth_request con 0x7f119006db40 auth_method 0 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119505c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient(hunting): handle_auth_done global_id 74182 payload 824 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 -- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f119006df20 msgr2=0x7f11901395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f119006df20 0x7f11901395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 -- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f119006db40 msgr2=0x7f11901390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f119006db40 0x7f11901390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 monclient: found mon.noname-c 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 -- 172.21.15.105:0/701727735 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119013e200 con 0x7f119006d220 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f119585d640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9b0 tx=0x7f118400ee70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 client.?.objecter ms_handle_connect 0x7f119006d220 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1184017070 con 0x7f119006d220 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: dump: 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.762 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 5 monclient: authenticate success, global_id 74182 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118400cdf0 con 0x7f119006d220 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _renew_subs 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119013e8c0 con 0x7f119006d220 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f11840138e0 con 0x7f119006d220 2024-09-17T16:39:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1183fff640 4 set_mon_vals no callback set 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 client.74182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 10 monclient: dump: 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _renew_subs 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f11900a3ef0 con 0x7f119006d220 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 librados: init done 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f118401a070 con 0x7f119006d220 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.712+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119006d600 con 0x7f119006d220 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 0x7f1174040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(447..447 src has 1..447) ==== 6217+0+0 (secure 0 0 0) 0x7f118400f040 con 0x7f119006d220 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 10 client.74182.objecter ms_dispatch 0x7f1190000ba0 osd_map(447..447 src has 1..447) 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 3 client.74182.objecter handle_osd_map got epochs [447,447] > 0 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 3 client.74182.objecter handle_osd_map decoding full epoch 447 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 20 client.74182.objecter dump_active .. 0 homeless 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f119505c640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 0x7f1174040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f119505c640 10 monclient: get_auth_request con 0x7f117403e550 auth_method 0 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f11827fc640 10 client.74182.objecter ms_handle_connect 0x7f117403e550 2024-09-17T16:39:37.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.716+0000 7f119505c640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 0x7f1174040a10 secure :-1 s=READY pgs=3558 cs=0 l=1 rev1=1 crypto rx=0x7f118c0ad000 tx=0x7f118c0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:37.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.720+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1184021070 con 0x7f119006d220 2024-09-17T16:39:37.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.720+0000 7f11827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:37.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.720+0000 7f11827fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:37.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.972+0000 7f1197ac1640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:39:37.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.972+0000 7f1197ac1640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:39:37.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.972+0000 7f1197ac1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:37.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.976+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f11900a0810 con 0x7f119006d220 2024-09-17T16:39:37.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.976+0000 7f11827fc640 1 -- 172.21.15.105:0/701727735 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f1184011350 con 0x7f119006d220 2024-09-17T16:39:37.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.976+0000 7f11827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:39:37.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.976+0000 7f11827fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:37.989 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:37.989 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=11,kv=10,logm=62} crc {auth=3563383066,config=692619927,health=2835163478,kv=2863082231,logm=2956682942}) 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=2068503077,config=692619927,health=3122397229,kv=1277811279,logm=1158830201}) 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:37.990 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:37.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 10 librados: watch_flush enter 2024-09-17T16:39:37.991 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=12,kv=7,logm=64} crc {auth=3563383066,config=692619927,health=1639843326,kv=1872789148,logm=1034523374}) 2024-09-17T16:39:37.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 10 librados: watch_flush exit 2024-09-17T16:39:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 20 client.74182.objecter shutdown clearing up homeless session... 2024-09-17T16:39:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 10 client.74182.objecter successfully canceled tick 2024-09-17T16:39:37.992 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:37.992 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 msgr2=0x7f1174040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 0x7f1174040a10 secure :-1 s=READY pgs=3558 cs=0 l=1 rev1=1 crypto rx=0x7f118c0ad000 tx=0x7f118c0a6000 comp rx=0 tx=0).stop 2024-09-17T16:39:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 10 monclient: shutdown 2024-09-17T16:39:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 msgr2=0x7f1190138b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:37.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f118400e9b0 tx=0x7f118400ee70 comp rx=0 tx=0).stop 2024-09-17T16:39:37.994 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=11,kv=10,logm=62} crc {auth=3563383066,config=692619927,health=801729837,kv=2863082231,logm=416243480}) 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 shutdown_connections 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f117403e550 0x7f1174040a10 unknown :-1 s=CLOSED pgs=3558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f119006df20 0x7f11901395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f119006db40 0x7f11901390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 --2- 172.21.15.105:0/701727735 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f119006d220 0x7f1190138b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 >> 172.21.15.105:0/701727735 conn(0x7f11900a69c0 msgr2=0x7f119009fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:37.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 shutdown_connections 2024-09-17T16:39:37.996 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:37.996 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:37.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 -- 172.21.15.105:0/701727735 wait complete. 2024-09-17T16:39:37.997 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=3557686605,config=692619927,health=3191320229,kv=2863082231,logm=3931957684}) 2024-09-17T16:39:37.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f1197ac1640 1 librados: shutdown 2024-09-17T16:39:38.004 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=13,config=2,health=12,kv=10,logm=63} crc {auth=3969757150,config=692619927,health=1639843326,kv=2745955052,logm=2339421877}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=15,config=2,health=11,kv=10,logm=62} crc {auth=3563383066,config=692619927,health=2287280854,kv=3818608592,logm=3779380454}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=12,config=2,health=12,kv=9,logm=65} crc {auth=96369170,config=692619927,health=1639843326,kv=3449169938,logm=404524707}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.980+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=14,config=2,health=12,kv=9,logm=63} crc {auth=1365059341,config=692619927,health=1639843326,kv=2653694516,logm=1497875123}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186982672}) 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=609617013}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3739221404}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.984+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2558380243}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1825548965}) 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.006 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4259570028}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1240550629}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847091669}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.988+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1433333877}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539927116}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2186157810}) 2024-09-17T16:39:38.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=827827926}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3374145221}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=764542009}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1428483187}) 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.009 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=836726730}) 2024-09-17T16:39:38.009 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.009 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101422358}) 2024-09-17T16:39:38.009 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519297693}) 2024-09-17T16:39:38.011 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.011 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.011 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1490226127}) 2024-09-17T16:39:38.011 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=837828299}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2879000169}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:37.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1857703250}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2985108894}) 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3392416539}) 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=266259918}) 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527251566}) 2024-09-17T16:39:38.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=531548075}) 2024-09-17T16:39:38.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:39:38.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622007488}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2131751782}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4121876644}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3339677436}) 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.021 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1556919962}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855193331}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=991380626}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=109266098}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3388099055}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=747193161}) 2024-09-17T16:39:38.022 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3088389224}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=381379918}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1657033996}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.004+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=468446942}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.023 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3627542246}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2438402058}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2806855157}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2501137945}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1117640468}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=744216419}) 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710181431}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4006625007}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2641958967}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1811885818}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4138297041}) 2024-09-17T16:39:38.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889313687}) 2024-09-17T16:39:38.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2557615370}) 2024-09-17T16:39:38.033 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677685216}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3017940289}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4154169602}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1507167292}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1465192873}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3716652175}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3488860983}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=464132637}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2968739826}) 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=925492019}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1295180783}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2440987191}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3588480994}) 2024-09-17T16:39:38.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1754660141}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1484328470}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2949261605}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2303835366}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1561877040}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985127924}) 2024-09-17T16:39:38.037 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1217683893}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2624421344}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2969908493}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=10369722}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.038 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1578592589}) 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2672741037}) 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802963494}) 2024-09-17T16:39:38.039 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208688478}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3895961768}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1191486903}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3389136795}) 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2298493692}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3749905166}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3250768060}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3171011176}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1615842181}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1166884124}) 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330913375}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=282306807}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1549462217}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2891170474}) 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1239476985}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2953931690}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1200729593}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305859003}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1336906812}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4069911505}) 2024-09-17T16:39:38.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1914611720}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3109301923}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1275111473}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2171483860}) 2024-09-17T16:39:38.054 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218598205}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2456148907}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740247785}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2830274558}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287898841}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3767969173}) 2024-09-17T16:39:38.055 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2743244450}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3331759133}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3629739115}) 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.056 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776793039}) 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2823366121}) 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2955612258}) 2024-09-17T16:39:38.057 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=815221012}) 2024-09-17T16:39:38.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3138486677}) 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1229204137}) 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2014333481}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2335789470}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3587460772}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2298058046}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1210104623}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1197595360}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1117117838}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3020224057}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1360006707}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3214767732}) 2024-09-17T16:39:38.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3885080393}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3299713725}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=159507516}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2513294060}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402573607}) 2024-09-17T16:39:38.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6438401}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2977175068}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043695219}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1424423227}) 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2932033427}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2477622682}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3444609479}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=623445467}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029068881}) 2024-09-17T16:39:38.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1271754204}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=67574064}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986562525}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=545453596}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2862469083}) 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.074 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1010489180}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2856704548}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3513925589}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713445705}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3048335159}) 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.075 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891708483}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2479588185}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3088016312}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1880995495}) 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.076 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3249927542}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115551305}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=88453146}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3079963159}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1868169580}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.077 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3414784804}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199899373}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395350021}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1134312971}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=10764267}) 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.078 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1235166516}) 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2708401434}) 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097642874}) 2024-09-17T16:39:38.079 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3964860661}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1038808456}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1028277005}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1409328843}) 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.097 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2964320280}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2729280260}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482908102}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=206017608}) 2024-09-17T16:39:38.098 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2365009633}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3701452226}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2011889685}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4159918877}) 2024-09-17T16:39:38.099 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3678356040}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=572744433}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3702116147}) 2024-09-17T16:39:38.100 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=784296991}) 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3938721719}) 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564468826}) 2024-09-17T16:39:38.101 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1849503320}) 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1743769814}) 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1129761299}) 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.102 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3916054178}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3780135202}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1908463745}) 2024-09-17T16:39:38.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=393738330}) 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=895253173}) 2024-09-17T16:39:38.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366955291}) 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1520628360}) 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1698699850}) 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=602391473}) 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3371302596}) 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3824990088}) 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1918739384}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1177863709}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1734245306}) 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1461479522}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463734785}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=723753009}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=180694344}) 2024-09-17T16:39:38.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4110313848}) 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3290023281}) 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4059454606}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=431775546}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3041034491}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322339732}) 2024-09-17T16:39:38.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=523342609}) 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=917144164}) 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2591375723}) 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4012272645}) 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1148137029}) 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=150535009}) 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2173322163}) 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=512875798}) 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1640623263}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=78054427}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=746533292}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2168793570}) 2024-09-17T16:39:38.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3884089394}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3181209619}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4036909486}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2733350726}) 2024-09-17T16:39:38.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4200269321}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=580633956}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=672918059}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077928671}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=858154131}) 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2100060947}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3485351470}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1097752177}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=966355499}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4104676789}) 2024-09-17T16:39:38.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1269203254}) 2024-09-17T16:39:38.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3076451135}) 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3224954843}) 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2721713125}) 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.147 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1000688814}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=270975367}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3464697570}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2263490708}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.148 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3282724109}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765943562}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1382116784}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3870412389}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1010597518}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=598234026}) 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.149 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2912465523}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3944302135}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3565899680}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=788429819}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404208049}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4152776526}) 2024-09-17T16:39:38.150 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3836136649}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1794263187}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2550635352}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=656993915}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=964336626}) 2024-09-17T16:39:38.151 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2795778416}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1910417970}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3180315241}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.096+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2780210590}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3246051553}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.152 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2806660806}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=326588125}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1987621363}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2850566903}) 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.153 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361698313}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1322880525}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2912509242}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1665541566}) 2024-09-17T16:39:38.154 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3657992371}) 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3728507234}) 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.155 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=88737129}) 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2280054588}) 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.156 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1468393345}) 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943180206}) 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.157 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1691418924}) 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2496151496}) 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2260377991}) 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.158 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212329153}) 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1117650377}) 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.159 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3056394965}) 2024-09-17T16:39:38.160 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.160 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.160 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3947318577}) 2024-09-17T16:39:38.160 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.160 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=254743641}) 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2027774434}) 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.161 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2261342514}) 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3249637880}) 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4193198486}) 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.162 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=325385638}) 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=969949647}) 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.163 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1283982166}) 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2495682802}) 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.164 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.165 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2974110768}) 2024-09-17T16:39:38.165 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.165 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.165 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865008086}) 2024-09-17T16:39:38.165 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1272044257}) 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2775553091}) 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3171218598}) 2024-09-17T16:39:38.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=534338297}) 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1783298443}) 2024-09-17T16:39:38.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=745967810}) 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=94750252}) 2024-09-17T16:39:38.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2169453206}) 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630341906}) 2024-09-17T16:39:38.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1775076187}) 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123878679}) 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2981680217}) 2024-09-17T16:39:38.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4203658401}) 2024-09-17T16:39:38.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2433919534}) 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3817805038}) 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1253265762}) 2024-09-17T16:39:38.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1798083368}) 2024-09-17T16:39:38.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3187381197}) 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=66683339}) 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2012987883}) 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1755500697}) 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2286829234}) 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3823176111}) 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=90455151}) 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1363973391}) 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1212946068}) 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4162166755}) 2024-09-17T16:39:38.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2728557479}) 2024-09-17T16:39:38.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4009700504}) 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3019812568}) 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151760777}) 2024-09-17T16:39:38.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3695444294}) 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3584282073}) 2024-09-17T16:39:38.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1114606259}) 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1734851137}) 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2439931925}) 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=989712142}) 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090550667}) 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2606419578}) 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=701593757}) 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1430452018}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=572723518}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=138572041}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2647881564}) 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=472001244}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1588164431}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4103098033}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2959794336}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014127663}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=980081487}) 2024-09-17T16:39:38.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=795188158}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=94130102}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1623680862}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1250785918}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1672154135}) 2024-09-17T16:39:38.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2771588123}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1139829806}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2711172586}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=534301059}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=255308623}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1356485778}) 2024-09-17T16:39:38.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3813575075}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258554951}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=373389684}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285890238}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2618494955}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3181974725}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2981209738}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1462032716}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=415850333}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3993752960}) 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3233851272}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3889226649}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420260539}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1663550306}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1089054165}) 2024-09-17T16:39:38.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1856717982}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3361401534}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1476533414}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1358447491}) 2024-09-17T16:39:38.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=921907124}) 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=546380245}) 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344098128}) 2024-09-17T16:39:38.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028349179}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=589067116}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1046632774}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2900296711}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1277718408}) 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1823063176}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3603456760}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1924281000}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=624836971}) 2024-09-17T16:39:38.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.136+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1128410259}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087807117}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1547741076}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.304 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3935926462}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1565639120}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4284236662}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390392260}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4034051665}) 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.305 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803141631}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1297906970}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2289519000}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2425578306}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=985423097}) 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.306 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3874710115}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1385959881}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=122494483}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2226339450}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890712347}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=206005991}) 2024-09-17T16:39:38.307 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1207582724}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1035085290}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2509715681}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1840125058}) 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.308 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3346389888}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342952257}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3482507846}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3298923575}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.144+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1899500285}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.309 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578346678}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3848911337}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1897545815}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2726914493}) 2024-09-17T16:39:38.310 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1089047511}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1422226498}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1929151758}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226767972}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1468749029}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.311 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2611113104}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1347529124}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3260377585}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=792799745}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.312 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3964421952}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409728203}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1381533792}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2492544085}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4141541483}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221310122}) 2024-09-17T16:39:38.313 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1656126940}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594576272}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2954526949}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3811189821}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.314 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3553533324}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4161839914}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3390748062}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.152+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=487763280}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=174537414}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2031221094}) 2024-09-17T16:39:38.315 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=716897370}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3586646015}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3348827350}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4164480041}) 2024-09-17T16:39:38.316 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1607604246}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3943067982}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3804142615}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=965803341}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.317 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1058382040}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=718821515}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1628458198}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=271380236}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1783476559}) 2024-09-17T16:39:38.318 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2574127166}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3937114767}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.156+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1654929549}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2858963039}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3409136439}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.319 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=116955841}) 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4082387191}) 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2827265185}) 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.320 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3452263299}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1867614365}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678437340}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1037002428}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4242009251}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.321 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3550962110}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721267795}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=978151391}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=886554304}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=478010721}) 2024-09-17T16:39:38.322 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4053193087}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4272162763}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.160+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768305372}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1705883992}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=730246668}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.323 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=897647016}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3660904149}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4176442318}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.324 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2547347488}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3660734233}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=557872893}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1229414454}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.164+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3583482691}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.325 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3608115227}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1662652208}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2240166736}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3755480703}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3950367760}) 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.326 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2803709596}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231157660}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2808393131}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4117566419}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=842604996}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1301513472}) 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.327 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=444240405}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3478389716}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1062147965}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=295745949}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093371024}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.168+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1085593977}) 2024-09-17T16:39:38.328 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=951770546}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1228456052}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=646512905}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=401951957}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.329 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3674236249}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3856325689}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2859297498}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4182701835}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933196612}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1926181530}) 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.330 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2786071523}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1446366959}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1862220463}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.331 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1377854672}) 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1509776722}) 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659713843}) 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.332 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615716637}) 2024-09-17T16:39:38.333 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.172+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2187678853}) 2024-09-17T16:39:38.333 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.333 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.333 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3959405909}) 2024-09-17T16:39:38.333 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1598456160}) 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1605316248}) 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.334 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1031047243}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=31240983}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2636628815}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3290002115}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2490189602}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2328397541}) 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.335 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436009559}) 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=286332175}) 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.336 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.337 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28865782}) 2024-09-17T16:39:38.337 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.337 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.337 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3924055770}) 2024-09-17T16:39:38.337 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2488120912}) 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2848827240}) 2024-09-17T16:39:38.338 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2297436078}) 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440573701}) 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.176+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1971620249}) 2024-09-17T16:39:38.339 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360677837}) 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=744399346}) 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.340 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=628082449}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=732423973}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1087512993}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3301415374}) 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.341 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=24840073}) 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1159285508}) 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2952587257}) 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.342 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=299095839}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3337548183}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1848561221}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.180+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2086530883}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3431370112}) 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.343 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3554993844}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2323404635}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216329237}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2336031349}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766483092}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=720796774}) 2024-09-17T16:39:38.344 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2254677240}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1385439550}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=176648646}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2420734119}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.345 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1184520754}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3081071183}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.184+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301050416}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309465066}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523219832}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3058915681}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.346 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808199578}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2328129745}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=741863340}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2826556499}) 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.347 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1595071350}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4223678271}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2426534304}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035145353}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2427538675}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4267002201}) 2024-09-17T16:39:38.348 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=409727100}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2056468199}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3661099414}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151000793}) 2024-09-17T16:39:38.349 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.188+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3944470895}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376765659}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2689643803}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1849716512}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1305422466}) 2024-09-17T16:39:38.350 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3056287933}) 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1239083858}) 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.351 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1888321655}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3980290027}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3780305836}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4143492483}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1635843406}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.352 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3885160834}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=786965671}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2048079973}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3867181968}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.353 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1748490630}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1583437900}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1843669187}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3163578187}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.354 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2418906137}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3668161299}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3703869223}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2320888659}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1715245766}) 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.355 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=527571103}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401799058}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2517744568}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3389231567}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.356 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1043447062}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3951484029}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=60240271}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1664416315}) 2024-09-17T16:39:38.357 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4258650035}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3593403796}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4281354238}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1636303125}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3061711810}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.358 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4283934399}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=493207428}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3275033592}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.359 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1267324488}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1787769438}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3508796224}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641839729}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2108992575}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680990626}) 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.360 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=936942329}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2751074462}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199079190}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1919083444}) 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.361 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3099626228}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1085193145}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891340160}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2280785532}) 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.362 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=780854668}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=859386840}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3695737415}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.363 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3802129837}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=268638323}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2747842668}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1634670978}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3740808089}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2257477069}) 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.364 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1645307009}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=133760989}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=731103876}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1367595316}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2642554048}) 2024-09-17T16:39:38.365 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1907006282}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2006723081}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268537587}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=945618812}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1232271216}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1421360002}) 2024-09-17T16:39:38.366 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3000574439}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=770588525}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2871254446}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3589739315}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.367 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2323886537}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1868072554}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=325290417}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879912986}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1043843163}) 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.368 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1827048267}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=808954474}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3232711452}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2686897968}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1537579218}) 2024-09-17T16:39:38.369 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2470468786}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066832834}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3175797532}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=738418820}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=857252706}) 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.370 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2404716229}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3433355929}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3211999300}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3337049250}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1100576178}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=977958783}) 2024-09-17T16:39:38.371 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2525658551}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1484741729}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3521341338}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3898518198}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.372 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1791187313}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3072725170}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1417802543}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980770892}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1544835760}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2811406755}) 2024-09-17T16:39:38.373 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2675985567}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196584669}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1352748147}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3542679938}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.374 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430414091}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3448179046}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803612580}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290186085}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211873046}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.375 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=454673869}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1829037196}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1997693772}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2682108220}) 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.376 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1140015610}) 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3576068053}) 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.377 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=782672378}) 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=634655325}) 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4288723277}) 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.378 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=583454873}) 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2374578782}) 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.379 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2594410712}) 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1638645913}) 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.380 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=110761875}) 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3985265363}) 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=253250510}) 2024-09-17T16:39:38.381 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1823227821}) 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1961672788}) 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.382 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3943054311}) 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2365105762}) 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.383 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3934729385}) 2024-09-17T16:39:38.384 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.384 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.384 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1810975144}) 2024-09-17T16:39:38.384 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.384 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2299416957}) 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3069350819}) 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.385 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=841959292}) 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986972035}) 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2285137675}) 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.386 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2944540866}) 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1362346022}) 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.387 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.388 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1068578130}) 2024-09-17T16:39:38.388 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.388 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.388 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1755017218}) 2024-09-17T16:39:38.388 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2904882667}) 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1641728786}) 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.389 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3647502773}) 2024-09-17T16:39:38.392 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3664881616}) 2024-09-17T16:39:38.392 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.392 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2424684126}) 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2609198789}) 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.393 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=69477830}) 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3494406968}) 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.394 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=961687387}) 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2688001307}) 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.395 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.396 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=518024538}) 2024-09-17T16:39:38.396 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.396 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=408431820}) 2024-09-17T16:39:38.396 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3963217125}) 2024-09-17T16:39:38.396 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=67492910}) 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129981441}) 2024-09-17T16:39:38.397 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.398 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.398 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=745626738}) 2024-09-17T16:39:38.398 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.398 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.398 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2014074275}) 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2409422350}) 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.399 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3788399639}) 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=892743535}) 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814569815}) 2024-09-17T16:39:38.400 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4007432398}) 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559702741}) 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.401 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=266792617}) 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1208483795}) 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.402 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1995065944}) 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2956301506}) 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.403 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651272496}) 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3871283940}) 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=702933624}) 2024-09-17T16:39:38.404 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=842780731}) 2024-09-17T16:39:38.405 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.405 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.405 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3893805108}) 2024-09-17T16:39:38.405 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.405 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3196377324}) 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4011518021}) 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.406 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2629095600}) 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4074581187}) 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.407 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2252878851}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3813763969}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1882640753}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=252563096}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.408 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=53952472}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4289576310}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3437632746}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3021203386}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3688126828}) 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.409 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=516741088}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=336203514}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3632634898}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2045736854}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=2,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=3758851135,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311848,mgrstat=171338351,mon_config_key=1589589211,monmap=835142563,nvmeofgw=1596038039,osd_metadata=2241924772,osd_pg_creating=388244172,osd_snap=3205370891}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.410 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=2370506962,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=392629442,mon_config_key=3263060063,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=2669840769}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15} crc {logm=1217044420,mdsmap=1248375647,mgr=3567252287,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=4155634237}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=36,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=19} crc {logm=1560301659,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=4257234632,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=1331419060}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=40,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=17} crc {logm=2370506962,mdsmap=1248375647,mgr=702605724,mgr_metadata=3792311847,mgrstat=3004398471,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=388244172,osd_snap=3526050911}) 2024-09-17T16:39:38.411 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=40,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=18} crc {logm=1591852063,mdsmap=3140514009,mgr=2639282762,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2022856068,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=2719151891,osd_pg_creating=388244172,osd_snap=2343178586}) 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=39,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=3451131008,mdsmap=952268285,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=346208105,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=839368740}) 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16} crc {logm=2380618600,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=2112433942,monmap=3140514009,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=388244172,osd_snap=464223838}) 2024-09-17T16:39:38.412 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=13} crc {logm=1590046271,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=556519678,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=49183994,osd_pg_creating=388244172,osd_snap=960934006}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=1188462597,osdmap=2711723508}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=16,osdmap=84} crc {osd_snap=2005985652,osdmap=3709170235}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=1535781176,osdmap=3669611074}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.413 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=3389791957,osdmap=3729911145}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=3949907280,osdmap=4022503928}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=1716665540,osdmap=1935968215}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=3949907280,osdmap=3760131492}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=18,osdmap=82} crc {osd_snap=3853720884,osdmap=2330397870}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=19,osdmap=81} crc {osd_snap=3949907280,osdmap=1824985237}) 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.414 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2178617447}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3901650839}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2522326449}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2426875812}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4259778882}) 2024-09-17T16:39:38.415 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1861070754}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3779655846}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2614994039}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3121620168}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4188538749}) 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.416 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=942543870}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4022560249}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=549551044}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1756248497}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2286306145}) 2024-09-17T16:39:38.417 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1848421357}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4069608902}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3033463129}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2010321402}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=14736622}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.418 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4207737875}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=702521773}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3756487761}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2042210818}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=718650874}) 2024-09-17T16:39:38.419 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3937288275}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=398032031}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=544934275}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1968413193}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3887924945}) 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.420 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1969781908}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2283879744}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4274408122}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=247558149}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=118942108}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=90413038}) 2024-09-17T16:39:38.421 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3574743012}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1867612039}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1166078756}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=632895822}) 2024-09-17T16:39:38.422 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2884833269}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=747711836}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3506213929}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3610357392}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=447170390}) 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.423 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=66} crc {osdmap=3852643743}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=69} crc {osdmap=3827948039}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=67} crc {osdmap=3498806967}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=71} crc {osdmap=1557303644}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=66} crc {osdmap=3832955175}) 2024-09-17T16:39:38.424 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=69} crc {osdmap=2626859513}) 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=69} crc {osdmap=3993426720}) 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=532765364}) 2024-09-17T16:39:38.425 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:39:38.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=70} crc {osdmap=3162022618}) 2024-09-17T16:39:39.015 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:39:39.015 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: get_monmap_and_config 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: monmap: 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:39.157021+0000 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:39.157021+0000 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:39.162 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 Processor -- start 2024-09-17T16:39:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 -- start start 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: init 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient: _add_conns ranks=[1,9,5,4,2,7,0,8,10,6,3] 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa7f40a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f40a4910 0x7fa7f413eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-a con 0x7fa7f40a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f40a73d0 0x7fa7f4141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-h con 0x7fa7f40a73d0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f406ebc0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f40a3fc0 con 0x7fa7f40a4910 2024-09-17T16:39:39.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa7f40a63c0 con 0x7fa7f40a73d0 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f898f640 10 monclient(hunting): authenticate will time out at 1662.422617s 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f40a4910 0x7fa7f413eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f40a4910 0x7fa7f413eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39014/0 (socket says 172.21.15.105:39014) 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/1083592400 learned_addr learned my addr 172.21.15.105:0/1083592400 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): get_auth_request con 0x7fa7f40a4910 auth_method 0 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f2d9d640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f40a73d0 0x7fa7f4141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_done global_id 35973 payload 274 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/1083592400 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f40a73d0 msgr2=0x7fa7f4141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f40a73d0 0x7fa7f4141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/1083592400 >> v1:172.21.15.160:6789/0 conn(0x7fa7f40a68e0 legacy=0x7fa7f40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 monclient: found mon.noname-a 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.152+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/1083592400 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f40a3690 con 0x7fa7f40a4910 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a4910 0x7fa7f413eaf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009870 tx=0x7fa7e801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689e98bc66024f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 1 -- 172.21.15.105:0/1083592400 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b0070 con 0x7fa7f40a4910 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: dump: 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.169 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 1 -- 172.21.15.105:0/1083592400 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e80aa810 con 0x7fa7f40a4910 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 5 monclient: authenticate success, global_id 35973 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 1 -- 172.21.15.105:0/1083592400 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80a9300 con 0x7fa7f40a4910 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 4 set_mon_vals no callback set 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f159a640 10 monclient: dump: 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.171 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: shutdown 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- 172.21.15.105:0/1083592400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a4910 msgr2=0x7fa7f413eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a4910 0x7fa7f413eaf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009870 tx=0x7fa7e801c3f0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- 172.21.15.105:0/1083592400 shutdown_connections 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7f40a73d0 0x7fa7f4141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- 172.21.15.105:0/1083592400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a4910 0x7fa7f413eaf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- 172.21.15.105:0/1083592400 >> 172.21.15.105:0/1083592400 conn(0x7fa7f4068ba0 msgr2=0x7fa7f4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- 172.21.15.105:0/1083592400 shutdown_connections 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- 172.21.15.105:0/1083592400 wait complete. 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: monmap: 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.173 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 librados: starting msgr at 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 librados: starting objecter 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 Processor -- start 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- start start 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 librados: setting wanted keys 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 librados: calling monclient init 2024-09-17T16:39:39.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: init 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient: _add_conns ranks=[3,0,2,5,1,7,4,6,8] 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 0x7fa7f4128ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-g con 0x7fa7f40a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-a con 0x7fa7f40a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f40a73d0 0x7fa7f41d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): picked mon.noname-b con 0x7fa7f40a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4143f20 con 0x7fa7f40a68e0 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f4143da0 con 0x7fa7f40a73d0 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f41440a0 con 0x7fa7f40a4910 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f898f640 10 monclient(hunting): authenticate will time out at 1662.427568s 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f40a73d0 0x7fa7f41d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39018/0 (socket says 172.21.15.105:39018) 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/4287359603 learned_addr learned my addr 172.21.15.105:0/4287359603 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): get_auth_request con 0x7fa7f40a68e0 auth_method 0 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): get_auth_request con 0x7fa7f40a73d0 auth_method 0 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f259c640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 0x7fa7f4128ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient(hunting): handle_auth_done global_id 35982 payload 824 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 msgr2=0x7fa7f4128ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 0x7fa7f4128ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f40a73d0 msgr2=0x7fa7f41d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f40a73d0 0x7fa7f41d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 1 monclient: found mon.noname-a 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.156+0000 7fa7f1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f1d9b640 1 -- 172.21.15.105:0/4287359603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f4129fe0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f259c640 10 monclient: get_auth_request con 0x7fa7f40a4910 auth_method 0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f259c640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 0x7fa7f4128ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f1d9b640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ae040 tx=0x7fa7e8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 client.?.objecter ms_handle_connect 0x7fa7f40a68e0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b9070 con 0x7fa7f40a68e0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:39.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: dump: 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.183 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e80044b0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 5 monclient: authenticate success, global_id 35982 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _renew_subs 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f406dad0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7e80b0040 con 0x7fa7f40a68e0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:39.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 monclient: dump: 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.185 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 client.35982.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _renew_subs 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f0d99640 4 set_mon_vals no callback set 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f40a2530 con 0x7fa7f40a68e0 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 1 librados: init done 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa7e8004040 con 0x7fa7f40a68e0 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f40a6cc0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 0x7fa7d4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7fa7e80e94c0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 client.35982.objecter ms_dispatch 0x7fa7f4000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 3 client.35982.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 3 client.35982.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 20 client.35982.objecter dump_active .. 0 homeless 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f259c640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 0x7fa7d4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f259c640 10 monclient: get_auth_request con 0x7fa7d403e550 auth_method 0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7f259c640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 0x7fa7d4040a10 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4003e40 tx=0x7fa7e4009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.160+0000 7fa7e2ffd640 10 client.35982.objecter ms_handle_connect 0x7fa7d403e550 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.164+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7e8016130 con 0x7fa7f40a68e0 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.164+0000 7fa7e2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.164+0000 7fa7e2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:39.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7f898f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:39:39.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7f898f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:39.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7f898f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:39.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa7f40008e0 con 0x7fa7f40a68e0 2024-09-17T16:39:39.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7e2ffd640 1 -- 172.21.15.105:0/4287359603 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7fa7e80aae10 con 0x7fa7f40a68e0 2024-09-17T16:39:39.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7e2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:39.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.400+0000 7fa7e2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:39.409 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:39:39.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 10 librados: watch_flush enter 2024-09-17T16:39:39.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 10 librados: watch_flush exit 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 20 client.35982.objecter shutdown clearing up homeless session... 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 10 client.35982.objecter successfully canceled tick 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 msgr2=0x7fa7d4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 0x7fa7d4040a10 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4003e40 tx=0x7fa7e4009040 comp rx=0 tx=0).stop 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 10 monclient: shutdown 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 msgr2=0x7fa7f41d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ae040 tx=0x7fa7e8019190 comp rx=0 tx=0).stop 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 shutdown_connections 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa7d403e550 0x7fa7d4040a10 unknown :-1 s=CLOSED pgs=3559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7f40a73d0 0x7fa7f41d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f40a68e0 0x7fa7f41d14e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 --2- 172.21.15.105:0/4287359603 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7f40a4910 0x7fa7f4128ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 >> 172.21.15.105:0/4287359603 conn(0x7fa7f4068ba0 msgr2=0x7fa7f409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 shutdown_connections 2024-09-17T16:39:39.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 -- 172.21.15.105:0/4287359603 wait complete. 2024-09-17T16:39:39.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.404+0000 7fa7f898f640 1 librados: shutdown 2024-09-17T16:39:39.427 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:39:39.427 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:39:39.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:39:39.427 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.536+0000 7f9608b4a640 10 monclient: get_monmap_and_config 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.536+0000 7f9608b4a640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.536+0000 7f9608b4a640 10 monclient: monmap: 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:39.542676+0000 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:39.542676+0000 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.546 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:39.547 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 Processor -- start 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 -- start start 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient: init 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient: _add_conns ranks=[10,2,5,6,9,0,1,4,7,8,3] 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 0x7f96040a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-f con 0x7f960409f670 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-b con 0x7f96040696c0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9604069ae0 0x7f96040a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-h con 0x7f9604069ae0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f96040a01d0 con 0x7f960409f670 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f96040a20a0 con 0x7f96040696c0 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f960409fef0 con 0x7f9604069ae0 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9608b4a640 10 monclient(hunting): authenticate will time out at 1662.808898s 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58560/0 (socket says 172.21.15.105:58560) 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 -- 172.21.15.105:0/348320760 learned_addr learned my addr 172.21.15.105:0/348320760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): get_auth_request con 0x7f96040696c0 auth_method 0 2024-09-17T16:39:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 1 --2- 172.21.15.105:0/348320760 >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 0x7f96040a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9604069ae0 0x7f96040a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 10 monclient(hunting): get_auth_request con 0x7f960409f670 auth_method 0 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): get_auth_request con 0x7f9604069ae0 auth_method 0 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9602d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient(hunting): handle_auth_done global_id 45011 payload 274 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 -- 172.21.15.105:0/348320760 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9604069ae0 msgr2=0x7f96040a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9604069ae0 0x7f96040a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 -- 172.21.15.105:0/348320760 >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 msgr2=0x7f96040a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 --2- 172.21.15.105:0/348320760 >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 0x7f96040a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 monclient: found mon.noname-b 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 -- 172.21.15.105:0/348320760 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96040a4070 con 0x7f96040696c0 2024-09-17T16:39:39.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960259c640 1 --2- 172.21.15.105:0/348320760 >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 0x7f96040a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f9601d9b640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098a0 tx=0x7f95f801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa65bc7b50deabe server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960159a640 1 -- 172.21.15.105:0/348320760 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f80b0070 con 0x7f96040696c0 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.540+0000 7f960159a640 10 monclient: dump: 2024-09-17T16:39:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 1 -- 172.21.15.105:0/348320760 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f80aa810 con 0x7f96040696c0 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 5 monclient: authenticate success, global_id 45011 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 1 -- 172.21.15.105:0/348320760 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f80a9300 con 0x7f96040696c0 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960159a640 10 monclient: dump: 2024-09-17T16:39:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 4 set_mon_vals no callback set 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: shutdown 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/348320760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 msgr2=0x7f96040a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098a0 tx=0x7f95f801c400 comp rx=0 tx=0).stop 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/348320760 shutdown_connections 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9604069ae0 0x7f96040a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- 172.21.15.105:0/348320760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96040696c0 0x7f96040a2620 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- 172.21.15.105:0/348320760 >> v2:172.21.15.160:3300/0 conn(0x7f960409f670 0x7f96040a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/348320760 >> 172.21.15.105:0/348320760 conn(0x7f9604068ba0 msgr2=0x7f9604068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/348320760 shutdown_connections 2024-09-17T16:39:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/348320760 wait complete. 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: monmap: 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 librados: starting msgr at 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 librados: starting objecter 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 Processor -- start 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- start start 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 librados: setting wanted keys 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 librados: calling monclient init 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: init 2024-09-17T16:39:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _add_conns ranks=[5,2,1,4,3,8,6,0,7] 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f96040696c0 0x7f960413ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-h con 0x7f96040696c0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 0x7f960413b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-b con 0x7f9604069ae0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f960409f670 0x7f960413b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): picked mon.noname-f con 0x7f960409f670 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9604136f90 con 0x7f960409f670 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9604136e10 con 0x7f9604069ae0 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9604137110 con 0x7f96040696c0 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): authenticate will time out at 1662.814344s 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 0x7f960413b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 0x7f960413b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58564/0 (socket says 172.21.15.105:58564) 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 -- 172.21.15.105:0/3764407862 learned_addr learned my addr 172.21.15.105:0/3764407862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): get_auth_request con 0x7f9604069ae0 auth_method 0 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960259c640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f96040696c0 0x7f960413ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9602d9d640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f960409f670 0x7f960413b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960259c640 10 monclient(hunting): get_auth_request con 0x7f96040696c0 auth_method 0 2024-09-17T16:39:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f960259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient(hunting): handle_auth_done global_id 45020 payload 824 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f96040696c0 msgr2=0x7f960413ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f96040696c0 0x7f960413ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f960409f670 msgr2=0x7f960413b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f960409f670 0x7f960413b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 monclient: found mon.noname-b 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96041206c0 con 0x7f9604069ae0 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9602d9d640 10 monclient: get_auth_request con 0x7f960409f670 auth_method 0 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9602d9d640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f960409f670 0x7f960413b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9601d9b640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 0x7f960413b0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95f80ae040 tx=0x7f95f80191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 client.?.objecter ms_handle_connect 0x7f9604069ae0 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f80b9070 con 0x7f9604069ae0 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: dump: 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f80044b0 con 0x7f9604069ae0 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 5 monclient: authenticate success, global_id 45020 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _renew_subs 2024-09-17T16:39:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96040738a0 con 0x7f9604069ae0 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f80b0040 con 0x7f9604069ae0 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f2ffd640 10 monclient: dump: 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f95f3fff640 4 set_mon_vals no callback set 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 client.45020.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _renew_subs 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f960411f850 con 0x7f9604069ae0 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 librados: init done 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 0x7f96041378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.544+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9604135480 con 0x7f9604137500 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 0x7f96041378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient: get_auth_request con 0x7f9604137500 auth_method 0 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f95f80a4d10 con 0x7f9604069ae0 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 10 monclient(hunting): handle_auth_done global_id 35991 payload 824 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f960411e3c0 con 0x7f9604137500 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f95e403e580 0x7f95e4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f95f80e8da0 con 0x7f9604069ae0 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 client.45020.objecter ms_dispatch 0x7f9604000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 3 client.45020.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 3 client.45020.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f960259c640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 0x7f96041378e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004610 tx=0x7f95f4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 20 client.45020.objecter dump_active .. 0 homeless 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 client.45020.objecter ms_handle_connect 0x7f9604137500 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 client.45020.objecter resend_mon_ops 2024-09-17T16:39:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4014880 con 0x7f9604137500 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 monclient: dump: 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f9602d9d640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f95e403e580 0x7f95e4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f9602d9d640 10 monclient: get_auth_request con 0x7f95e403e580 auth_method 0 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95f4019070 con 0x7f9604137500 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 msgr2=0x7f96041378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 0x7f96041378e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004610 tx=0x7f95f4007700 comp rx=0 tx=0).stop 2024-09-17T16:39:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 msgr2=0x7f96041378e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137500 0x7f96041378e0 secure :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95f4004610 tx=0x7f95f4007700 comp rx=0 tx=0).stop 2024-09-17T16:39:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f95f2ffd640 10 client.45020.objecter ms_handle_connect 0x7f95e403e580 2024-09-17T16:39:39.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.548+0000 7f9602d9d640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f95e403e580 0x7f95e4040a40 secure :-1 s=READY pgs=3560 cs=0 l=1 rev1=1 crypto rx=0x7f9604073f90 tx=0x7f95fc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f9608b4a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f9608b4a640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 0x7f96041db940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f9608b4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9604051e80 con 0x7f9604137e20 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 0x7f96041db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient: get_auth_request con 0x7f9604137e20 auth_method 0 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 10 monclient(hunting): handle_auth_done global_id 36000 payload 824 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 1 -- 172.21.15.105:0/3764407862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f96040a1b60 con 0x7f9604137e20 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f960259c640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 0x7f96041db940 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f90 tx=0x7f95f4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 10 client.45020.objecter ms_handle_connect 0x7f9604137e20 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 10 client.45020.objecter resend_mon_ops 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4014d40 con 0x7f9604137e20 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.552+0000 7f95f2ffd640 10 monclient: dump: 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f95f40148d0 con 0x7f9604137e20 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 msgr2=0x7f96041db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 0x7f96041db940 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f90 tx=0x7f95f4004a50 comp rx=0 tx=0).stop 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 msgr2=0x7f96041db940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f95f2ffd640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9604137e20 0x7f96041db940 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000f90 tx=0x7f95f4004a50 comp rx=0 tx=0).stop 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:39.583 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 570356, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.584 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.585 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.586 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:39.587 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.588 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.589 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.590 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.597 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.599 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.601 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:39.603 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:39.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:39.606 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 10 librados: watch_flush enter 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 10 librados: watch_flush exit 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 20 client.45020.objecter shutdown clearing up homeless session... 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 10 client.45020.objecter successfully canceled tick 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f95e403e580 msgr2=0x7f95e4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f95e403e580 0x7f95e4040a40 secure :-1 s=READY pgs=3560 cs=0 l=1 rev1=1 crypto rx=0x7f9604073f90 tx=0x7f95fc008040 comp rx=0 tx=0).stop 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 10 monclient: shutdown 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 msgr2=0x7f960413b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 --2- 172.21.15.105:0/3764407862 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9604069ae0 0x7f960413b0c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95f80ae040 tx=0x7f95f80191a0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9602d9d640 1 -- 172.21.15.105:0/3764407862 reap_dead start 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9602d9d640 1 -- 172.21.15.105:0/3764407862 reap_dead start 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 shutdown_connections 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 >> 172.21.15.105:0/3764407862 conn(0x7f9604068ba0 msgr2=0x7f9604070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 shutdown_connections 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 -- 172.21.15.105:0/3764407862 wait complete. 2024-09-17T16:39:39.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.556+0000 7f9608b4a640 1 librados: shutdown 2024-09-17T16:39:39.609 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:39.609 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:39:39.610 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:39:39.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: get_monmap_and_config 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: monmap: 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:39.728585+0000 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:39.728585+0000 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.732 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:39.733 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 Processor -- start 2024-09-17T16:39:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 -- start start 2024-09-17T16:39:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: init 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient: _add_conns ranks=[3,6,7,0,5,4,2,1,10,8,9] 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b40a68e0 0x7f53b40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-g con 0x7f53b40a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-d con 0x7f53b40a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f53b41396a0 0x7f53b4139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-i con 0x7f53b41396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f53b40a3fc0 con 0x7f53b40a68e0 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f53b40a63c0 con 0x7f53b40a4910 2024-09-17T16:39:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f53b40a7740 con 0x7f53b41396a0 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b9f6b640 10 monclient(hunting): authenticate will time out at 1662.994167s 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:53056/0 (socket says 172.21.15.105:53056) 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 -- 172.21.15.105:0/1832744305 learned_addr learned my addr 172.21.15.105:0/1832744305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request con 0x7f53b40a4910 auth_method 0 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b3fff640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f53b41396a0 0x7f53b4139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b37fe640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b40a68e0 0x7f53b40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b3fff640 10 monclient(hunting): get_auth_request con 0x7f53b41396a0 auth_method 0 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b37fe640 10 monclient(hunting): get_auth_request con 0x7f53b40a68e0 auth_method 0 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_done global_id 24657 payload 274 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 -- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f53b41396a0 msgr2=0x7f53b4139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f53b41396a0 0x7f53b4139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 -- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b40a68e0 msgr2=0x7f53b40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b40a68e0 0x7f53b40a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 monclient: found mon.noname-d 2024-09-17T16:39:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.724+0000 7f53b2ffd640 1 -- 172.21.15.105:0/1832744305 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b406ec10 con 0x7f53b40a4910 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f53a80098a0 tx=0x7f53a801cea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9611a61e691e05cb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 1 -- 172.21.15.105:0/1832744305 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a80b0070 con 0x7f53b40a4910 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: dump: 2024-09-17T16:39:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 1 -- 172.21.15.105:0/1832744305 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a80aa810 con 0x7f53b40a4910 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 5 monclient: authenticate success, global_id 24657 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 4 set_mon_vals no callback set 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 1 -- 172.21.15.105:0/1832744305 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a80a9300 con 0x7f53b40a4910 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b27fc640 10 monclient: dump: 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: shutdown 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- 172.21.15.105:0/1832744305 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 msgr2=0x7f53b40a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f53a80098a0 tx=0x7f53a801cea0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- 172.21.15.105:0/1832744305 shutdown_connections 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f53b41396a0 0x7f53b4139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f53b40a4910 0x7f53b40a7200 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/1832744305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b40a68e0 0x7f53b40a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- 172.21.15.105:0/1832744305 >> 172.21.15.105:0/1832744305 conn(0x7f53b4068ba0 msgr2=0x7f53b4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- 172.21.15.105:0/1832744305 shutdown_connections 2024-09-17T16:39:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- 172.21.15.105:0/1832744305 wait complete. 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: monmap: 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 librados: starting msgr at 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 librados: starting objecter 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 Processor -- start 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- start start 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 librados: setting wanted keys 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 librados: calling monclient init 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: init 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient: _add_conns ranks=[2,8,0,1,4,5,3,7,6] 2024-09-17T16:39:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 0x7f53b41d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-b con 0x7f53b40a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f53b40a68e0 0x7f53b41d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-e con 0x7f53b40a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53b41396a0 0x7f53b413d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): picked mon.noname-a con 0x7f53b41396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4143320 con 0x7f53b41396a0 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f53b406e630 con 0x7f53b40a4910 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f53b41434a0 con 0x7f53b40a68e0 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.728+0000 7f53b9f6b640 10 monclient(hunting): authenticate will time out at 1662.999938s 2024-09-17T16:39:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 0x7f53b41d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f53b40a68e0 0x7f53b41d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 0x7f53b41d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58574/0 (socket says 172.21.15.105:58574) 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53b41396a0 0x7f53b413d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 -- 172.21.15.105:0/3750703492 learned_addr learned my addr 172.21.15.105:0/3750703492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): get_auth_request con 0x7f53b40a4910 auth_method 0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): get_auth_request con 0x7f53b41396a0 auth_method 0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request con 0x7f53b40a68e0 auth_method 0 2024-09-17T16:39:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient(hunting): handle_auth_done global_id 45029 payload 824 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f53b40a68e0 msgr2=0x7f53b41d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f53b40a68e0 0x7f53b41d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53b41396a0 msgr2=0x7f53b413d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53b41396a0 0x7f53b413d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 monclient: found mon.noname-b 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b413fc40 con 0x7f53b40a4910 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b37fe640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 0x7f53b41d1650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53a400d800 tx=0x7f53a400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.?.objecter ms_handle_connect 0x7f53b40a4910 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a4018020 con 0x7f53b40a4910 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: dump: 2024-09-17T16:39:39.755 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a4009b10 con 0x7f53b40a4910 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 5 monclient: authenticate success, global_id 45029 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a4009cc0 con 0x7f53b40a4910 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: dump: 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.758 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b1ffb640 4 set_mon_vals no callback set 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 monclient: _renew_subs 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b4123cf0 con 0x7f53b40a4910 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 client.45029.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 monclient: _renew_subs 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b4077ba0 con 0x7f53b40a4910 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 1 librados: init done 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 0x7f53b413db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f53b4051e80 con 0x7f53b4124110 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 0x7f53b413db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient: get_auth_request con 0x7f53b4124110 auth_method 0 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f53a401b020 con 0x7f53b40a4910 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_done global_id 45038 payload 824 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53b4139ae0 con 0x7f53b4124110 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53a003e550 0x7f53a0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f53a400f040 con 0x7f53b40a4910 2024-09-17T16:39:39.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.45029.objecter ms_dispatch 0x7f53b4000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 3 client.45029.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 3 client.45029.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 20 client.45029.objecter dump_active .. 0 homeless 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.45029.objecter ms_handle_connect 0x7f53b4124110 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.45029.objecter resend_mon_ops 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 0x7f53b413db20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53a80ad000 tx=0x7f53a80191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a80ba070 con 0x7f53b4124110 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53a003e550 0x7f53a0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: dump: 2024-09-17T16:39:39.761 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.762 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 10 monclient: get_auth_request con 0x7f53a003e550 auth_method 0 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f53a8003620 con 0x7f53b4124110 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 msgr2=0x7f53b413db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 0x7f53b413db20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53a80ad000 tx=0x7f53a80191e0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 msgr2=0x7f53b413db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b4124110 0x7f53b413db20 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53a80ad000 tx=0x7f53a80191e0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f539bfff640 10 client.45029.objecter ms_handle_connect 0x7f53a003e550 2024-09-17T16:39:39.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.732+0000 7f53b3fff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53a003e550 0x7f53a0040a10 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 crypto rx=0x7f53b4074620 tx=0x7f53ac009780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 0x7f53b413e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f53b413e9b0 con 0x7f53b413e060 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 0x7f53b413e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient: get_auth_request con 0x7f53b413e060 auth_method 0 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 10 monclient(hunting): handle_auth_done global_id 45047 payload 824 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 1 -- 172.21.15.105:0/3750703492 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f53b4123a70 con 0x7f53b413e060 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b2ffd640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 0x7f53b413e470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53a801c270 tx=0x7f53a80a6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 client.45029.objecter ms_handle_connect 0x7f53b413e060 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 client.45029.objecter resend_mon_ops 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53a80b0040 con 0x7f53b413e060 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 monclient: dump: 2024-09-17T16:39:39.766 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.767 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9621 (secure 0 0 0) 0x7f53a80c3070 con 0x7f53b413e060 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 msgr2=0x7f53b413e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 0x7f53b413e470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53a801c270 tx=0x7f53a80a6000 comp rx=0 tx=0).stop 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 msgr2=0x7f53b413e470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f539bfff640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b413e060 0x7f53b413e470 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53a801c270 tx=0x7f53a80a6000 comp rx=0 tx=0).stop 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:39.768 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 430393, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:39.769 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.772 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:39.774 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.775 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.776 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.777 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.778 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:39.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.782 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.783 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.786 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.788 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.789 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.790 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:39.791 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 librados: watch_flush enter 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 librados: watch_flush exit 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 20 client.45029.objecter shutdown clearing up homeless session... 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 client.45029.objecter successfully canceled tick 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53a003e550 msgr2=0x7f53a0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f53a003e550 0x7f53a0040a10 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 crypto rx=0x7f53b4074620 tx=0x7f53ac009780 comp rx=0 tx=0).stop 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 10 monclient: shutdown 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 msgr2=0x7f53b41d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 --2- 172.21.15.105:0/3750703492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53b40a4910 0x7f53b41d1650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53a400d800 tx=0x7f53a400dcc0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b3fff640 1 -- 172.21.15.105:0/3750703492 reap_dead start 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b3fff640 1 -- 172.21.15.105:0/3750703492 reap_dead start 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 shutdown_connections 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 >> 172.21.15.105:0/3750703492 conn(0x7f53b4068ba0 msgr2=0x7f53b40a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 shutdown_connections 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 -- 172.21.15.105:0/3750703492 wait complete. 2024-09-17T16:39:39.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.740+0000 7f53b9f6b640 1 librados: shutdown 2024-09-17T16:39:39.794 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:39.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:39:39.795 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: get_monmap_and_config 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: monmap: 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:39.908876+0000 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:39.908876+0000 2024-09-17T16:39:39.912 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:39.913 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 Processor -- start 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 -- start start 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: init 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient: _add_conns ranks=[10,7,3,4,6,2,9,0,1,5,8] 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 0x7f6d9409b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-f con 0x7f6d9409b410 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-i con 0x7f6d940a2690 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 0x7f6d94136300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-g con 0x7f6d94135e50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6d940a03f0 con 0x7f6d9409b410 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6d940a4cd0 con 0x7f6d94135e50 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6d940a0e40 con 0x7f6d940a2690 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9baa5640 10 monclient(hunting): authenticate will time out at 1663.174358s 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 0x7f6d9409b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 0x7f6d94136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:34764/0 (socket says 172.21.15.105:34764) 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 -- 172.21.15.105:0/545221548 learned_addr learned my addr 172.21.15.105:0/545221548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): get_auth_request con 0x7f6d940a2690 auth_method 0 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): get_auth_request con 0x7f6d94135e50 auth_method 0 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): get_auth_request con 0x7f6d9409b410 auth_method 0 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient(hunting): handle_auth_done global_id 64330 payload 274 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 -- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 msgr2=0x7f6d94136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 0x7f6d94136300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 -- 172.21.15.105:0/545221548 >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 msgr2=0x7f6d9409b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:39.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 --2- 172.21.15.105:0/545221548 >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 0x7f6d9409b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 monclient: found mon.noname-i 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99040640 1 -- 172.21.15.105:0/545221548 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d940a1120 con 0x7f6d940a2690 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d99841640 1 --2- 172.21.15.105:0/545221548 >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 0x7f6d9409b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:39.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.904+0000 7f6d9a042640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 0x7f6d94136300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d99040640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009870 tx=0x7f6d8401c3f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=d924bb8810983c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 1 -- 172.21.15.105:0/545221548 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840b0070 con 0x7f6d940a2690 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: dump: 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.927 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 1 -- 172.21.15.105:0/545221548 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840aa810 con 0x7f6d940a2690 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 5 monclient: authenticate success, global_id 64330 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 4 set_mon_vals no callback set 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 1 -- 172.21.15.105:0/545221548 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840a9300 con 0x7f6d940a2690 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9883f640 10 monclient: dump: 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 10 monclient: shutdown 2024-09-17T16:39:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 msgr2=0x7f6d94135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009870 tx=0x7f6d8401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- 172.21.15.105:0/545221548 shutdown_connections 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d94135e50 0x7f6d94136300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/545221548 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d940a2690 0x7f6d94135910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/545221548 >> v2:172.21.15.160:3300/0 conn(0x7f6d9409b410 0x7f6d9409b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- 172.21.15.105:0/545221548 >> 172.21.15.105:0/545221548 conn(0x7f6d940a83e0 msgr2=0x7f6d940a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- 172.21.15.105:0/545221548 shutdown_connections 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- 172.21.15.105:0/545221548 wait complete. 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 10 monclient: build_initial_monmap 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 10 monclient: monmap: 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 librados: starting msgr at 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 librados: starting objecter 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 Processor -- start 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 -- start start 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 librados: setting wanted keys 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 1 librados: calling monclient init 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.908+0000 7f6d9baa5640 10 monclient: init 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _add_conns ranks=[3,1,7,8,6,2,5,0,4] 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d9409b410 0x7f6d941cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-g con 0x7f6d9409b410 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 0x7f6d941cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-f con 0x7f6d94135e50 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d94071e50 0x7f6d940722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): picked mon.noname-i con 0x7f6d94071e50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9409beb0 con 0x7f6d94135e50 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9409bd30 con 0x7f6d9409b410 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6d9413cc80 con 0x7f6d94071e50 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): authenticate will time out at 1663.180299s 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 0x7f6d941cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d94071e50 0x7f6d940722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d9409b410 0x7f6d941cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 0x7f6d941cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:51530/0 (socket says 172.21.15.105:51530) 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 -- 172.21.15.105:0/332811689 learned_addr learned my addr 172.21.15.105:0/332811689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): get_auth_request con 0x7f6d94135e50 auth_method 0 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d94071e50 0x7f6d940722b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:34768/0 (socket says 172.21.15.105:34768) 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): get_auth_request con 0x7f6d94071e50 auth_method 0 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): get_auth_request con 0x7f6d9409b410 auth_method 0 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9a042640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient(hunting): handle_auth_done global_id 54595 payload 824 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d94071e50 msgr2=0x7f6d940722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6d94071e50 0x7f6d940722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d9409b410 msgr2=0x7f6d941cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d9409b410 0x7f6d941cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient: _finish_hunting 0 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 monclient: found mon.noname-f 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d94073730 con 0x7f6d94135e50 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99040640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 0x7f6d941cf540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d840ae040 tx=0x7f6d84019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 client.?.objecter ms_handle_connect 0x7f6d94135e50 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840b9070 con 0x7f6d94135e50 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: dump: 2024-09-17T16:39:39.937 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840044b0 con 0x7f6d94135e50 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 5 monclient: authenticate success, global_id 54595 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d840b0040 con 0x7f6d94135e50 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8bfff640 4 set_mon_vals no callback set 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 monclient: dump: 2024-09-17T16:39:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.940 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _renew_subs 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d94137d40 con 0x7f6d94135e50 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 client.54595.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _renew_subs 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d941cfa80 con 0x7f6d94135e50 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 librados: init done 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 0x7f6d940727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6d94136fe0 con 0x7f6d940a2690 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 0x7f6d940727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient: get_auth_request con 0x7f6d940a2690 auth_method 0 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 10 monclient(hunting): handle_auth_done global_id 74191 payload 824 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d940a8ae0 con 0x7f6d940a2690 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 client.54595.objecter ms_handle_connect 0x7f6d940a2690 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 client.54595.objecter resend_mon_ops 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d99841640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 0x7f6d940727f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d900119d0 tx=0x7f6d90011e90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6d840aac00 con 0x7f6d94135e50 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7803e550 0x7f6d78040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f6d840e8ff0 con 0x7f6d94135e50 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.912+0000 7f6d8a7fc640 10 client.54595.objecter ms_dispatch 0x7f6d94000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 3 client.54595.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 3 client.54595.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 20 client.54595.objecter dump_active .. 0 homeless 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d9000fc20 con 0x7f6d940a2690 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d9a042640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7803e550 0x7f6d78040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 monclient: dump: 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.943 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.944 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6d9000fdc0 con 0x7f6d940a2690 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 msgr2=0x7f6d940727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 0x7f6d940727f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d900119d0 tx=0x7f6d90011e90 comp rx=0 tx=0).stop 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 msgr2=0x7f6d940727f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d940a2690 0x7f6d940727f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d9a042640 10 monclient: get_auth_request con 0x7f6d7803e550 auth_method 0 2024-09-17T16:39:39.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d9a042640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7803e550 0x7f6d78040a10 secure :-1 s=READY pgs=3562 cs=0 l=1 rev1=1 crypto rx=0x7f6d94073530 tx=0x7f6d8c00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.916+0000 7f6d8a7fc640 10 client.54595.objecter ms_handle_connect 0x7f6d7803e550 2024-09-17T16:39:39.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 0x7f6d94099240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6d94051e80 con 0x7f6d94072d30 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 0x7f6d94099240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient: get_auth_request con 0x7f6d94072d30 auth_method 0 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:39.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 10 monclient(hunting): handle_auth_done global_id 74200 payload 824 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 1 -- 172.21.15.105:0/332811689 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6d9406a1c0 con 0x7f6d94072d30 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d99841640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 0x7f6d94099240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d900119d0 tx=0x7f6d900046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 client.54595.objecter ms_handle_connect 0x7f6d94072d30 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 client.54595.objecter resend_mon_ops 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d9000fc20 con 0x7f6d94072d30 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 monclient: dump: 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:39.949 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f6d9002c420 con 0x7f6d94072d30 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:39.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 msgr2=0x7f6d94099240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 0x7f6d94099240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d900119d0 tx=0x7f6d900046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 msgr2=0x7f6d94099240 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d8a7fc640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6d94072d30 0x7f6d94099240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19383, 2024-09-17T16:39:39.951 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.952 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.953 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.954 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:39.955 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:39.956 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:39.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.958 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:39.960 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:39.961 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.963 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:39.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.967 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.970 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:39.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:39.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:39.979 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:39.980 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:39.981 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:39.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 10 librados: watch_flush enter 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 10 librados: watch_flush exit 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 20 client.54595.objecter shutdown clearing up homeless session... 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.920+0000 7f6d9baa5640 10 client.54595.objecter successfully canceled tick 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7803e550 msgr2=0x7f6d78040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6d7803e550 0x7f6d78040a10 secure :-1 s=READY pgs=3562 cs=0 l=1 rev1=1 crypto rx=0x7f6d94073530 tx=0x7f6d8c00b040 comp rx=0 tx=0).stop 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 10 monclient: shutdown 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 msgr2=0x7f6d941cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 --2- 172.21.15.105:0/332811689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d94135e50 0x7f6d941cf540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d840ae040 tx=0x7f6d84019190 comp rx=0 tx=0).stop 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9a042640 1 -- 172.21.15.105:0/332811689 reap_dead start 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9a042640 1 -- 172.21.15.105:0/332811689 reap_dead start 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 shutdown_connections 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 >> 172.21.15.105:0/332811689 conn(0x7f6d940a83e0 msgr2=0x7f6d940a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:39.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 shutdown_connections 2024-09-17T16:39:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 -- 172.21.15.105:0/332811689 wait complete. 2024-09-17T16:39:39.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:39.924+0000 7f6d9baa5640 1 librados: shutdown 2024-09-17T16:39:39.984 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:39.984 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:39:39.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:39:40.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: get_monmap_and_config 2024-09-17T16:39:40.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: monmap: 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:40.100280+0000 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:40.100280+0000 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.104 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:40.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 Processor -- start 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 -- start start 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: init 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient: _add_conns ranks=[5,3,2,0,4,1,6,10,7,9,8] 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3720 0x7f0bb00a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-h con 0x7f0bb00a3720 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bb00a3b00 0x7f0bb00a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-g con 0x7f0bb00a3b00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-b con 0x7f0bb00a3f00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb006db50 con 0x7f0bb00a3f00 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb00a0e80 con 0x7f0bb00a3b00 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0bb00a1490 con 0x7f0bb00a3720 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb82a7640 10 monclient(hunting): authenticate will time out at 1663.365960s 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58604/0 (socket says 172.21.15.105:58604) 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 -- 172.21.15.105:0/1212745986 learned_addr learned my addr 172.21.15.105:0/1212745986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): get_auth_request con 0x7f0bb00a3f00 auth_method 0 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6043640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3720 0x7f0bb00a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient(hunting): handle_auth_done global_id 45056 payload 274 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 -- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3720 msgr2=0x7f0bb00a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3720 0x7f0bb00a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb5842640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bb00a3b00 0x7f0bb00a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 -- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bb00a3b00 msgr2=0x7f0bb00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bb00a3b00 0x7f0bb00a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 monclient: found mon.noname-b 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.096+0000 7f0bb6844640 1 -- 172.21.15.105:0/1212745986 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb00a7240 con 0x7f0bb00a3f00 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6844640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29aa9ee35aa1064d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 1 -- 172.21.15.105:0/1212745986 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80090d0 con 0x7f0bb00a3f00 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: dump: 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.111 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 1 -- 172.21.15.105:0/1212745986 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba80a5a70 con 0x7f0bb00a3f00 2024-09-17T16:39:40.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 5 monclient: authenticate success, global_id 45056 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 1 -- 172.21.15.105:0/1212745986 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80a5c60 con 0x7f0bb00a3f00 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 4 set_mon_vals no callback set 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5041640 10 monclient: dump: 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.113 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.114 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: shutdown 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- 172.21.15.105:0/1212745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 msgr2=0x7f0bb00a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba801c910 comp rx=0 tx=0).stop 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- 172.21.15.105:0/1212745986 shutdown_connections 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3f00 0x7f0bb00a5c00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bb00a3b00 0x7f0bb00a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/1212745986 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3720 0x7f0bb00a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- 172.21.15.105:0/1212745986 >> 172.21.15.105:0/1212745986 conn(0x7f0bb00aa390 msgr2=0x7f0bb00a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- 172.21.15.105:0/1212745986 shutdown_connections 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- 172.21.15.105:0/1212745986 wait complete. 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: monmap: 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 librados: starting msgr at 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 librados: starting objecter 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 Processor -- start 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- start start 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 librados: setting wanted keys 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 librados: calling monclient init 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: init 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient: _add_conns ranks=[2,5,1,7,6,0,8,4,3] 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-b con 0x7f0bb00a3720 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3b00 0x7f0bb01d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-h con 0x7f0bb00a3b00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bb00a3f00 0x7f0bb01d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): picked mon.noname-f con 0x7f0bb00a3f00 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01432d0 con 0x7f0bb00a3f00 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb00a6550 con 0x7f0bb00a3720 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0bb0143450 con 0x7f0bb00a3b00 2024-09-17T16:39:40.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb82a7640 10 monclient(hunting): authenticate will time out at 1663.371095s 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58610/0 (socket says 172.21.15.105:58610) 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 learned_addr learned my addr 172.21.15.105:0/2010495747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): get_auth_request con 0x7f0bb00a3720 auth_method 0 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3b00 0x7f0bb01d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient(hunting): handle_auth_done global_id 45065 payload 824 2024-09-17T16:39:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3b00 msgr2=0x7f0bb01d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3b00 0x7f0bb01d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bb00a3f00 msgr2=0x7f0bb01d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bb00a3f00 0x7f0bb01d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 monclient: found mon.noname-b 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb01268b0 con 0x7f0bb00a3720 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5842640 10 monclient: get_auth_request con 0x7f0bb00a3b00 auth_method 0 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0bb00a3b00 0x7f0bb01d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c7f0 tx=0x7f0bac00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 client.?.objecter ms_handle_connect 0x7f0bb00a3720 2024-09-17T16:39:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 msgr2=0x7f0bb01d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 msgr2=0x7f0bb01d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c7f0 tx=0x7f0bac00ccb0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c7f0 tx=0x7f0bac00ccb0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 msgr2=0x7f0bb01d1370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bb00a3720 0x7f0bb01d1370 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient: _add_conns ranks=[0,3,7,4,6,8] 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 0x7f0b940073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f0b94004fa0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b94007f40 0x7f0b9400a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f0b94007f40 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0b9400af70 0x7f0b9400d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f0b9400af70 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9400e020 con 0x7f0b94004fa0 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9400e1a0 con 0x7f0b94007f40 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0b9400e3a0 con 0x7f0b9400af70 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0ba6ffd640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.100+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 0x7f0b940073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): get_auth_request con 0x7f0b94004fa0 auth_method 0 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient(hunting): handle_auth_done global_id 45065 payload 293 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0b9400af70 msgr2=0x7f0b9400d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b94007f40 0x7f0b9400a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6844640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0b9400af70 0x7f0b9400d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0b9400af70 0x7f0b9400d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b94007f40 msgr2=0x7f0b9400a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b94007f40 0x7f0b9400a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 monclient: found mon.noname-a 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6844640 1 -- 172.21.15.105:0/2010495747 reap_dead start 2024-09-17T16:39:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9400e660 con 0x7f0b94004fa0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient: get_auth_request con 0x7f0b94007f40 auth_method 0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b94007f40 0x7f0b9400a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6043640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 0x7f0b940073f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0bac016000 tx=0x7f0bac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.?.objecter ms_handle_connect 0x7f0b94004fa0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bac015070 con 0x7f0b94004fa0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: dump: 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.133 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 5 monclient: authenticate success, global_id 45065 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 monclient: _renew_subs 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac004340 con 0x7f0b94004fa0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb00a7cb0 con 0x7f0b94004fa0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bac003620 con 0x7f0b94004fa0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 client.45065.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb4840640 4 set_mon_vals no callback set 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: dump: 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 monclient: _renew_subs 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb0126e70 con 0x7f0b94004fa0 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 1 librados: init done 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0bb0051e80 con 0x7f0bb01d1370 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient: get_auth_request con 0x7f0bb01d1370 auth_method 0 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0bac0037c0 con 0x7f0b94004fa0 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 10 monclient(hunting): handle_auth_done global_id 24666 payload 824 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bb00a1e70 con 0x7f0bb01d1370 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 0x7f0b94045140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f0bac0515f0 con 0x7f0b94004fa0 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.45065.objecter ms_dispatch 0x7f0bb0000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 3 client.45065.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 3 client.45065.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0b94004ec0 tx=0x7f0ba000ea30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 20 client.45065.objecter dump_active .. 0 homeless 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.45065.objecter ms_handle_connect 0x7f0bb01d1370 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.45065.objecter resend_mon_ops 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba0012ab0 con 0x7f0bb01d1370 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: dump: 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6844640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 0x7f0b94045140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6844640 10 monclient: get_auth_request con 0x7f0b94042c80 auth_method 0 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ba0012640 con 0x7f0bb01d1370 2024-09-17T16:39:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 msgr2=0x7f0bb01d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0b94004ec0 tx=0x7f0ba000ea30 comp rx=0 tx=0).stop 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 msgr2=0x7f0bb01d1750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0bb6844640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 0x7f0b94045140 secure :-1 s=READY pgs=3563 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80ae000 tx=0x7f0ba80a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.104+0000 7f0ba6ffd640 10 client.45065.objecter ms_handle_connect 0x7f0b94042c80 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0bb013f350 con 0x7f0bb01d1c90 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient: get_auth_request con 0x7f0bb01d1c90 auth_method 0 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 10 monclient(hunting): handle_auth_done global_id 24675 payload 824 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 1 -- 172.21.15.105:0/2010495747 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0bb01265d0 con 0x7f0bb01d1c90 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb5842640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0019000 tx=0x7f0ba0004740 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 client.45065.objecter ms_handle_connect 0x7f0bb01d1c90 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 client.45065.objecter resend_mon_ops 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba001f570 con 0x7f0bb01d1c90 2024-09-17T16:39:40.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 monclient: dump: 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.142 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f0ba0009a90 con 0x7f0bb01d1c90 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 msgr2=0x7f0bb0071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0019000 tx=0x7f0ba0004740 comp rx=0 tx=0).stop 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 msgr2=0x7f0bb0071090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0ba6ffd640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.143 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 727752, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:40.144 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.145 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:40.147 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.148 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.149 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.150 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:40.153 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.155 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.158 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.159 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:40.161 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.163 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:40.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.165 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:40.166 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:40.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 librados: watch_flush enter 2024-09-17T16:39:40.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 librados: watch_flush exit 2024-09-17T16:39:40.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 20 client.45065.objecter shutdown clearing up homeless session... 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 client.45065.objecter successfully canceled tick 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 msgr2=0x7f0b94045140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 0x7f0b94045140 secure :-1 s=READY pgs=3563 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80ae000 tx=0x7f0ba80a7000 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 10 monclient: shutdown 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 msgr2=0x7f0b940073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 0x7f0b940073f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0bac016000 tx=0x7f0bac008040 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 shutdown_connections 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0b94042c80 0x7f0b94045140 unknown :-1 s=CLOSED pgs=3563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b94004fa0 0x7f0b940073f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1370 0x7f0bb01d1750 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 --2- 172.21.15.105:0/2010495747 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f0bb01d1c90 0x7f0bb0071090 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 >> 172.21.15.105:0/2010495747 conn(0x7f0bb00aa390 msgr2=0x7f0bb00a8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 shutdown_connections 2024-09-17T16:39:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 -- 172.21.15.105:0/2010495747 wait complete. 2024-09-17T16:39:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.112+0000 7f0bb82a7640 1 librados: shutdown 2024-09-17T16:39:40.169 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:40.169 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:39:40.169 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: get_monmap_and_config 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: monmap: 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:40.283853+0000 2024-09-17T16:39:40.287 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:40.283853+0000 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:40.288 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 Processor -- start 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 -- start start 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: init 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient: _add_conns ranks=[6,3,10,4,2,5,9,8,0,7,1] 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): picked mon.noname-d con 0x7f50940a2de0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50940a3700 0x7f509409b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): picked mon.noname-g con 0x7f50940a3700 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f50940a3ae0 0x7f509409b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): picked mon.noname-f con 0x7f50940a3ae0 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f509409c9a0 con 0x7f50940a3ae0 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f50940a20b0 con 0x7f50940a3700 2024-09-17T16:39:40.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f50940a2770 con 0x7f50940a2de0 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5099697640 10 monclient(hunting): authenticate will time out at 1663.549505s 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:53092/0 (socket says 172.21.15.105:53092) 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 -- 172.21.15.105:0/1115038565 learned_addr learned my addr 172.21.15.105:0/1115038565 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): get_auth_request con 0x7f50940a2de0 auth_method 0 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50927fc640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50940a3700 0x7f509409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50937fe640 1 --2- 172.21.15.105:0/1115038565 >> v2:172.21.15.160:3300/0 conn(0x7f50940a3ae0 0x7f509409b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50927fc640 10 monclient(hunting): get_auth_request con 0x7f50940a3700 auth_method 0 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50927fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50937fe640 10 monclient(hunting): get_auth_request con 0x7f50940a3ae0 auth_method 0 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50937fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f50937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient(hunting): handle_auth_done global_id 24684 payload 274 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 -- 172.21.15.105:0/1115038565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50940a3700 msgr2=0x7f509409b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50940a3700 0x7f509409b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 -- 172.21.15.105:0/1115038565 >> v2:172.21.15.160:3300/0 conn(0x7f50940a3ae0 msgr2=0x7f509409b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 --2- 172.21.15.105:0/1115038565 >> v2:172.21.15.160:3300/0 conn(0x7f50940a3ae0 0x7f509409b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 monclient: found mon.noname-d 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.280+0000 7f5092ffd640 1 -- 172.21.15.105:0/1115038565 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50940a24f0 con 0x7f50940a2de0 2024-09-17T16:39:40.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f508c009e80 tx=0x7f508c01c8d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=30b6520ee0b21a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 1 -- 172.21.15.105:0/1115038565 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f508c0090d0 con 0x7f50940a2de0 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: dump: 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.301 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 5 monclient: authenticate success, global_id 24684 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 1 -- 172.21.15.105:0/1115038565 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c0a5980 con 0x7f50940a2de0 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 1 -- 172.21.15.105:0/1115038565 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f508c0ad3f0 con 0x7f50940a2de0 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 4 set_mon_vals no callback set 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5091ffb640 10 monclient: dump: 2024-09-17T16:39:40.303 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.304 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: shutdown 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- 172.21.15.105:0/1115038565 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 msgr2=0x7f50940a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f508c009e80 tx=0x7f508c01c8d0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- 172.21.15.105:0/1115038565 shutdown_connections 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- 172.21.15.105:0/1115038565 >> v2:172.21.15.160:3300/0 conn(0x7f50940a3ae0 0x7f509409b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50940a3700 0x7f509409b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- 172.21.15.105:0/1115038565 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f50940a2de0 0x7f50940a31c0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- 172.21.15.105:0/1115038565 >> 172.21.15.105:0/1115038565 conn(0x7f50940a83e0 msgr2=0x7f50940a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- 172.21.15.105:0/1115038565 shutdown_connections 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- 172.21.15.105:0/1115038565 wait complete. 2024-09-17T16:39:40.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: monmap: 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.306 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 librados: starting msgr at 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 librados: starting objecter 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 Processor -- start 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- start start 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 librados: setting wanted keys 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 librados: calling monclient init 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: init 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient: _add_conns ranks=[8,4,2,5,3,6,7,0,1] 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 0x7f50941cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): picked mon.noname-e con 0x7f50940a2de0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f50940a3700 0x7f50941cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): picked mon.noname-c con 0x7f50940a3700 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50940a3ae0 0x7f50941d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): picked mon.noname-b con 0x7f50940a3ae0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f50941413d0 con 0x7f50940a3ae0 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f509409bee0 con 0x7f50940a3700 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5094141550 con 0x7f50940a2de0 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5099697640 10 monclient(hunting): authenticate will time out at 1663.555068s 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 0x7f50941cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f50940a3700 0x7f50941cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 0x7f50941cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34818/0 (socket says 172.21.15.105:34818) 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 1 -- 172.21.15.105:0/3935548687 learned_addr learned my addr 172.21.15.105:0/3935548687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): get_auth_request con 0x7f50940a2de0 auth_method 0 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50940a3ae0 0x7f50941d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): get_auth_request con 0x7f50940a3700 auth_method 0 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): get_auth_request con 0x7f50940a3ae0 auth_method 0 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient(hunting): handle_auth_done global_id 74209 payload 824 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 msgr2=0x7f50941cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 0x7f50941cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50940a3ae0 msgr2=0x7f50941d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50940a3ae0 0x7f50941d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 monclient: found mon.noname-c 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5094074b40 con 0x7f50940a3700 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f50940a2de0 0x7f50941cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50937fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50940a3ae0 0x7f50941d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f50927fc640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f50940a3700 0x7f50941cfb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f508800d620 tx=0x7f508800da20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f507b7fe640 10 client.?.objecter ms_handle_connect 0x7f50940a3700 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.284+0000 7f507b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5088012a60 con 0x7f50940a3700 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: dump: 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.312 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5088019070 con 0x7f50940a3700 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 5 monclient: authenticate success, global_id 74209 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 monclient: _renew_subs 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509412f910 con 0x7f50940a3700 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5088012c60 con 0x7f50940a3700 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: dump: 2024-09-17T16:39:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.315 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 client.74209.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f50917fa640 4 set_mon_vals no callback set 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 monclient: _renew_subs 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f50940a58c0 con 0x7f50940a3700 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 1 librados: init done 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 0x7f509413c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5094051e80 con 0x7f509412fd30 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 0x7f509413c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient: get_auth_request con 0x7f509412fd30 auth_method 0 2024-09-17T16:39:40.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f50880125f0 con 0x7f50940a3700 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 10 monclient(hunting): handle_auth_done global_id 54521 payload 824 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50940a31c0 con 0x7f509412fd30 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f508003e580 0x7f5080040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f508800f040 con 0x7f50940a3700 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 client.74209.objecter ms_dispatch 0x7f5094000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 0x7f509413c100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f508c0af040 tx=0x7f508c019670 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 3 client.74209.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 3 client.74209.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 20 client.74209.objecter dump_active .. 0 homeless 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 client.74209.objecter ms_handle_connect 0x7f509412fd30 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 client.74209.objecter resend_mon_ops 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f508c0a5d30 con 0x7f509412fd30 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: dump: 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.318 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f50937fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f508003e580 0x7f5080040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f50937fe640 10 monclient: get_auth_request con 0x7f508003e580 auth_method 0 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f508c0a58c0 con 0x7f509412fd30 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 msgr2=0x7f509413c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 0x7f509413c100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f508c0af040 tx=0x7f508c019670 comp rx=0 tx=0).stop 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 msgr2=0x7f509413c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509412fd30 0x7f509413c100 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f508c0af040 tx=0x7f508c019670 comp rx=0 tx=0).stop 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f507b7fe640 10 client.74209.objecter ms_handle_connect 0x7f508003e580 2024-09-17T16:39:40.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.288+0000 7f50937fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f508003e580 0x7f5080040a40 secure :-1 s=READY pgs=3564 cs=0 l=1 rev1=1 crypto rx=0x7f509412eb70 tx=0x7f5084008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.292+0000 7f5099697640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:40.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.292+0000 7f5099697640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.292+0000 7f5099697640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.292+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f509413cf90 con 0x7f509413c640 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient: get_auth_request con 0x7f509413c640 auth_method 0 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 10 monclient(hunting): handle_auth_done global_id 54530 payload 824 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 1 -- 172.21.15.105:0/3935548687 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f509406a1c0 con 0x7f509413c640 2024-09-17T16:39:40.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 msgr2=0x7f509413ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send send error: (32) Broken pipe 2024-09-17T16:39:40.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f508c009e80 tx=0x7f508c0a7000 comp rx=0 tx=0).write compression request write failed r=-32 ((32) Broken pipe) 2024-09-17T16:39:40.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.296+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:40.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: get_auth_request con 0x7f509413c640 auth_method 0 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: _init_auth method 2 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: _init_auth already have auth, reseting 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: handle_auth_reply_more payload 9 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-17T16:39:40.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 10 monclient: handle_auth_done global_id 54530 payload 293 2024-09-17T16:39:40.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5092ffd640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f508c0196b0 tx=0x7f508c0b9000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 client.74209.objecter ms_handle_connect 0x7f509413c640 2024-09-17T16:39:40.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 client.74209.objecter resend_mon_ops 2024-09-17T16:39:40.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f508c0c3090 con 0x7f509413c640 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 monclient: dump: 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.511 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f508c0041b0 con 0x7f509413c640 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 msgr2=0x7f509413ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f508c0196b0 tx=0x7f508c0b9000 comp rx=0 tx=0).stop 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 msgr2=0x7f509413ca50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f507b7fe640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f509413c640 0x7f509413ca50 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f508c0196b0 tx=0x7f508c0b9000 comp rx=0 tx=0).stop 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 287498, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:40.513 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.514 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.515 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:40.516 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.517 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.518 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:40.522 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:40.526 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.528 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:40.529 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.530 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.532 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.533 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.534 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.535 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:40.536 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5099697640 10 librados: watch_flush enter 2024-09-17T16:39:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5099697640 10 librados: watch_flush exit 2024-09-17T16:39:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5099697640 20 client.74209.objecter shutdown clearing up homeless session... 2024-09-17T16:39:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.500+0000 7f5099697640 10 client.74209.objecter successfully canceled tick 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f508003e580 msgr2=0x7f5080040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f508003e580 0x7f5080040a40 secure :-1 s=READY pgs=3564 cs=0 l=1 rev1=1 crypto rx=0x7f509412eb70 tx=0x7f5084008040 comp rx=0 tx=0).stop 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 10 monclient: shutdown 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f50940a3700 msgr2=0x7f50941cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 --2- 172.21.15.105:0/3935548687 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f50940a3700 0x7f50941cfb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f508800d620 tx=0x7f508800da20 comp rx=0 tx=0).stop 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f50937fe640 1 -- 172.21.15.105:0/3935548687 reap_dead start 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f50937fe640 1 -- 172.21.15.105:0/3935548687 reap_dead start 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 shutdown_connections 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 >> 172.21.15.105:0/3935548687 conn(0x7f50940a83e0 msgr2=0x7f509413de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 shutdown_connections 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 -- 172.21.15.105:0/3935548687 wait complete. 2024-09-17T16:39:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.504+0000 7f5099697640 1 librados: shutdown 2024-09-17T16:39:40.539 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:40.539 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:39:40.539 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:39:40.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: get_monmap_and_config 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: monmap: 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:40.660524+0000 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:40.660524+0000 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.664 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:40.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 Processor -- start 2024-09-17T16:39:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 -- start start 2024-09-17T16:39:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: init 2024-09-17T16:39:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient: _add_conns ranks=[10,4,2,1,7,5,3,6,0,8,9] 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff97c0a5860 0x7ff97c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-f con 0x7ff97c0a5860 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 0x7ff97c139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-c con 0x7ff97c0a6180 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-b con 0x7ff97c139f40 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ff97c06d8a0 con 0x7ff97c0a5860 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c06d460 con 0x7ff97c139f40 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff97c06ebc0 con 0x7ff97c0a6180 2024-09-17T16:39:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff98175c640 10 monclient(hunting): authenticate will time out at 1663.926358s 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 0x7ff97c139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 0x7ff97c139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:33586/0 (socket says 172.21.15.105:33586) 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58630/0 (socket says 172.21.15.105:58630) 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 1 -- 172.21.15.105:0/1921503184 learned_addr learned my addr 172.21.15.105:0/1921503184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): get_auth_request con 0x7ff97c0a6180 auth_method 0 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): get_auth_request con 0x7ff97c139f40 auth_method 0 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient(hunting): handle_auth_done global_id 45083 payload 274 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 -- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 msgr2=0x7ff97c139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 0x7ff97c139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 -- 172.21.15.105:0/1921503184 >> v2:172.21.15.160:3300/0 conn(0x7ff97c0a5860 msgr2=0x7ff97c0a5c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/1921503184 >> v2:172.21.15.160:3300/0 conn(0x7ff97c0a5860 0x7ff97c0a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 monclient: found mon.noname-b 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.656+0000 7ff97b7fe640 1 -- 172.21.15.105:0/1921503184 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0a7320 con 0x7ff97c139f40 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff974009d90 tx=0x7ff97401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771752c65c1b4348 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 1 -- 172.21.15.105:0/1921503184 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9740b2070 con 0x7ff97c139f40 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: dump: 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.682 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 1 -- 172.21.15.105:0/1921503184 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9740a5a10 con 0x7ff97c139f40 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 1 -- 172.21.15.105:0/1921503184 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9740a5c40 con 0x7ff97c139f40 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 5 monclient: authenticate success, global_id 45083 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 4 set_mon_vals no callback set 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97a7fc640 10 monclient: dump: 2024-09-17T16:39:40.683 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: shutdown 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 msgr2=0x7ff97c13a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff974009d90 tx=0x7ff97401c850 comp rx=0 tx=0).stop 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- 172.21.15.105:0/1921503184 shutdown_connections 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff97c139f40 0x7ff97c13a3c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- 172.21.15.105:0/1921503184 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff97c0a6180 0x7ff97c139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- 172.21.15.105:0/1921503184 >> v2:172.21.15.160:3300/0 conn(0x7ff97c0a5860 0x7ff97c0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- 172.21.15.105:0/1921503184 >> 172.21.15.105:0/1921503184 conn(0x7ff97c0aa450 msgr2=0x7ff97c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- 172.21.15.105:0/1921503184 shutdown_connections 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- 172.21.15.105:0/1921503184 wait complete. 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: monmap: 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 librados: starting msgr at 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 librados: starting objecter 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 Processor -- start 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- start start 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 librados: setting wanted keys 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 librados: calling monclient init 2024-09-17T16:39:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: init 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient: _add_conns ranks=[5,8,7,4,0,6,2,1,3] 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff97c0a5860 0x7ff97c1d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-h con 0x7ff97c0a5860 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 0x7ff97c1d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-e con 0x7ff97c0a6180 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff97c139f40 0x7ff97c13b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): picked mon.noname-i con 0x7ff97c139f40 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff97c06f390 con 0x7ff97c0a5860 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff97c0a0f00 con 0x7ff97c139f40 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff97c143470 con 0x7ff97c0a6180 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff98175c640 10 monclient(hunting): authenticate will time out at 1663.931458s 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 0x7ff97c1d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 0x7ff97c1d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:34848/0 (socket says 172.21.15.105:34848) 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 -- 172.21.15.105:0/3580017605 learned_addr learned my addr 172.21.15.105:0/3580017605 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): get_auth_request con 0x7ff97c0a6180 auth_method 0 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff97c139f40 0x7ff97c13b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff97c0a5860 0x7ff97c1d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient(hunting): handle_auth_done global_id 54539 payload 824 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff97c139f40 msgr2=0x7ff97c13b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff97c139f40 0x7ff97c13b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff97c0a5860 msgr2=0x7ff97c1d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff97c0a5860 0x7ff97c1d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 monclient: found mon.noname-e 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff973fff640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c13ef40 con 0x7ff97c0a6180 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97b7fe640 10 monclient: get_auth_request con 0x7ff97c139f40 auth_method 0 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97affd640 10 monclient: get_auth_request con 0x7ff97c0a5860 auth_method 0 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff97c139f40 0x7ff97c13b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:40.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.660+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff97c0a5860 0x7ff97c1d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff973fff640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 0x7ff97c1d1b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e880 tx=0x7ff96400ed40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.?.objecter ms_handle_connect 0x7ff97c0a6180 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff964017070 con 0x7ff97c0a6180 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: dump: 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.693 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96400cdb0 con 0x7ff97c0a6180 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9640138e0 con 0x7ff97c0a6180 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: dump: 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.695 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff979ffb640 4 set_mon_vals no callback set 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 5 monclient: authenticate success, global_id 54539 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 monclient: _renew_subs 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c0a3930 con 0x7ff97c0a6180 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 client.54539.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 monclient: _renew_subs 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c13f220 con 0x7ff97c0a6180 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 1 librados: init done 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 0x7ff97c13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c051e80 con 0x7ff97c13b910 2024-09-17T16:39:40.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff96401a070 con 0x7ff97c0a6180 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 0x7ff97c13bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient: get_auth_request con 0x7ff97c13b910 auth_method 0 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff95c03e550 0x7ff95c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7ff96400f040 con 0x7ff97c0a6180 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.54539.objecter ms_dispatch 0x7ff97c000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 3 client.54539.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:40.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 3 client.54539.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 20 client.54539.objecter dump_active .. 0 homeless 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff95c03e550 0x7ff95c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97b7fe640 10 monclient: get_auth_request con 0x7ff95c03e550 auth_method 0 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 10 monclient(hunting): handle_auth_done global_id 54613 payload 824 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff97c0a8d60 con 0x7ff97c13b910 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 0x7ff97c13bcf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c00cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.54539.objecter ms_handle_connect 0x7ff97c13b910 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.54539.objecter resend_mon_ops 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff96c015070 con 0x7ff97c13b910 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:39:40.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: dump: 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.700 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 client.54539.objecter ms_handle_connect 0x7ff95c03e550 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff97b7fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff95c03e550 0x7ff95c040a10 secure :-1 s=READY pgs=3565 cs=0 l=1 rev1=1 crypto rx=0x7ff9740af000 tx=0x7ff9740a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff96c007690 con 0x7ff97c13b910 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 msgr2=0x7ff97c13bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 0x7ff97c13bcf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c00cbe0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 msgr2=0x7ff97c13bcf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.664+0000 7ff9737fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13b910 0x7ff97c13bcf0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c00cbe0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff98175c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:40.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff98175c640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 0x7ff97c09da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff98175c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c13e5f0 con 0x7ff97c13c230 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 0x7ff97c09da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient: get_auth_request con 0x7ff97c13c230 auth_method 0 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 10 monclient(hunting): handle_auth_done global_id 54622 payload 824 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 1 -- 172.21.15.105:0/3580017605 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff97c1399e0 con 0x7ff97c13c230 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 10 client.54539.objecter ms_handle_connect 0x7ff97c13c230 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 10 client.54539.objecter resend_mon_ops 2024-09-17T16:39:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff97affd640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 0x7ff97c09da00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff96c028020 con 0x7ff97c13c230 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.672+0000 7ff9737fe640 10 monclient: dump: 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.706 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.707 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff96c0113f0 con 0x7ff97c13c230 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 msgr2=0x7ff97c09da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 0x7ff97c09da00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c004760 comp rx=0 tx=0).stop 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 msgr2=0x7ff97c09da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff9737fe640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff97c13c230 0x7ff97c09da00 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c720 tx=0x7ff96c004760 comp rx=0 tx=0).stop 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 287681, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:40.709 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.710 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.711 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:40.712 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.713 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.714 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.715 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.716 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.718 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.719 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.721 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.724 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.727 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.729 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.731 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:40.732 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 10 librados: watch_flush enter 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 10 librados: watch_flush exit 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 20 client.54539.objecter shutdown clearing up homeless session... 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 10 client.54539.objecter successfully canceled tick 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff95c03e550 msgr2=0x7ff95c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff95c03e550 0x7ff95c040a10 secure :-1 s=READY pgs=3565 cs=0 l=1 rev1=1 crypto rx=0x7ff9740af000 tx=0x7ff9740a7070 comp rx=0 tx=0).stop 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 10 monclient: shutdown 2024-09-17T16:39:40.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 msgr2=0x7ff97c1d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 --2- 172.21.15.105:0/3580017605 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff97c0a6180 0x7ff97c1d1b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e880 tx=0x7ff96400ed40 comp rx=0 tx=0).stop 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff97b7fe640 1 -- 172.21.15.105:0/3580017605 reap_dead start 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff97b7fe640 1 -- 172.21.15.105:0/3580017605 reap_dead start 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 shutdown_connections 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 >> 172.21.15.105:0/3580017605 conn(0x7ff97c0aa450 msgr2=0x7ff97c140320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 shutdown_connections 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 -- 172.21.15.105:0/3580017605 wait complete. 2024-09-17T16:39:40.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.676+0000 7ff98175c640 1 librados: shutdown 2024-09-17T16:39:40.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:40.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:39:40.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: get_monmap_and_config 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: monmap: 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:40.851962+0000 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:40.851962+0000 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:40.857 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 Processor -- start 2024-09-17T16:39:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 -- start start 2024-09-17T16:39:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: init 2024-09-17T16:39:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient: _add_conns ranks=[7,3,4,10,8,2,6,9,5,1,0] 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe83c0a4750 0x7fe83c0a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-i con 0x7fe83c0a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c0a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-g con 0x7fe83c0a4b30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-c con 0x7fe83c136500 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c09de80 con 0x7fe83c0a4b30 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe83c09e850 con 0x7fe83c136500 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe83c09f570 con 0x7fe83c0a4750 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe841f37640 10 monclient(hunting): authenticate will time out at 1664.117853s 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:33590/0 (socket says 172.21.15.105:33590) 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 -- 172.21.15.105:0/1702437771 learned_addr learned my addr 172.21.15.105:0/1702437771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): get_auth_request con 0x7fe83c136500 auth_method 0 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe83c0a4750 0x7fe83c0a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83affd640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c0a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request con 0x7fe83c0a4750 auth_method 0 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83affd640 10 monclient(hunting): get_auth_request con 0x7fe83c0a4b30 auth_method 0 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient(hunting): handle_auth_done global_id 74227 payload 274 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 -- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe83c0a4750 msgr2=0x7fe83c0a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe83c0a4750 0x7fe83c0a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 -- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 msgr2=0x7fe83c0a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c0a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 monclient: found mon.noname-c 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.848+0000 7fe83bfff640 1 -- 172.21.15.105:0/1702437771 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c09f850 con 0x7fe83c136500 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83bfff640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ee0 tx=0x7fe83401c570 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1e5f0da46b757fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 1 -- 172.21.15.105:0/1702437771 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8340090d0 con 0x7fe83c136500 2024-09-17T16:39:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: dump: 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 1 -- 172.21.15.105:0/1702437771 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8340ac3f0 con 0x7fe83c136500 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 5 monclient: authenticate success, global_id 74227 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 1 -- 172.21.15.105:0/1702437771 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8340aca50 con 0x7fe83c136500 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 4 set_mon_vals no callback set 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe83a7fc640 10 monclient: dump: 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.868 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.868 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.868 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.868 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 10 monclient: shutdown 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- 172.21.15.105:0/1702437771 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 msgr2=0x7fe83c136960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ee0 tx=0x7fe83401c570 comp rx=0 tx=0).stop 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- 172.21.15.105:0/1702437771 shutdown_connections 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe83c136500 0x7fe83c136960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c0a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 --2- 172.21.15.105:0/1702437771 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe83c0a4750 0x7fe83c0a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- 172.21.15.105:0/1702437771 >> 172.21.15.105:0/1702437771 conn(0x7fe83c0a93d0 msgr2=0x7fe83c0a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- 172.21.15.105:0/1702437771 shutdown_connections 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- 172.21.15.105:0/1702437771 wait complete. 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 10 monclient: build_initial_monmap 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 10 monclient: monmap: 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:40.878 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 librados: starting msgr at 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 librados: starting objecter 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 Processor -- start 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 -- start start 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 librados: setting wanted keys 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 1 librados: calling monclient init 2024-09-17T16:39:40.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.852+0000 7fe841f37640 10 monclient: init 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _add_conns ranks=[5,3,0,1,4,6,2,8,7] 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe83c0a4750 0x7fe83c134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-h con 0x7fe83c0a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-g con 0x7fe83c0a4b30 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 0x7fe83c13a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): picked mon.noname-a con 0x7fe83c136500 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c142180 con 0x7fe83c136500 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c142300 con 0x7fe83c0a4b30 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe83c142480 con 0x7fe83c0a4750 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): authenticate will time out at 1664.124201s 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 0x7fe83c13a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 0x7fe83c13a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39058/0 (socket says 172.21.15.105:39058) 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 learned_addr learned my addr 172.21.15.105:0/2543075789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): get_auth_request con 0x7fe83c136500 auth_method 0 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe83c0a4750 0x7fe83c134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request con 0x7fe83c0a4750 auth_method 0 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): get_auth_request con 0x7fe83c0a4b30 auth_method 0 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient(hunting): handle_auth_done global_id 36018 payload 824 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe83c0a4750 msgr2=0x7fe83c134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe83c0a4750 0x7fe83c134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 msgr2=0x7fe83c135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c0a4b30 0x7fe83c135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 monclient: found mon.noname-a 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:40.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c120ad0 con 0x7fe83c136500 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83bfff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 0x7fe83c13a360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ae040 tx=0x7fe834019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.?.objecter ms_handle_connect 0x7fe83c136500 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8340bb020 con 0x7fe83c136500 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: dump: 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.884 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: _finish_auth 0 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: _check_auth_tickets 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe834003e10 con 0x7fe83c136500 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 5 monclient: authenticate success, global_id 36018 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _renew_subs 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c0a1f70 con 0x7fe83c136500 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe834009190 con 0x7fe83c136500 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: dump: 2024-09-17T16:39:40.886 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 client.36018.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _renew_subs 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe839ffb640 4 set_mon_vals no callback set 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1207f0 con 0x7fe83c136500 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 librados: init done 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 0x7fe83c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c06cf90 con 0x7fe83c13b060 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe8340be020 con 0x7fe83c136500 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 0x7fe83c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe82803e580 0x7fe828040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient: get_auth_request con 0x7fe83c13b060 auth_method 0 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7fe8340b2040 con 0x7fe83c136500 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.36018.objecter ms_dispatch 0x7fe83c000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 3 client.36018.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:40.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 3 client.36018.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 20 client.36018.objecter dump_active .. 0 homeless 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe82803e580 0x7fe828040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 10 monclient: get_auth_request con 0x7fe82803e580 auth_method 0 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_done global_id 54615 payload 824 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe83c13c550 con 0x7fe83c13b060 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.36018.objecter ms_handle_connect 0x7fe83c13b060 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.36018.objecter resend_mon_ops 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 0x7fe83c02d450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe82c010780 tx=0x7fe82c010c40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c018020 con 0x7fe83c13b060 2024-09-17T16:39:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 monclient: dump: 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe823fff640 10 client.36018.objecter ms_handle_connect 0x7fe82803e580 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.856+0000 7fe83affd640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe82803e580 0x7fe828040a40 secure :-1 s=READY pgs=3566 cs=0 l=1 rev1=1 crypto rx=0x7fe83c11f520 tx=0x7fe830008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe82c004240 con 0x7fe83c13b060 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 msgr2=0x7fe83c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 0x7fe83c02d450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe82c010780 tx=0x7fe82c010c40 comp rx=0 tx=0).stop 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 msgr2=0x7fe83c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.860+0000 7fe823fff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b060 0x7fe83c02d450 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe82c010780 tx=0x7fe82c010c40 comp rx=0 tx=0).stop 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe841f37640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe841f37640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 0x7fe83c02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe841f37640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c051e80 con 0x7fe83c13b440 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 0x7fe83c02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient: get_auth_request con 0x7fe83c13b440 auth_method 0 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.864+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_done global_id 54624 payload 824 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe83b7fe640 1 -- 172.21.15.105:0/2543075789 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe83c13c930 con 0x7fe83c13b440 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe83b7fe640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 0x7fe83c02d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe82c019000 tx=0x7fe82c0046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 client.36018.objecter ms_handle_connect 0x7fe83c13b440 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 client.36018.objecter resend_mon_ops 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c018040 con 0x7fe83c13b440 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 monclient: dump: 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fe82c020e50 con 0x7fe83c13b440 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:40.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 msgr2=0x7fe83c02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 0x7fe83c02d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe82c019000 tx=0x7fe82c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 msgr2=0x7fe83c02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe823fff640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe83c13b440 0x7fe83c02d990 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe82c019000 tx=0x7fe82c0046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:40.896 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 287873, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:39:40.897 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.898 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.899 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:40.900 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:40.901 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.902 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:40.904 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.905 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:40.906 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:40.907 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:40.908 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.910 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:40.913 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:40.914 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:40.915 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:40.916 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.917 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:40.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.920 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:40.921 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:40.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:40.922 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:40.922 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 10 librados: watch_flush enter 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 10 librados: watch_flush exit 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 20 client.36018.objecter shutdown clearing up homeless session... 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 10 client.36018.objecter successfully canceled tick 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe82803e580 msgr2=0x7fe828040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe82803e580 0x7fe828040a40 secure :-1 s=READY pgs=3566 cs=0 l=1 rev1=1 crypto rx=0x7fe83c11f520 tx=0x7fe830008040 comp rx=0 tx=0).stop 2024-09-17T16:39:40.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 10 monclient: shutdown 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 msgr2=0x7fe83c13a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 --2- 172.21.15.105:0/2543075789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe83c136500 0x7fe83c13a360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8340ae040 tx=0x7fe834019690 comp rx=0 tx=0).stop 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 reap_dead start 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe83bfff640 1 -- 172.21.15.105:0/2543075789 reap_dead start 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 shutdown_connections 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 >> 172.21.15.105:0/2543075789 conn(0x7fe83c0a93d0 msgr2=0x7fe83c0a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 shutdown_connections 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 -- 172.21.15.105:0/2543075789 wait complete. 2024-09-17T16:39:40.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:40.868+0000 7fe841f37640 1 librados: shutdown 2024-09-17T16:39:40.926 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:40.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:39:40.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: get_monmap_and_config 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: monmap: 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.091 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:41.087934+0000 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:41.087934+0000 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.092 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.093 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.093 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:41.093 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:41.093 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 Processor -- start 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 -- start start 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: init 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient: _add_conns ranks=[6,4,9,10,8,1,2,0,3,7,5] 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-d con 0x7f8be00a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8be00a5070 0x7f8be0137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-c con 0x7f8be00a5070 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 0x7f8be0137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-a con 0x7f8be0137a50 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8be00a11e0 con 0x7f8be0137a50 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8be006c750 con 0x7f8be00a5070 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8be006c4d0 con 0x7f8be00a4750 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be7aa0640 10 monclient(hunting): authenticate will time out at 1664.353706s 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:53118/0 (socket says 172.21.15.105:53118) 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 -- 172.21.15.105:0/2891479819 learned_addr learned my addr 172.21.15.105:0/2891479819 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): get_auth_request con 0x7f8be00a4750 auth_method 0 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 1 --2- 172.21.15.105:0/2891479819 >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 0x7f8be0137eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be503b640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8be00a5070 0x7f8be0137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be503b640 10 monclient(hunting): get_auth_request con 0x7f8be00a5070 auth_method 0 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be503b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be503b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be503b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 10 monclient(hunting): get_auth_request con 0x7f8be0137a50 auth_method 0 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient(hunting): handle_auth_done global_id 24693 payload 274 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 -- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8be00a5070 msgr2=0x7f8be0137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8be00a5070 0x7f8be0137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 -- 172.21.15.105:0/2891479819 >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 msgr2=0x7f8be0137eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 --2- 172.21.15.105:0/2891479819 >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 0x7f8be0137eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 monclient: found mon.noname-d 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be583c640 1 -- 172.21.15.105:0/2891479819 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be00a08b0 con 0x7f8be00a4750 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.084+0000 7f8be603d640 1 --2- 172.21.15.105:0/2891479819 >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 0x7f8be0137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be583c640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd801c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=199a567260d9f083 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 1 -- 172.21.15.105:0/2891479819 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd80b0070 con 0x7f8be00a4750 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: dump: 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.107 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.108 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 5 monclient: authenticate success, global_id 24693 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 1 -- 172.21.15.105:0/2891479819 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd80aa810 con 0x7f8be00a4750 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 4 set_mon_vals no callback set 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 1 -- 172.21.15.105:0/2891479819 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd80a9300 con 0x7f8be00a4750 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:41.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be483a640 10 monclient: dump: 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.110 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: shutdown 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 msgr2=0x7f8be00a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd801c420 comp rx=0 tx=0).stop 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/2891479819 shutdown_connections 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/2891479819 >> v2:172.21.15.105:3300/0 conn(0x7f8be0137a50 0x7f8be0137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8be00a5070 0x7f8be0137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/2891479819 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8be00a4750 0x7f8be00a4b30 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/2891479819 >> 172.21.15.105:0/2891479819 conn(0x7f8be00a93d0 msgr2=0x7f8be00a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/2891479819 shutdown_connections 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/2891479819 wait complete. 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: monmap: 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 librados: starting msgr at 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 librados: starting objecter 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 Processor -- start 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- start start 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 librados: setting wanted keys 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 librados: calling monclient init 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: init 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _add_conns ranks=[5,2,3,6,1,8,4,0,7] 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be00a4750 0x7f8be01cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-h con 0x7f8be00a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 0x7f8be01cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-b con 0x7f8be00a5070 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8be0137a50 0x7f8be01cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): picked mon.noname-g con 0x7f8be0137a50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00a5450 con 0x7f8be00a5070 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00a02b0 con 0x7f8be0137a50 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8be0121a80 con 0x7f8be00a4750 2024-09-17T16:39:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient(hunting): authenticate will time out at 1664.358929s 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 0x7f8be01cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 0x7f8be01cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58644/0 (socket says 172.21.15.105:58644) 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 -- 172.21.15.105:0/3650460503 learned_addr learned my addr 172.21.15.105:0/3650460503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): get_auth_request con 0x7f8be00a5070 auth_method 0 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be583c640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be00a4750 0x7f8be01cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be603d640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8be0137a50 0x7f8be01cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient(hunting): handle_auth_done global_id 45092 payload 824 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be00a4750 msgr2=0x7f8be01cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be00a4750 0x7f8be01cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8be0137a50 msgr2=0x7f8be01cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8be0137a50 0x7f8be01cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 monclient: found mon.noname-b 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01214c0 con 0x7f8be00a5070 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be503b640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 0x7f8be01cf880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4007ec0 tx=0x7f8bd400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 client.?.objecter ms_handle_connect 0x7f8be00a5070 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd401a070 con 0x7f8be00a5070 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: dump: 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.117 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd4003e10 con 0x7f8be00a5070 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 5 monclient: authenticate success, global_id 45092 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd40044a0 con 0x7f8be00a5070 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bde7fc640 10 monclient: dump: 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.119 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8bdffff640 4 set_mon_vals no callback set 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _renew_subs 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be013d150 con 0x7f8be00a5070 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 client.45092.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _renew_subs 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01211e0 con 0x7f8be00a5070 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.088+0000 7f8be7aa0640 1 librados: init done 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be7aa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 0x7f8be0138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8be013d640 con 0x7f8be0138730 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8bd4004660 con 0x7f8be00a5070 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 0x7f8be0138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient: get_auth_request con 0x7f8be0138730 auth_method 0 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bcc03e550 0x7f8bcc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f8bd4050190 con 0x7f8be00a5070 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 client.45092.objecter ms_dispatch 0x7f8be0000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 3 client.45092.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 3 client.45092.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 20 client.45092.objecter dump_active .. 0 homeless 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be603d640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bcc03e550 0x7f8bcc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be603d640 10 monclient: get_auth_request con 0x7f8bcc03e550 auth_method 0 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 10 monclient(hunting): handle_auth_done global_id 74201 payload 824 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8be013ccb0 con 0x7f8be0138730 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 client.45092.objecter ms_handle_connect 0x7f8bcc03e550 2024-09-17T16:39:41.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be603d640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bcc03e550 0x7f8bcc040a10 secure :-1 s=READY pgs=3567 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00095d0 tx=0x7f8bd0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 client.45092.objecter ms_handle_connect 0x7f8be0138730 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 client.45092.objecter resend_mon_ops 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8be583c640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 0x7f8be0138b10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8be009fd90 tx=0x7f8bd80191c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd80b0040 con 0x7f8be0138730 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.092+0000 7f8bde7fc640 10 monclient: dump: 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.123 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8bd8003620 con 0x7f8be0138730 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 msgr2=0x7f8be0138b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 0x7f8be0138b10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8be009fd90 tx=0x7f8bd80191c0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 msgr2=0x7f8be0138b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.096+0000 7f8bde7fc640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0138730 0x7f8be0138b10 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8be009fd90 tx=0x7f8bd80191c0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be7aa0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:41.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 0x7f8be011f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be7aa0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8be00a55d0 con 0x7f8be0139050 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 0x7f8be011f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient: get_auth_request con 0x7f8be0139050 auth_method 0 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 10 monclient(hunting): handle_auth_done global_id 74210 payload 824 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 1 -- 172.21.15.105:0/3650460503 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8be0137510 con 0x7f8be0139050 2024-09-17T16:39:41.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8be583c640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 0x7f8be011f390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd80a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 10 client.45092.objecter ms_handle_connect 0x7f8be0139050 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 10 client.45092.objecter resend_mon_ops 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bd80bb040 con 0x7f8be0139050 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.100+0000 7f8bde7fc640 10 monclient: dump: 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.128 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f8bd80bc8e0 con 0x7f8be0139050 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:41.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 msgr2=0x7f8be011f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 0x7f8be011f390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd80a8000 comp rx=0 tx=0).stop 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 msgr2=0x7f8be011f390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8bde7fc640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8be0139050 0x7f8be011f390 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd80a8000 comp rx=0 tx=0).stop 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 20571, 2024-09-17T16:39:41.130 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:41.131 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.132 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.133 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:41.134 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.135 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:41.136 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:41.139 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:41.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.141 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.143 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.144 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:41.153 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:41.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.154 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:41.154 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 10 librados: watch_flush enter 2024-09-17T16:39:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 10 librados: watch_flush exit 2024-09-17T16:39:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 20 client.45092.objecter shutdown clearing up homeless session... 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 10 client.45092.objecter successfully canceled tick 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bcc03e550 msgr2=0x7f8bcc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bcc03e550 0x7f8bcc040a10 secure :-1 s=READY pgs=3567 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00095d0 tx=0x7f8bd0008040 comp rx=0 tx=0).stop 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 10 monclient: shutdown 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 msgr2=0x7f8be01cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 --2- 172.21.15.105:0/3650460503 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8be00a5070 0x7f8be01cf880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4007ec0 tx=0x7f8bd400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be603d640 1 -- 172.21.15.105:0/3650460503 reap_dead start 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be603d640 1 -- 172.21.15.105:0/3650460503 reap_dead start 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 shutdown_connections 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 >> 172.21.15.105:0/3650460503 conn(0x7f8be00a93d0 msgr2=0x7f8be00a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 shutdown_connections 2024-09-17T16:39:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 -- 172.21.15.105:0/3650460503 wait complete. 2024-09-17T16:39:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.104+0000 7f8be7aa0640 1 librados: shutdown 2024-09-17T16:39:41.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:41.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:39:41.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:39:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: get_monmap_and_config 2024-09-17T16:39:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: monmap: 2024-09-17T16:39:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:41.272046+0000 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:41.272046+0000 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:41.277 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 Processor -- start 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 -- start start 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: init 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient: _add_conns ranks=[3,2,6,7,8,9,1,10,5,0,4] 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9900a68e0 0x7ff9900a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-g con 0x7ff9900a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-b con 0x7ff9900a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 0x7ff990139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-d con 0x7ff9901396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900a3fc0 con 0x7ff9900a4910 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff9900a63c0 con 0x7ff9900a68e0 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff9900a7740 con 0x7ff9901396a0 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff994ab0640 10 monclient(hunting): authenticate will time out at 1664.537678s 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 0x7ff990139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58652/0 (socket says 172.21.15.105:58652) 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): get_auth_request con 0x7ff9901396a0 auth_method 0 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 -- 172.21.15.105:0/1553887546 learned_addr learned my addr 172.21.15.105:0/1553887546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request con 0x7ff9900a4910 auth_method 0 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 -- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 msgr2=0x7ff9900a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 -- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 msgr2=0x7ff9900a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98e59c640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9900a68e0 0x7ff9900a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient(hunting): handle_auth_done global_id 24702 payload 274 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 -- 172.21.15.105:0/1553887546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9900a68e0 msgr2=0x7ff9900a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9900a68e0 0x7ff9900a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 -- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 msgr2=0x7ff9900a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 monclient: found mon.noname-d 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:39:41.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.268+0000 7ff98ed9d640 1 -- 172.21.15.105:0/1553887546 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99006ec10 con 0x7ff9901396a0 2024-09-17T16:39:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 0x7ff990139ae0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff980009820 tx=0x7ff98001cde0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3f9ccb5a1f62dc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 1 -- 172.21.15.105:0/1553887546 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9800b0070 con 0x7ff9901396a0 2024-09-17T16:39:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: dump: 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 1 -- 172.21.15.105:0/1553887546 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9800aa810 con 0x7ff9901396a0 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 1 -- 172.21.15.105:0/1553887546 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9800a9300 con 0x7ff9901396a0 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98d59a640 10 monclient: dump: 2024-09-17T16:39:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.292 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 5 monclient: authenticate success, global_id 24702 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 4 set_mon_vals no callback set 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: shutdown 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 msgr2=0x7ff990139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 0x7ff990139ae0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff980009820 tx=0x7ff98001cde0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- 172.21.15.105:0/1553887546 shutdown_connections 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9901396a0 0x7ff990139ae0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9900a4910 0x7ff9900a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- 172.21.15.105:0/1553887546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9900a68e0 0x7ff9900a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- 172.21.15.105:0/1553887546 >> 172.21.15.105:0/1553887546 conn(0x7ff990068ba0 msgr2=0x7ff990068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- 172.21.15.105:0/1553887546 shutdown_connections 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- 172.21.15.105:0/1553887546 wait complete. 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: monmap: 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 librados: starting msgr at 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 librados: starting objecter 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 Processor -- start 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- start start 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 librados: setting wanted keys 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 librados: calling monclient init 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: init 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient: _add_conns ranks=[4,0,3,7,5,6,1,2,8] 2024-09-17T16:39:41.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-c con 0x7ff9900a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 0x7ff9901d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-a con 0x7ff9900a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9901396a0 0x7ff99013d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): picked mon.noname-g con 0x7ff9901396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff990143320 con 0x7ff9900a68e0 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff99006e630 con 0x7ff9901396a0 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff9901434a0 con 0x7ff9900a4910 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff994ab0640 10 monclient(hunting): authenticate will time out at 1664.543564s 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 0x7ff9901d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 0x7ff9901d1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39086/0 (socket says 172.21.15.105:39086) 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:33602/0 (socket says 172.21.15.105:33602) 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 1 -- 172.21.15.105:0/727930111 learned_addr learned my addr 172.21.15.105:0/727930111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request con 0x7ff9900a68e0 auth_method 0 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): get_auth_request con 0x7ff9900a4910 auth_method 0 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.272+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9901396a0 0x7ff99013d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 10 monclient(hunting): handle_auth_done global_id 74245 payload 824 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9901396a0 msgr2=0x7ff99013d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9901396a0 0x7ff99013d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 msgr2=0x7ff9901d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 0x7ff9901d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 monclient: found mon.noname-c 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901233e0 con 0x7ff9900a4910 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98ed9d640 10 monclient: get_auth_request con 0x7ff9901396a0 auth_method 0 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9901396a0 0x7ff99013d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9900a68e0 0x7ff9901d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98e59c640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cd90 tx=0x7ff98800a660 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.?.objecter ms_handle_connect 0x7ff9900a4910 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff98800e860 con 0x7ff9900a4910 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: dump: 2024-09-17T16:39:41.299 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.300 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98800ea20 con 0x7ff9900a4910 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff98800ebd0 con 0x7ff9900a4910 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 5 monclient: authenticate success, global_id 74245 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 monclient: _renew_subs 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9901236c0 con 0x7ff9900a4910 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98cd99640 4 set_mon_vals no callback set 2024-09-17T16:39:41.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: dump: 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.302 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 client.74245.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 monclient: _renew_subs 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901228c0 con 0x7ff9900a4910 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 1 librados: init done 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 0x7ff99013de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff990051e80 con 0x7ff99013da70 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff98800e3f0 con 0x7ff9900a4910 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 0x7ff99013de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient: get_auth_request con 0x7ff99013da70 auth_method 0 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff97403e550 0x7ff974040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7ff9880509d0 con 0x7ff9900a4910 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.74245.objecter ms_dispatch 0x7ff990000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 3 client.74245.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 3 client.74245.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 20 client.74245.objecter dump_active .. 0 homeless 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff97403e550 0x7ff974040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98ed9d640 10 monclient: get_auth_request con 0x7ff97403e550 auth_method 0 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_done global_id 64357 payload 824 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9901226c0 con 0x7ff99013da70 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.74245.objecter ms_handle_connect 0x7ff99013da70 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.74245.objecter resend_mon_ops 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 0x7ff99013de50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9780119d0 tx=0x7ff978011e90 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff978019070 con 0x7ff99013da70 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: dump: 2024-09-17T16:39:41.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.306 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 client.74245.objecter ms_handle_connect 0x7ff97403e550 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff98ed9d640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff97403e550 0x7ff974040a10 secure :-1 s=READY pgs=3568 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ad000 tx=0x7ff980004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff97800fb10 con 0x7ff99013da70 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 msgr2=0x7ff99013de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 0x7ff99013de50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9780119d0 tx=0x7ff978011e90 comp rx=0 tx=0).stop 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 msgr2=0x7ff99013de50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.276+0000 7ff986ffd640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013da70 0x7ff99013de50 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9780119d0 tx=0x7ff978011e90 comp rx=0 tx=0).stop 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff994ab0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff994ab0640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 0x7ff9900a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff994ab0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff990139ae0 con 0x7ff99013e390 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 0x7ff9900a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient: get_auth_request con 0x7ff99013e390 auth_method 0 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 10 monclient(hunting): handle_auth_done global_id 64366 payload 824 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 1 -- 172.21.15.105:0/727930111 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff990123900 con 0x7ff99013e390 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff98dd9b640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 0x7ff9900a0a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9780119d0 tx=0x7ff9780046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 10 client.74245.objecter ms_handle_connect 0x7ff99013e390 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 10 client.74245.objecter resend_mon_ops 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff97800f040 con 0x7ff99013e390 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.284+0000 7ff986ffd640 10 monclient: dump: 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.310 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff9780283f0 con 0x7ff99013e390 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 msgr2=0x7ff9900a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 0x7ff9900a0a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff9780119d0 tx=0x7ff9780046d0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 msgr2=0x7ff9900a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff986ffd640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff99013e390 0x7ff9900a0a20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:41.312 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 74, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 154386, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.314 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.315 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:39:41.316 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:39:41.317 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.318 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.319 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:39:41.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:39:41.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.324 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:39:41.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.329 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:39:41.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:39:41.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:41.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:39:41.334 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 10 librados: watch_flush enter 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 10 librados: watch_flush exit 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 20 client.74245.objecter shutdown clearing up homeless session... 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 10 client.74245.objecter successfully canceled tick 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff97403e550 msgr2=0x7ff974040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff97403e550 0x7ff974040a10 secure :-1 s=READY pgs=3568 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ad000 tx=0x7ff980004740 comp rx=0 tx=0).stop 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 10 monclient: shutdown 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 msgr2=0x7ff9901d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff98800cd90 tx=0x7ff98800a660 comp rx=0 tx=0).stop 2024-09-17T16:39:41.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff98ed9d640 1 -- 172.21.15.105:0/727930111 reap_dead start 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 shutdown_connections 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 --2- 172.21.15.105:0/727930111 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9900a4910 0x7ff9901d1680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 >> 172.21.15.105:0/727930111 conn(0x7ff990068ba0 msgr2=0x7ff9900a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 shutdown_connections 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 -- 172.21.15.105:0/727930111 wait complete. 2024-09-17T16:39:41.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.288+0000 7ff994ab0640 1 librados: shutdown 2024-09-17T16:39:41.337 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:39:41.337 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'b'] 2024-09-17T16:39:41.338 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-17T16:39:41.338 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-17T16:39:41.338 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-17T16:39:41.338 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-17T16:39:41.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:39:41.362 INFO:tasks.ceph.mon.f:Stopped 2024-09-17T16:39:41.362 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-17T16:39:41.362 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-09-17T16:39:41.363 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-09-17T16:39:41.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: get_monmap_and_config 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: monmap: 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:41.485301+0000 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:41.485301+0000 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.489 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:41.490 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 Processor -- start 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 -- start start 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: init 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient: _add_conns ranks=[7,10,1,4,5,9,0,8,2,6,3] 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): picked mon.noname-i con 0x7f90080a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 0x7f90080a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): picked mon.noname-f con 0x7f90080a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f90081396a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900806ebc0 con 0x7f90081396a0 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f90080a3fc0 con 0x7f90080a4910 2024-09-17T16:39:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f90080a63c0 con 0x7f90080a68e0 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900f256640 10 monclient(hunting): authenticate will time out at 1664.751053s 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900d7f3640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f90081396a0 legacy=0x7f9008139b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f9007fff640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 msgr2=0x7f90080a7200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f9007fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 0x7f90080a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:34826/0 (socket says 172.21.15.105:34826) 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 1 -- 172.21.15.105:0/1139615414 learned_addr learned my addr 172.21.15.105:0/1139615414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): get_auth_request con 0x7f90080a68e0 auth_method 0 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 10 monclient(hunting): handle_auth_done global_id 64375 payload 274 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 1 -- 172.21.15.105:0/1139615414 >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 msgr2=0x7f90080a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.480+0000 7f900cff2640 1 --2- 172.21.15.105:0/1139615414 >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 0x7f90080a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 1 -- 172.21.15.105:0/1139615414 >> v1:172.21.15.160:6789/0 conn(0x7f90081396a0 legacy=0x7f9008139b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 1 monclient: found mon.noname-i 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 1 -- 172.21.15.105:0/1139615414 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90080a3690 con 0x7f90080a68e0 2024-09-17T16:39:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900cff2640 1 --2- 172.21.15.105:0/1139615414 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009870 tx=0x7f8ff801c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=5c005c21c7416ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 1 -- 172.21.15.105:0/1139615414 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ff80b0070 con 0x7f90080a68e0 2024-09-17T16:39:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: dump: 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 1 -- 172.21.15.105:0/1139615414 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80aa810 con 0x7f90080a68e0 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 1 -- 172.21.15.105:0/1139615414 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ff80aa3a0 con 0x7f90080a68e0 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f90077fe640 10 monclient: dump: 2024-09-17T16:39:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.505 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 5 monclient: authenticate success, global_id 64375 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 4 set_mon_vals no callback set 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 10 monclient: shutdown 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 -- 172.21.15.105:0/1139615414 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 msgr2=0x7f90080a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 --2- 172.21.15.105:0/1139615414 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009870 tx=0x7f8ff801c440 comp rx=0 tx=0).stop 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 -- 172.21.15.105:0/1139615414 shutdown_connections 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 --2- 172.21.15.105:0/1139615414 >> v2:172.21.15.160:3300/0 conn(0x7f90080a4910 0x7f90080a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 --2- 172.21.15.105:0/1139615414 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90080a68e0 0x7f90080a6cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 -- 172.21.15.105:0/1139615414 >> 172.21.15.105:0/1139615414 conn(0x7f9008068ba0 msgr2=0x7f9008068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 -- 172.21.15.105:0/1139615414 shutdown_connections 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.484+0000 7f900f256640 1 -- 172.21.15.105:0/1139615414 wait complete. 2024-09-17T16:39:41.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: monmap: 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.507 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 librados: starting msgr at 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 librados: starting objecter 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 Processor -- start 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- start start 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 librados: setting wanted keys 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 librados: calling monclient init 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: init 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _add_conns ranks=[2,1,4,7,3,0,5,6,8] 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 0x7f90081d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): picked mon.noname-b con 0x7f90080a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90080a68e0 0x7f90081d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): picked mon.noname-f con 0x7f90080a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f90081396a0 0x7f90081d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): picked mon.noname-c con 0x7f90081396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f90081444b0 con 0x7f90080a68e0 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9008144330 con 0x7f90080a4910 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9008144630 con 0x7f90081396a0 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): authenticate will time out at 1664.757021s 2024-09-17T16:39:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 0x7f90081d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90080a68e0 msgr2=0x7f90081d1a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90080a68e0 0x7f90081d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 0x7f90081d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58656/0 (socket says 172.21.15.105:58656) 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f90081396a0 0x7f90081d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 -- 172.21.15.105:0/3307692297 learned_addr learned my addr 172.21.15.105:0/3307692297 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): get_auth_request con 0x7f90080a4910 auth_method 0 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): get_auth_request con 0x7f90081396a0 auth_method 0 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient(hunting): handle_auth_done global_id 45110 payload 824 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f90081396a0 msgr2=0x7f90081d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f90081396a0 0x7f90081d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90080a68e0 msgr2=0x7f90081d1a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90080a68e0 0x7f90081d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 monclient: found mon.noname-b 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900813eec0 con 0x7f90080a4910 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f90081396a0 0x7f90081d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900cff2640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 0x7f90081d1540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80ae040 tx=0x7f8ff80191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 client.?.objecter ms_handle_connect 0x7f90080a4910 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ff80b0040 con 0x7f90080a4910 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: dump: 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80044b0 con 0x7f90080a4910 2024-09-17T16:39:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 5 monclient: authenticate success, global_id 45110 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _renew_subs 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90080a9e70 con 0x7f90080a4910 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ff8004040 con 0x7f90080a4910 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 client.45110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: dump: 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _renew_subs 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9006ffd640 4 set_mon_vals no callback set 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90081403b0 con 0x7f90080a4910 2024-09-17T16:39:41.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 librados: init done 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 0x7f900802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9008051e80 con 0x7f900813dbe0 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8ff8003620 con 0x7f90080a4910 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 0x7f900802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient: get_auth_request con 0x7f900813dbe0 auth_method 0 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8ff403e550 0x7f8ff4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f8ff80e9330 con 0x7f90080a4910 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 client.45110.objecter ms_dispatch 0x7f9008000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 3 client.45110.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 3 client.45110.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 10 monclient(hunting): handle_auth_done global_id 45119 payload 824 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90080a8830 con 0x7f900813dbe0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 20 client.45110.objecter dump_active .. 0 homeless 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 client.45110.objecter ms_handle_connect 0x7f900813dbe0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 client.45110.objecter resend_mon_ops 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f9007fff640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 0x7f900802d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f900813eb80 tx=0x7f8ffc00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8ff403e550 0x7f8ff4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f900d7f3640 10 monclient: get_auth_request con 0x7f8ff403e550 auth_method 0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ffc016070 con 0x7f900813dbe0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.488+0000 7f90057fa640 10 monclient: dump: 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8ffc003ab0 con 0x7f900813dbe0 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 msgr2=0x7f900802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 0x7f900802d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f900813eb80 tx=0x7f8ffc00b810 comp rx=0 tx=0).stop 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 msgr2=0x7f900802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dbe0 0x7f900802d450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f900d7f3640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8ff403e550 0x7f8ff4040a10 secure :-1 s=READY pgs=3569 cs=0 l=1 rev1=1 crypto rx=0x7f9000003bd0 tx=0x7f900000d040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.492+0000 7f90057fa640 10 client.45110.objecter ms_handle_connect 0x7f8ff403e550 2024-09-17T16:39:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f900f256640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:39:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f900f256640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 0x7f900802d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f900f256640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9008139b50 con 0x7f900813dfc0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 0x7f900802d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient: get_auth_request con 0x7f900813dfc0 auth_method 0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 10 monclient(hunting): handle_auth_done global_id 45128 payload 824 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 1 -- 172.21.15.105:0/3307692297 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f90080a1120 con 0x7f900813dfc0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 10 client.45110.objecter ms_handle_connect 0x7f900813dfc0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f9007fff640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 0x7f900802d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000eb0 tx=0x7f8ffc005920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 10 client.45110.objecter resend_mon_ops 2024-09-17T16:39:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ffc016040 con 0x7f900813dfc0 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.496+0000 7f90057fa640 10 monclient: dump: 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f8ffc018920 con 0x7f900813dfc0 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 msgr2=0x7f900802d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 0x7f900802d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000eb0 tx=0x7f8ffc005920 comp rx=0 tx=0).stop 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 msgr2=0x7f900802d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:39:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f90057fa640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f900813dfc0 0x7f900802d990 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000eb0 tx=0x7f8ffc005920 comp rx=0 tx=0).stop 2024-09-17T16:39:41.525 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:39:41.525 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:39:41.525 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:39:41.525 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 10 librados: watch_flush enter 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 10 librados: watch_flush exit 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 20 client.45110.objecter shutdown clearing up homeless session... 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 10 client.45110.objecter successfully canceled tick 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8ff403e550 msgr2=0x7f8ff4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8ff403e550 0x7f8ff4040a10 secure :-1 s=READY pgs=3569 cs=0 l=1 rev1=1 crypto rx=0x7f9000003bd0 tx=0x7f900000d040 comp rx=0 tx=0).stop 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 10 monclient: shutdown 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 msgr2=0x7f90081d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 --2- 172.21.15.105:0/3307692297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90080a4910 0x7f90081d1540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80ae040 tx=0x7f8ff80191e0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900d7f3640 1 -- 172.21.15.105:0/3307692297 reap_dead start 2024-09-17T16:39:41.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900d7f3640 1 -- 172.21.15.105:0/3307692297 reap_dead start 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 shutdown_connections 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 >> 172.21.15.105:0/3307692297 conn(0x7f9008068ba0 msgr2=0x7f90081413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 shutdown_connections 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 -- 172.21.15.105:0/3307692297 wait complete. 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.500+0000 7f900f256640 1 librados: shutdown 2024-09-17T16:39:41.528 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-17T16:39:41.528 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T16:39:41.528 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:39:41.544 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T16:39:41.544 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-17T16:39:41.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:41.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: get_monmap_and_config 2024-09-17T16:39:41.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: monmap: 2024-09-17T16:39:41.700 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:41.697060+0000 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:41.697060+0000 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.702 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:41.702 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:41.702 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 Processor -- start 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 -- start start 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: init 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient: _add_conns ranks=[8,7,6,4,0,2,10,3,1,9,5] 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 0x7f49440a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): picked mon.noname-e con 0x7f49440a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f49440a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): picked mon.noname-i con 0x7f49440a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): picked mon.noname-d con 0x7f49441396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f49440a3fc0 con 0x7f49441396a0 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f49440a63c0 con 0x7f49440a4910 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f49440a7740 con 0x7f49440a68e0 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f494934b640 10 monclient(hunting): authenticate will time out at 1664.963281s 2024-09-17T16:39:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 0x7f49440a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:53146/0 (socket says 172.21.15.105:53146) 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 -- 172.21.15.105:0/26955630 learned_addr learned my addr 172.21.15.105:0/26955630 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): get_auth_request con 0x7f49440a68e0 auth_method 0 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): get_auth_request con 0x7f49441396a0 auth_method 0 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49427fc640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f49440a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49427fc640 10 monclient(hunting): get_auth_request con 0x7f49440a4910 auth_method 0 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 -- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 msgr2=0x7f4944139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49427fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 -- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 msgr2=0x7f4944139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f49437fe640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient(hunting): handle_auth_done global_id 54548 payload 274 2024-09-17T16:39:41.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 -- 172.21.15.105:0/26955630 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 msgr2=0x7f49440a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f49440a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 -- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 msgr2=0x7f4944139ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 monclient: found mon.noname-e 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:39:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.692+0000 7f4942ffd640 1 -- 172.21.15.105:0/26955630 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494406ec10 con 0x7f49440a68e0 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4942ffd640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 0x7f49440a6cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4934009df0 tx=0x7f493401d2b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e211c34b70158eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 1 -- 172.21.15.105:0/26955630 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49340b2070 con 0x7f49440a68e0 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: dump: 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 1 -- 172.21.15.105:0/26955630 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49340a59b0 con 0x7f49440a68e0 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 5 monclient: authenticate success, global_id 54548 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 1 -- 172.21.15.105:0/26955630 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49340a5be0 con 0x7f49440a68e0 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 4 set_mon_vals no callback set 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f4941ffb640 10 monclient: dump: 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 10 monclient: shutdown 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 msgr2=0x7f49440a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 0x7f49440a6cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4934009df0 tx=0x7f493401d2b0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- 172.21.15.105:0/26955630 shutdown_connections 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f49441396a0 0x7f4944139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f49440a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 --2- 172.21.15.105:0/26955630 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f49440a68e0 0x7f49440a6cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- 172.21.15.105:0/26955630 >> 172.21.15.105:0/26955630 conn(0x7f4944068ba0 msgr2=0x7f4944068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- 172.21.15.105:0/26955630 shutdown_connections 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- 172.21.15.105:0/26955630 wait complete. 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 10 monclient: build_initial_monmap 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 10 monclient: monmap: 2024-09-17T16:39:41.718 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:41.719 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 librados: starting msgr at 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 librados: starting objecter 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 Processor -- start 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 -- start start 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 librados: setting wanted keys 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 1 librados: calling monclient init 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.696+0000 7f494934b640 10 monclient: init 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _add_conns ranks=[7,2,1,5,8,0,6,3,4] 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): picked mon.noname-i con 0x7f49440a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 0x7f4944138460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): picked mon.noname-b con 0x7f49440a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:41.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 0x7f494413d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): picked mon.noname-f con 0x7f49441396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4944143320 con 0x7f49441396a0 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f494406e630 con 0x7f49440a68e0 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f49441434a0 con 0x7f49440a4910 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient(hunting): authenticate will time out at 1664.968456s 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f49427fc640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 msgr2=0x7f4944138460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f49427fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 0x7f4944138460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f49437fe640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 msgr2=0x7f494413d7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:41.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f49437fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 0x7f494413d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:34854/0 (socket says 172.21.15.105:34854) 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 -- 172.21.15.105:0/1531356629 learned_addr learned my addr 172.21.15.105:0/1531356629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): get_auth_request con 0x7f49440a4910 auth_method 0 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient(hunting): handle_auth_done global_id 64393 payload 824 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 -- 172.21.15.105:0/1531356629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 msgr2=0x7f4944138460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 0x7f4944138460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 -- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 msgr2=0x7f494413d7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 0x7f494413d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 monclient: found mon.noname-i 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 -- 172.21.15.105:0/1531356629 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494413f5e0 con 0x7f49440a4910 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f4942ffd640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49340b0040 tx=0x7f49340195f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 client.?.objecter ms_handle_connect 0x7f49440a4910 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49340b2050 con 0x7f49440a4910 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:39:41.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: dump: 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.724 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 5 monclient: authenticate success, global_id 64393 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _renew_subs 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4944123890 con 0x7f49440a4910 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49340be070 con 0x7f49440a4910 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4934003e10 con 0x7f49440a4910 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 client.64393.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:41.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f49417fa640 4 set_mon_vals no callback set 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 monclient: dump: 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:41.726 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _renew_subs 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f49440a9de0 con 0x7f49440a4910 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 librados: init done 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49440aa230 con 0x7f49440a4910 2024-09-17T16:39:41.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f49340039a0 con 0x7f49440a4910 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 0x7f4924040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f49340eb9d0 con 0x7f49440a4910 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.700+0000 7f493b7fe640 10 client.64393.objecter ms_dispatch 0x7f4944000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f493b7fe640 3 client.64393.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f493b7fe640 3 client.64393.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f493b7fe640 20 client.64393.objecter dump_active .. 0 homeless 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f49427fc640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 0x7f4924040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f49427fc640 10 monclient: get_auth_request con 0x7f492403e580 auth_method 0 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f493b7fe640 10 client.64393.objecter ms_handle_connect 0x7f492403e580 2024-09-17T16:39:41.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.704+0000 7f49427fc640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 0x7f4924040a40 secure :-1 s=READY pgs=3570 cs=0 l=1 rev1=1 crypto rx=0x7f494413d6c0 tx=0x7f492c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:41.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.716+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f494413f5e0 con 0x7f49440a4910 2024-09-17T16:39:41.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.716+0000 7f493b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:41.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.716+0000 7f493b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:41.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4944139ae0 con 0x7f49440a4910 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f493b7fe640 1 -- 172.21.15.105:0/1531356629 <== mon.7 v2:172.21.15.160:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f4934016560 con 0x7f49440a4910 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f493b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:41.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f493b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:41.961 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 librados: watch_flush enter 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 librados: watch_flush exit 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 20 client.64393.objecter shutdown clearing up homeless session... 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 client.64393.objecter successfully canceled tick 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 msgr2=0x7f4924040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 0x7f4924040a40 secure :-1 s=READY pgs=3570 cs=0 l=1 rev1=1 crypto rx=0x7f494413d6c0 tx=0x7f492c008040 comp rx=0 tx=0).stop 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 10 monclient: shutdown 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:41.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 msgr2=0x7f4944137f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.952+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49340b0040 tx=0x7f49340195f0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 shutdown_connections 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f492403e580 0x7f4924040a40 unknown :-1 s=CLOSED pgs=3570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49440a68e0 0x7f4944138460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49441396a0 0x7f494413d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 --2- 172.21.15.105:0/1531356629 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f49440a4910 0x7f4944137f20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 >> 172.21.15.105:0/1531356629 conn(0x7f4944068ba0 msgr2=0x7f49440a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 shutdown_connections 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 -- 172.21.15.105:0/1531356629 wait complete. 2024-09-17T16:39:41.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:41.956+0000 7f494934b640 1 librados: shutdown 2024-09-17T16:39:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-17T16:39:44.981 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: get_monmap_and_config 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: build_initial_monmap 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: monmap: 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:45.105472+0000 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:45.105472+0000 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:45.109 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:45.110 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 Processor -- start 2024-09-17T16:39:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 -- start start 2024-09-17T16:39:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: init 2024-09-17T16:39:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient: _add_conns ranks=[7,0,8,6,4,5,1,10,2,9,3] 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f73040a6a30 0x7f73040a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-i con 0x7f73040a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f73040a4ab0 addr v1:172.21.15.105:6789/0 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 0x7f7304139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-e con 0x7f73040a7350 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73040a4080 con 0x7f73040a4ab0 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f73040a7c40 con 0x7f73040a6a30 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f73040a37e0 con 0x7f73040a7350 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730c4b4640 10 monclient(hunting): authenticate will time out at 1668.371219s 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f7309a4f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f73040a4ab0 0x7f73041397f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:32872/0 (socket says 172.21.15.105:32872) 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f7309a4f640 1 -- 172.21.15.105:0/2708523457 learned_addr learned my addr 172.21.15.105:0/2708523457 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 0x7f7304139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): get_auth_request con 0x7f73040a7350 auth_method 0 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730a250640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f73040a6a30 0x7f73040a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730a250640 10 monclient(hunting): get_auth_request con 0x7f73040a6a30 auth_method 0 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730a250640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730a250640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730a250640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient(hunting): handle_auth_done global_id 54557 payload 274 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 -- 172.21.15.105:0/2708523457 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f73040a6a30 msgr2=0x7f73040a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f73040a6a30 0x7f73040a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 -- 172.21.15.105:0/2708523457 >> v1:172.21.15.105:6789/0 conn(0x7f73040a4ab0 legacy=0x7f73041397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient: _finish_hunting 0 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 monclient: found mon.noname-e 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.100+0000 7f730aa51640 1 -- 172.21.15.105:0/2708523457 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73040a3ac0 con 0x7f73040a7350 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730aa51640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 0x7f7304139f00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009dc0 tx=0x7f72fc01c820 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7c872ab7a9f0492b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 1 -- 172.21.15.105:0/2708523457 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72fc0b2070 con 0x7f73040a7350 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: dump: 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: _finish_auth 0 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: _check_auth_tickets 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 1 -- 172.21.15.105:0/2708523457 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc0a5af0 con 0x7f73040a7350 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 1 -- 172.21.15.105:0/2708523457 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72fc0a5d20 con 0x7f73040a7350 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 5 monclient: authenticate success, global_id 54557 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 4 set_mon_vals no callback set 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730924e640 10 monclient: dump: 2024-09-17T16:39:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:45.127 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 10 monclient: shutdown 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 -- 172.21.15.105:0/2708523457 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 msgr2=0x7f7304139f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 0x7f7304139f00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009dc0 tx=0x7f72fc01c820 comp rx=0 tx=0).stop 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 -- 172.21.15.105:0/2708523457 shutdown_connections 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f73040a7350 0x7f7304139f00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 --2- 172.21.15.105:0/2708523457 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f73040a6a30 0x7f73040a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 -- 172.21.15.105:0/2708523457 >> 172.21.15.105:0/2708523457 conn(0x7f73040a8d00 msgr2=0x7f73040a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 -- 172.21.15.105:0/2708523457 shutdown_connections 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 1 -- 172.21.15.105:0/2708523457 wait complete. 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 10 monclient: build_initial_monmap 2024-09-17T16:39:45.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.104+0000 7f730c4b4640 10 monclient: monmap: 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 librados: starting msgr at 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 librados: starting objecter 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 Processor -- start 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- start start 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 librados: setting wanted keys 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 librados: calling monclient init 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: init 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _add_conns ranks=[3,2,4,1,8,6,5,0,7] 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73040a4ab0 0x7f7304138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-g con 0x7f73040a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 0x7f7304139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-b con 0x7f73040a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): picked mon.noname-c con 0x7f73040a7350 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f730413a440 con 0x7f73040a6a30 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f730406e770 con 0x7f73040a4ab0 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f730413a5c0 con 0x7f73040a7350 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient(hunting): authenticate will time out at 1668.376910s 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f7309a4f640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 msgr2=0x7f7304139170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f7309a4f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 0x7f7304139170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:33628/0 (socket says 172.21.15.105:33628) 2024-09-17T16:39:45.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 -- 172.21.15.105:0/1907136379 learned_addr learned my addr 172.21.15.105:0/1907136379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): get_auth_request con 0x7f73040a7350 auth_method 0 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73040a4ab0 0x7f7304138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 10 monclient(hunting): get_auth_request con 0x7f73040a4ab0 auth_method 0 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:45.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient(hunting): handle_auth_done global_id 74263 payload 824 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 -- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73040a4ab0 msgr2=0x7f7304138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73040a4ab0 0x7f7304138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 -- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 msgr2=0x7f7304139170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 0x7f7304139170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient: _finish_hunting 0 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 monclient: found mon.noname-c 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 -- 172.21.15.105:0/1907136379 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7304122d60 con 0x7f73040a7350 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730aa51640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b0040 tx=0x7f72fc0195c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 client.?.objecter ms_handle_connect 0x7f73040a7350 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72fc0b2050 con 0x7f73040a7350 2024-09-17T16:39:45.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: dump: 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.134 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:39:45.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc0be070 con 0x7f73040a7350 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72fc003e10 con 0x7f73040a7350 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 5 monclient: authenticate success, global_id 74263 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _renew_subs 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73040a2000 con 0x7f73040a7350 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f7308a4d640 4 set_mon_vals no callback set 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 monclient: dump: 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:45.136 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:45.137 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 client.74263.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _renew_subs 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f73041238f0 con 0x7f73040a7350 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 librados: init done 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:45.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73040a6e10 con 0x7f73040a7350 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f72fc0039a0 con 0x7f73040a7350 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 0x7f72e8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f72fc0eaee0 con 0x7f73040a7350 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 10 client.74263.objecter ms_dispatch 0x7f7304000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 3 client.74263.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 3 client.74263.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f72f6ffd640 20 client.74263.objecter dump_active .. 0 homeless 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 0x7f72e8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.108+0000 7f730a250640 10 monclient: get_auth_request con 0x7f72e803e550 auth_method 0 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.112+0000 7f72f6ffd640 10 client.74263.objecter ms_handle_connect 0x7f72e803e550 2024-09-17T16:39:45.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.112+0000 7f730a250640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 0x7f72e8040a10 secure :-1 s=READY pgs=3571 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b9d0 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:45.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.112+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f72fc016560 con 0x7f73040a7350 2024-09-17T16:39:45.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.112+0000 7f72f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:45.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.112+0000 7f72f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:45.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:39:45.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:45.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:39:45.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f73041407c0 con 0x7f73040a7350 2024-09-17T16:39:45.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f72f6ffd640 1 -- 172.21.15.105:0/1907136379 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f72fc0adca0 con 0x7f73040a7350 2024-09-17T16:39:45.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f72f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:45.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f72f6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:45.351 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 librados: watch_flush enter 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 librados: watch_flush exit 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 20 client.74263.objecter shutdown clearing up homeless session... 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 10 client.74263.objecter successfully canceled tick 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 msgr2=0x7f72e8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.340+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 0x7f72e8040a10 secure :-1 s=READY pgs=3571 cs=0 l=1 rev1=1 crypto rx=0x7f72f800b9d0 tx=0x7f72f8008040 comp rx=0 tx=0).stop 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 10 monclient: shutdown 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 msgr2=0x7f730413e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b0040 tx=0x7f72fc0195c0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 shutdown_connections 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f72e803e550 0x7f72e8040a10 unknown :-1 s=CLOSED pgs=3571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f73040a7350 0x7f730413e200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73040a6a30 0x7f7304139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 --2- 172.21.15.105:0/1907136379 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73040a4ab0 0x7f7304138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 >> 172.21.15.105:0/1907136379 conn(0x7f73040a8d00 msgr2=0x7f730409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 shutdown_connections 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 -- 172.21.15.105:0/1907136379 wait complete. 2024-09-17T16:39:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:45.344+0000 7f730c4b4640 1 librados: shutdown 2024-09-17T16:39:48.367 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: get_monmap_and_config 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: build_initial_monmap 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: monmap: 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:48.493386+0000 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:48.493386+0000 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:48.498 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 Processor -- start 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 -- start start 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: init 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient: _add_conns ranks=[7,2,8,10,4,3,5,6,9,1,0] 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01380a68e0 0x7f01380a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): picked mon.noname-i con 0x7f01380a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 0x7f01380a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): picked mon.noname-b con 0x7f01380a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): picked mon.noname-e con 0x7f01381396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f01380a3fc0 con 0x7f01380a4910 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f01380a63c0 con 0x7f01380a68e0 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f01380a7740 con 0x7f01381396a0 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f0140324640 10 monclient(hunting): authenticate will time out at 1671.759132s 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013d8bf640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 msgr2=0x7f01380a7200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013d8bf640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 0x7f01380a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:42402/0 (socket says 172.21.15.105:42402) 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 -- 172.21.15.105:0/2949226395 learned_addr learned my addr 172.21.15.105:0/2949226395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): get_auth_request con 0x7f01381396a0 auth_method 0 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e0c0640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01380a68e0 0x7f01380a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e0c0640 10 monclient(hunting): get_auth_request con 0x7f01380a68e0 auth_method 0 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e0c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e0c0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e0c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient(hunting): handle_auth_done global_id 54566 payload 274 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 -- 172.21.15.105:0/2949226395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01380a68e0 msgr2=0x7f01380a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01380a68e0 0x7f01380a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 -- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 msgr2=0x7f01380a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 0x7f01380a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient: _finish_hunting 0 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 monclient: found mon.noname-e 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.488+0000 7f013e8c1640 1 -- 172.21.15.105:0/2949226395 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013806ec10 con 0x7f01381396a0 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01300098d0 tx=0x7f013001cec0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f916e634386f3a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 1 -- 172.21.15.105:0/2949226395 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01300b0070 con 0x7f01381396a0 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: dump: 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: _finish_auth 0 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: _check_auth_tickets 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 1 -- 172.21.15.105:0/2949226395 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01300aa810 con 0x7f01381396a0 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 5 monclient: authenticate success, global_id 54566 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 10 monclient: get_monmap_and_config success 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 4 set_mon_vals no callback set 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 1 -- 172.21.15.105:0/2949226395 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01300a9300 con 0x7f01381396a0 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f013d0be640 10 monclient: dump: 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 10 monclient: shutdown 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 msgr2=0x7f0138139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f01300098d0 tx=0x7f013001cec0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- 172.21.15.105:0/2949226395 shutdown_connections 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01381396a0 0x7f0138139ae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a4910 0x7f01380a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 --2- 172.21.15.105:0/2949226395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f01380a68e0 0x7f01380a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- 172.21.15.105:0/2949226395 >> 172.21.15.105:0/2949226395 conn(0x7f0138068ba0 msgr2=0x7f0138068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- 172.21.15.105:0/2949226395 shutdown_connections 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- 172.21.15.105:0/2949226395 wait complete. 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 10 monclient: build_initial_monmap 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 10 monclient: monmap: 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:39:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 librados: starting msgr at 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 librados: starting objecter 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 Processor -- start 2024-09-17T16:39:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 -- start start 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 librados: setting wanted keys 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 1 librados: calling monclient init 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.492+0000 7f0140324640 10 monclient: init 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _add_conns ranks=[1,2,3,4,0,5,8,7,6] 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 0x7f013813be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): picked mon.noname-f con 0x7f01380a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 0x7f013813df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): picked mon.noname-b con 0x7f01380a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): picked mon.noname-g con 0x7f01381396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0138143320 con 0x7f01380a4910 2024-09-17T16:39:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f013806e630 con 0x7f01380a68e0 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f01381434a0 con 0x7f01381396a0 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient(hunting): authenticate will time out at 1671.764321s 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013d8bf640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 msgr2=0x7f013813df40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013d8bf640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 0x7f013813df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e0c0640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 msgr2=0x7f013813be80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e0c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 0x7f013813be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:36518/0 (socket says 172.21.15.105:36518) 2024-09-17T16:39:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 -- 172.21.15.105:0/2856982564 learned_addr learned my addr 172.21.15.105:0/2856982564 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): get_auth_request con 0x7f01381396a0 auth_method 0 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient(hunting): handle_auth_done global_id 54642 payload 824 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 -- 172.21.15.105:0/2856982564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 msgr2=0x7f013813df40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 0x7f013813df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 -- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 msgr2=0x7f013813be80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 0x7f013813be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient: _finish_hunting 0 2024-09-17T16:39:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 monclient: found mon.noname-g 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 -- 172.21.15.105:0/2856982564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01380aa100 con 0x7f01381396a0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013e8c1640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01300098d0 tx=0x7f0130019200 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 client.?.objecter ms_handle_connect 0x7f01381396a0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01300bb070 con 0x7f01381396a0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: dump: 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: _finish_auth 0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: _check_auth_tickets 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 5 monclient: authenticate success, global_id 54642 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01300044b0 con 0x7f01381396a0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _renew_subs 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01300b0040 con 0x7f01381396a0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01380a30d0 con 0x7f01381396a0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:39:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f013c8bd640 4 set_mon_vals no callback set 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 monclient: dump: 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:39:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 client.54642.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _renew_subs 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f013813f740 con 0x7f01381396a0 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 librados: init done 2024-09-17T16:39:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013813cea0 con 0x7f01381396a0 2024-09-17T16:39:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f01300aacf0 con 0x7f01381396a0 2024-09-17T16:39:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 0x7f011c040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f01300e9780 con 0x7f01381396a0 2024-09-17T16:39:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 10 client.54642.objecter ms_dispatch 0x7f0138000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 3 client.54642.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 3 client.54642.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.496+0000 7f012effd640 20 client.54642.objecter dump_active .. 0 homeless 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.500+0000 7f013e0c0640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 0x7f011c040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.500+0000 7f013e0c0640 10 monclient: get_auth_request con 0x7f011c03e480 auth_method 0 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.500+0000 7f012effd640 10 client.54642.objecter ms_handle_connect 0x7f011c03e480 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.500+0000 7f013e0c0640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 0x7f011c040940 secure :-1 s=READY pgs=3572 cs=0 l=1 rev1=1 crypto rx=0x7f0134009610 tx=0x7f01340098e0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.512+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0130016170 con 0x7f01381396a0 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.512+0000 7f012effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:39:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.512+0000 7f012effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:39:48.660 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-17T16:39:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f0140324640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:39:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f0140324640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f0140324640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:39:48.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f013813f3c0 con 0x7f01381396a0 2024-09-17T16:39:48.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f012effd640 1 -- 172.21.15.105:0/2856982564 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7f0130003620 con 0x7f01381396a0 2024-09-17T16:39:48.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f012effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:39:48.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.744+0000 7f012effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:39:48.754 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:39:48.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 10 librados: watch_flush enter 2024-09-17T16:39:48.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 10 librados: watch_flush exit 2024-09-17T16:39:48.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 20 client.54642.objecter shutdown clearing up homeless session... 2024-09-17T16:39:48.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 10 client.54642.objecter successfully canceled tick 2024-09-17T16:39:48.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 msgr2=0x7f011c040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 0x7f011c040940 secure :-1 s=READY pgs=3572 cs=0 l=1 rev1=1 crypto rx=0x7f0134009610 tx=0x7f01340098e0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 10 monclient: shutdown 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 msgr2=0x7f01381367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01300098d0 tx=0x7f0130019200 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 shutdown_connections 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f011c03e480 0x7f011c040940 unknown :-1 s=CLOSED pgs=3572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01381396a0 0x7f01381367b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01380a68e0 0x7f013813df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 --2- 172.21.15.105:0/2856982564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01380a4910 0x7f013813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 >> 172.21.15.105:0/2856982564 conn(0x7f0138068ba0 msgr2=0x7f01380a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 shutdown_connections 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 -- 172.21.15.105:0/2856982564 wait complete. 2024-09-17T16:39:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:48.748+0000 7f0140324640 1 librados: shutdown 2024-09-17T16:39:49.134 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-17T16:39:49.124+0000 7f92fd702640 -1 snap_mapper.add_oid found existing snaps mapped on 102:3ce4cc87:test-rados-api-smithi160-29504-97::foo:21, removing 2024-09-17T16:39:49.135 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-17T16:39:49.124+0000 7f51d48e3640 -1 snap_mapper.add_oid found existing snaps mapped on 102:3ce4cc87:test-rados-api-smithi160-29504-97::foo:21, removing 2024-09-17T16:39:51.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: get_monmap_and_config 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: build_initial_monmap 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: monmap: 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:39:51.901550+0000 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:39:51.901550+0000 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:39:51.905 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:39:51.906 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:39:51.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 Processor -- start 2024-09-17T16:39:51.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 -- start start 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: init 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: _reopen_session rank -1 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient: _add_conns ranks=[6,9,2,7,5,1,4,0,10,8,3] 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-d con 0x7f8c080a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-a con 0x7f8c080a3f80 addr v2:172.21.15.105:3300/0 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-b con 0x7f8c08137610 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8c080746f0 con 0x7f8c080a3f80 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:51.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c080a66c0 con 0x7f8c08137610 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8c080a35a0 con 0x7f8c080a5860 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0ff28640 10 monclient(hunting): authenticate will time out at 1675.167544s 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0e4c5640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 msgr2=0x7f8c08137ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0e4c5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:49586/0 (socket says 172.21.15.105:49586) 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59460/0 (socket says 172.21.15.105:59460) 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 learned_addr learned my addr 172.21.15.105:0/2405060076 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8c080a5860 auth_method 0 2024-09-17T16:39:51.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8c080a3f80 auth_method 0 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:51.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:51.896+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:52.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:52.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:52.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8c080a3f80 auth_method 0 2024-09-17T16:39:52.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:52.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8c080a5860 auth_method 0 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:52.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:52.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 msgr2=0x7f8c08137ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:52.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.100+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 msgr2=0x7f8c08137ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8c080a3f80 auth_method 0 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8c080a5860 auth_method 0 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:52.504+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:53.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8c080a3f80 auth_method 0 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8c080a5860 auth_method 0 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 msgr2=0x7f8c08137ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:39:53.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:53.308+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): tick 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 msgr2=0x7f8c080a5c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8c080a5860 0x7f8c080a5c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 msgr2=0x7f8c08137ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c08137610 0x7f8c08137ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 msgr2=0x7f8c080a6180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.105:3300/0 conn(0x7f8c080a3f80 0x7f8c080a6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:54.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient: _add_conns ranks=[3,8,10,1,7,0,5,4] 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-g con 0x7f8bf4003830 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-e con 0x7f8bf40061c0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-f con 0x7f8bf4008d20 addr v2:172.21.15.160:3300/0 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8c080a66c0 con 0x7f8bf4008d20 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8c080a35a0 con 0x7f8bf4003830 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8bf400b980 con 0x7f8bf40061c0 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.896+0000 7f8c077fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:54.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 msgr2=0x7f8bf400b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf40061c0 auth_method 0 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4003830 auth_method 0 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:54.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:54.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:54.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:55.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf40061c0 auth_method 0 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 msgr2=0x7f8bf400b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4003830 auth_method 0 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:55.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:55.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:55.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 msgr2=0x7f8bf400b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf40061c0 auth_method 0 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4003830 auth_method 0 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:55.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:55.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:39:55.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:55.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:55.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:55.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 msgr2=0x7f8bf400b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf40061c0 auth_method 0 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:56.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4003830 auth_method 0 2024-09-17T16:39:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:56.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:56.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:56.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:56.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:56.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:56.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:56.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 10 monclient(hunting): tick 2024-09-17T16:39:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 msgr2=0x7f8bf4008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.896+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8bf40061c0 0x7f8bf4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 msgr2=0x7f8bf4005c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bf4003830 0x7f8bf4005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 msgr2=0x7f8bf400b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> v2:172.21.15.160:3300/0 conn(0x7f8bf4008d20 0x7f8bf400b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 reap_dead start 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient: _add_conns ranks=[4,5,0,1,7] 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 reap_dead start 2024-09-17T16:39:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-c con 0x7f8bf40091d0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-h con 0x7f8bf4006b80 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bf4006fe0 addr v1:172.21.15.105:6789/0 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c080a19e0 con 0x7f8bf4006fe0 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8c080a66c0 con 0x7f8bf40091d0 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8c080a35a0 con 0x7f8bf4006b80 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c077fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf40091d0 auth_method 0 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:57.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:57.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf4006b80 auth_method 0 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:57.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:57.900+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:39:58.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf40091d0 auth_method 0 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf4006b80 auth_method 0 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:58.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:58.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.104+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:39:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf4006b80 auth_method 0 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf40091d0 auth_method 0 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:58.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:58.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:58.508+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:39:59.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8bf4006b80 auth_method 0 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf40091d0 auth_method 0 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:39:59.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:39:59.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:39:59.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:39:59.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:39:59.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:39:59.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:39:59.312+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:40:00.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): tick 2024-09-17T16:40:00.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:40:00.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:40:00.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 msgr2=0x7f8bf4005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8bf4006b80 0x7f8bf4005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 msgr2=0x7f8bf4006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8bf40091d0 0x7f8bf4006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 >> v1:172.21.15.105:6789/0 conn(0x7f8bf4006fe0 legacy=0x7f8bf4007460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient: _add_conns ranks=[7,1] 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-i con 0x7f8bf4008080 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8bf4011000 addr v1:172.21.15.160:6789/0 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf4007cd0 con 0x7f8bf4011000 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8c080a66c0 con 0x7f8bf4008080 2024-09-17T16:40:00.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c077fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v1:172.21.15.160:6789/0 conn(0x7f8bf4011000 legacy=0x7f8bf40133f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4008080 auth_method 0 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 msgr2=0x7f8bf4008460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 msgr2=0x7f8bf4008460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:40:00.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:00.900+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2405060076 >> v1:172.21.15.160:6789/0 conn(0x7f8bf4011000 legacy=0x7f8bf40133f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:40:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request con 0x7f8bf4008080 auth_method 0 2024-09-17T16:40:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient(hunting): handle_auth_done global_id 64420 payload 274 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 >> v1:172.21.15.160:6789/0 conn(0x7f8bf4011000 legacy=0x7f8bf40133f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 1 monclient: found mon.noname-i 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:40:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.104+0000 7f8c0dcc4640 1 -- 172.21.15.105:0/2405060076 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf4015c90 con 0x7f8bf4008080 2024-09-17T16:40:01.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.108+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40036b0 tx=0x7f8c0000a460 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6aba903351b97d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 1 -- 172.21.15.105:0/2405060076 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c000b4070 con 0x7f8bf4008080 2024-09-17T16:40:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:40:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: dump: 2024-09-17T16:40:01.119 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.120 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 1 -- 172.21.15.105:0/2405060076 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c000a7ad0 con 0x7f8bf4008080 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 5 monclient: authenticate success, global_id 64420 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 4 set_mon_vals no callback set 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 1 -- 172.21.15.105:0/2405060076 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0000f040 con 0x7f8bf4008080 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ccc2640 10 monclient: dump: 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.123 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 10 monclient: shutdown 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 msgr2=0x7f8bf4008460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2405060076 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8bf4008080 0x7f8bf4008460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40036b0 tx=0x7f8c0000a460 comp rx=0 tx=0).stop 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0e4c5640 1 -- 172.21.15.105:0/2405060076 reap_dead start 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2405060076 shutdown_connections 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2405060076 >> 172.21.15.105:0/2405060076 conn(0x7f8c080aa450 msgr2=0x7f8c080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2405060076 shutdown_connections 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2405060076 wait complete. 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.112+0000 7f8c0ff28640 10 monclient: monmap: 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 librados: starting msgr at 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 librados: starting objecter 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 Processor -- start 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- start start 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 librados: setting wanted keys 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 librados: calling monclient init 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: init 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _add_conns ranks=[2,4,7,8,0,6,1,5,3] 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 0x7f8c081cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-b con 0x7f8bf4008080 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-c con 0x7f8c08136760 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c0806d220 0x7f8c0806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0dcc4640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 msgr2=0x7f8c081cdd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): picked mon.noname-i con 0x7f8c0806d220 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c080a73b0 con 0x7f8bf4008080 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0dcc4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 0x7f8c081cdd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f8c080a1c60 con 0x7f8c08136760 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8c080a0df0 con 0x7f8c0806d220 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient(hunting): authenticate will time out at 1684.386314s 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39604/0 (socket says 172.21.15.105:39604) 2024-09-17T16:40:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2294238126 learned_addr learned my addr 172.21.15.105:0/2294238126 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request con 0x7f8c08136760 auth_method 0 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c0806d220 0x7f8c0806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): get_auth_request con 0x7f8c0806d220 auth_method 0 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0e4c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient(hunting): handle_auth_done global_id 74272 payload 824 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c0806d220 msgr2=0x7f8c0806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c0806d220 0x7f8c0806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 msgr2=0x7f8c081cdd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 0x7f8c081cdd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 monclient: found mon.noname-c 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 -- 172.21.15.105:0/2294238126 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c081370a0 con 0x7f8c08136760 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0d4c3640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ac40 tx=0x7f8bfc0187f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 client.?.objecter ms_handle_connect 0x7f8c08136760 2024-09-17T16:40:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bfc028070 con 0x7f8c08136760 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: dump: 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 5 monclient: authenticate success, global_id 74272 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _renew_subs 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00ad30 con 0x7f8c08136760 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c08125e80 con 0x7f8c08136760 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8bfc021c60 con 0x7f8c08136760 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c07fff640 4 set_mon_vals no callback set 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 client.74272.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c067fc640 10 monclient: dump: 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _renew_subs 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0806ee20 con 0x7f8c08136760 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.116+0000 7f8c0ff28640 1 librados: init done 2024-09-17T16:40:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8bfc015040 con 0x7f8c08136760 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c0ff28640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c0ff28640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c0ff28640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c08137ef0 con 0x7f8c08136760 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 0x7f8bf4045c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f8bfc012040 con 0x7f8c08136760 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 10 client.74272.objecter ms_dispatch 0x7f8c08000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 3 client.74272.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 3 client.74272.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:40:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.120+0000 7f8c067fc640 20 client.74272.objecter dump_active .. 0 homeless 2024-09-17T16:40:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 0x7f8bf4045c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c0dcc4640 10 monclient: get_auth_request con 0x7f8bf40437a0 auth_method 0 2024-09-17T16:40:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c067fc640 10 client.74272.objecter ms_handle_connect 0x7f8bf40437a0 2024-09-17T16:40:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c0dcc4640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 0x7f8bf4045c60 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 crypto rx=0x7f8c000b1000 tx=0x7f8c0001a040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c081370a0 con 0x7f8c08136760 2024-09-17T16:40:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.124+0000 7f8c067fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:01.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8c080a03e0 con 0x7f8c08136760 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c067fc640 1 -- 172.21.15.105:0/2294238126 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3200 (secure 0 0 0) 0x7f8bfc017da0 con 0x7f8c08136760 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:40:01.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c067fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:01.381 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":80,"quorum":[0,3,4,5,6,7,8],"quorum_names":["a","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 librados: watch_flush enter 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 librados: watch_flush exit 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 20 client.74272.objecter shutdown clearing up homeless session... 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 client.74272.objecter successfully canceled tick 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 msgr2=0x7f8bf4045c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 0x7f8bf4045c60 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 crypto rx=0x7f8c000b1000 tx=0x7f8c0001a040 comp rx=0 tx=0).stop 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 10 monclient: shutdown 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 msgr2=0x7f8c08136b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ac40 tx=0x7f8bfc0187f0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 shutdown_connections 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8bf40437a0 0x7f8bf4045c60 unknown :-1 s=CLOSED pgs=3573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8c0806d220 0x7f8c0806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f8c08136760 0x7f8c08136b60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 --2- 172.21.15.105:0/2294238126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8bf4008080 0x7f8c081cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 >> 172.21.15.105:0/2294238126 conn(0x7f8c080aa450 msgr2=0x7f8c081bf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 shutdown_connections 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.372+0000 7f8c0ff28640 1 -- 172.21.15.105:0/2294238126 wait complete. 2024-09-17T16:40:01.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.376+0000 7f8c0ff28640 1 librados: shutdown 2024-09-17T16:40:01.401 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-17T16:40:01.402 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-17T16:40:01.402 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:40:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 10 monclient: get_monmap_and_config 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 10 monclient: monmap: 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:01.514387+0000 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:01.514387+0000 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.518 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:01.519 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 1 Processor -- start 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 1 -- start start 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.508+0000 7fa402cbb640 10 monclient: init 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient: _add_conns ranks=[5,4,10,6,3,0,8,2,7,9,1] 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3fc0a68e0 0x7fa3fc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-h con 0x7fa3fc0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-c con 0x7fa3fc0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 0x7fa3fc139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-f con 0x7fa3fc1396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fa3fc0a3fc0 con 0x7fa3fc1396a0 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa3fc0a63c0 con 0x7fa3fc0a4910 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa3fc0a7740 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient(hunting): authenticate will time out at 1684.780180s 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa401258640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 msgr2=0x7fa3fc139ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39620/0 (socket says 172.21.15.105:39620) 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 -- 172.21.15.105:0/4254184547 learned_addr learned my addr 172.21.15.105:0/4254184547 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): get_auth_request con 0x7fa3fc0a4910 auth_method 0 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa401258640 1 --2- 172.21.15.105:0/4254184547 >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 0x7fa3fc139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_done global_id 74281 payload 274 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa400a57640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3fc0a68e0 0x7fa3fc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 -- 172.21.15.105:0/4254184547 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3fc0a68e0 msgr2=0x7fa3fc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3fc0a68e0 0x7fa3fc0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 -- 172.21.15.105:0/4254184547 >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 msgr2=0x7fa3fc139ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/4254184547 >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 0x7fa3fc139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 monclient: found mon.noname-c 2024-09-17T16:40:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 -- 172.21.15.105:0/4254184547 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc06ebc0 con 0x7fa3fc0a4910 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009870 tx=0x7fa3ec01c410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=429d970b65d9951b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 1 -- 172.21.15.105:0/4254184547 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3ec0b2070 con 0x7fa3fc0a4910 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: dump: 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.532 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 1 -- 172.21.15.105:0/4254184547 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ec0ab7b0 con 0x7fa3fc0a4910 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 5 monclient: authenticate success, global_id 74281 2024-09-17T16:40:01.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 4 set_mon_vals no callback set 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 1 -- 172.21.15.105:0/4254184547 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3ec0ab9e0 con 0x7fa3fc0a4910 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa3fb7fe640 10 monclient: dump: 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.534 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 10 monclient: shutdown 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- 172.21.15.105:0/4254184547 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 msgr2=0x7fa3fc0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009870 tx=0x7fa3ec01c410 comp rx=0 tx=0).stop 2024-09-17T16:40:01.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- 172.21.15.105:0/4254184547 shutdown_connections 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- 172.21.15.105:0/4254184547 >> v2:172.21.15.160:3300/0 conn(0x7fa3fc1396a0 0x7fa3fc139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3fc0a4910 0x7fa3fc0a7200 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 --2- 172.21.15.105:0/4254184547 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3fc0a68e0 0x7fa3fc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- 172.21.15.105:0/4254184547 >> 172.21.15.105:0/4254184547 conn(0x7fa3fc068ba0 msgr2=0x7fa3fc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- 172.21.15.105:0/4254184547 shutdown_connections 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.512+0000 7fa402cbb640 1 -- 172.21.15.105:0/4254184547 wait complete. 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: monmap: 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.536 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 librados: starting msgr at 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 librados: starting objecter 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 Processor -- start 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- start start 2024-09-17T16:40:01.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 librados: setting wanted keys 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 librados: calling monclient init 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: init 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _add_conns ranks=[2,8,3,7,0,6,4,1,5] 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3fc0a4910 0x7fa3fc1d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-b con 0x7fa3fc0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 0x7fa3fc1d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-e con 0x7fa3fc0a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3fc1396a0 0x7fa3fc13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): picked mon.noname-g con 0x7fa3fc1396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc06e630 con 0x7fa3fc0a4910 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc0a7af0 con 0x7fa3fc1396a0 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa3fc143320 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): authenticate will time out at 1684.785182s 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3fc0a4910 msgr2=0x7fa3fc1d1500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3fc0a4910 0x7fa3fc1d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 0x7fa3fc1d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 0x7fa3fc1d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43718/0 (socket says 172.21.15.105:43718) 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 -- 172.21.15.105:0/576006444 learned_addr learned my addr 172.21.15.105:0/576006444 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): get_auth_request con 0x7fa3fc0a68e0 auth_method 0 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa401258640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3fc1396a0 0x7fa3fc13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa401258640 10 monclient(hunting): get_auth_request con 0x7fa3fc1396a0 auth_method 0 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa401258640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa401258640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa401258640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient(hunting): handle_auth_done global_id 54575 payload 824 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3fc1396a0 msgr2=0x7fa3fc13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3fc1396a0 0x7fa3fc13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3fc0a4910 msgr2=0x7fa3fc1d1500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3fc0a4910 0x7fa3fc1d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 monclient: found mon.noname-e 2024-09-17T16:40:01.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc123cf0 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3fbfff640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 0x7fa3fc1d1a40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0b0040 tx=0x7fa3ec0191b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 client.?.objecter ms_handle_connect 0x7fa3fc0a68e0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3ec0b2040 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: dump: 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.541 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3ec0abdd0 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 5 monclient: authenticate success, global_id 54575 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _renew_subs 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc0aa230 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3ec004420 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 client.54575.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: dump: 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.543 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3faffd640 4 set_mon_vals no callback set 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _renew_subs 2024-09-17T16:40:01.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc123a10 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 librados: init done 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 0x7fa3fc13dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc051e80 con 0x7fa3fc124110 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 0x7fa3fc13dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient: get_auth_request con 0x7fa3fc124110 auth_method 0 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 10 monclient(hunting): handle_auth_done global_id 36045 payload 824 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3fc139ae0 con 0x7fa3fc124110 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa3ec0ab780 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa400a57640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 0x7fa3fc13dc10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c6f0 tx=0x7fa3f000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3e803e480 0x7fa3e8040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 client.54575.objecter ms_handle_connect 0x7fa3fc124110 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 client.54575.objecter resend_mon_ops 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7fa3ec0e9ea0 con 0x7fa3fc0a68e0 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 client.54575.objecter ms_dispatch 0x7fa3fc000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 3 client.54575.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 3 client.54575.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 20 client.54575.objecter dump_active .. 0 homeless 2024-09-17T16:40:01.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f00044a0 con 0x7fa3fc124110 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.516+0000 7fa3f97fa640 10 monclient: dump: 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.547 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa3f0004030 con 0x7fa3fc124110 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 msgr2=0x7fa3fc13dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 0x7fa3fc13dc10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c6f0 tx=0x7fa3f000cbb0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 msgr2=0x7fa3fc13dc10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa401258640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3e803e480 0x7fa3e8040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc124110 0x7fa3fc13dc10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa401258640 10 monclient: get_auth_request con 0x7fa3e803e480 auth_method 0 2024-09-17T16:40:01.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa3f97fa640 10 client.54575.objecter ms_handle_connect 0x7fa3e803e480 2024-09-17T16:40:01.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.520+0000 7fa401258640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3e803e480 0x7fa3e8040940 secure :-1 s=READY pgs=3574 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4003cc0 tx=0x7fa3f40093f0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 0x7fa3fc13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc0a9dc0 con 0x7fa3fc13e150 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 0x7fa3fc13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient: get_auth_request con 0x7fa3fc13e150 auth_method 0 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 10 monclient(hunting): handle_auth_done global_id 36054 payload 824 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 1 -- 172.21.15.105:0/576006444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa3fc0746d0 con 0x7fa3fc13e150 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa400a57640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 0x7fa3fc13e560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c10 tx=0x7fa3f0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 client.54575.objecter ms_handle_connect 0x7fa3fc13e150 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 client.54575.objecter resend_mon_ops 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f0003d60 con 0x7fa3fc13e150 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 monclient: dump: 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.552 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9432 (secure 0 0 0) 0x7fa3f00038f0 con 0x7fa3fc13e150 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:01.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 msgr2=0x7fa3fc13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 0x7fa3fc13e560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c10 tx=0x7fa3f0004740 comp rx=0 tx=0).stop 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 msgr2=0x7fa3fc13e560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa3f97fa640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3fc13e150 0x7fa3fc13e560 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c10 tx=0x7fa3f0004740 comp rx=0 tx=0).stop 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 592328, 2024-09-17T16:40:01.554 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:01.555 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:01.556 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:01.557 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:01.558 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:01.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.560 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:01.561 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.563 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.564 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:01.565 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.566 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.567 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:01.568 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.570 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.572 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.573 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:40:01.575 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:01.576 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 librados: watch_flush enter 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 librados: watch_flush exit 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 20 client.54575.objecter shutdown clearing up homeless session... 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 client.54575.objecter successfully canceled tick 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3e803e480 msgr2=0x7fa3e8040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3e803e480 0x7fa3e8040940 secure :-1 s=READY pgs=3574 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4003cc0 tx=0x7fa3f40093f0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 10 monclient: shutdown 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 msgr2=0x7fa3fc1d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 --2- 172.21.15.105:0/576006444 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3fc0a68e0 0x7fa3fc1d1a40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0b0040 tx=0x7fa3ec0191b0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa401258640 1 -- 172.21.15.105:0/576006444 reap_dead start 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa401258640 1 -- 172.21.15.105:0/576006444 reap_dead start 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 shutdown_connections 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 >> 172.21.15.105:0/576006444 conn(0x7fa3fc068ba0 msgr2=0x7fa3fc0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 shutdown_connections 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 -- 172.21.15.105:0/576006444 wait complete. 2024-09-17T16:40:01.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.524+0000 7fa402cbb640 1 librados: shutdown 2024-09-17T16:40:01.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:01.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-17T16:40:01.581 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: get_monmap_and_config 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: monmap: 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.695 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:01.691945+0000 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:01.691945+0000 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.696 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.697 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:01.697 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:01.697 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 Processor -- start 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 -- start start 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: init 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient: _add_conns ranks=[6,8,0,1,3,7,10,2,5,4,9] 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 0x7fdd580a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-d con 0x7fdd580a5860 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 0x7fdd58139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-e con 0x7fdd580a6180 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdd58139f40 addr v1:172.21.15.105:6789/0 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5806db60 con 0x7fdd58139f40 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdd5806d260 con 0x7fdd580a5860 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fdd5806ec00 con 0x7fdd580a6180 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5ca2b640 10 monclient(hunting): authenticate will time out at 1684.957729s 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 0x7fdd580a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd56d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdd58139f40 0x7fdd58140c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33820/0 (socket says 172.21.15.105:33820) 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2004384257 learned_addr learned my addr 172.21.15.105:0/2004384257 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 0x7fdd58139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): get_auth_request con 0x7fdd580a5860 auth_method 0 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): get_auth_request con 0x7fdd580a6180 auth_method 0 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient(hunting): handle_auth_done global_id 54584 payload 274 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5559a640 1 -- 172.21.15.105:0/2004384257 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209628860 0 0) 0x7fdd5806db60 con 0x7fdd58139f40 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 -- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 msgr2=0x7fdd580a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 0x7fdd580a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 -- 172.21.15.105:0/2004384257 >> v1:172.21.15.105:6789/0 conn(0x7fdd58139f40 legacy=0x7fdd58140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 monclient: found mon.noname-e 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd55d9b640 1 -- 172.21.15.105:0/2004384257 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5806eee0 con 0x7fdd580a6180 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5559a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.688+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 0x7fdd580a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 0x7fdd58139a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009dc0 tx=0x7fdd5001c850 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3353b4a02ba0ce32 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 1 -- 172.21.15.105:0/2004384257 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd500b2070 con 0x7fdd580a6180 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: dump: 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 1 -- 172.21.15.105:0/2004384257 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd500a5a10 con 0x7fdd580a6180 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 1 -- 172.21.15.105:0/2004384257 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd500a5c00 con 0x7fdd580a6180 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 5 monclient: authenticate success, global_id 54584 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 4 set_mon_vals no callback set 2024-09-17T16:40:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5559a640 10 monclient: dump: 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: shutdown 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 msgr2=0x7fdd58139a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 0x7fdd58139a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009dc0 tx=0x7fdd5001c850 comp rx=0 tx=0).stop 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2004384257 shutdown_connections 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdd580a6180 0x7fdd58139a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2004384257 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdd580a5860 0x7fdd580a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2004384257 >> 172.21.15.105:0/2004384257 conn(0x7fdd580aa450 msgr2=0x7fdd580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2004384257 shutdown_connections 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2004384257 wait complete. 2024-09-17T16:40:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: monmap: 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 librados: starting msgr at 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 librados: starting objecter 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 Processor -- start 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- start start 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 librados: setting wanted keys 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 librados: calling monclient init 2024-09-17T16:40:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: init 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _add_conns ranks=[7,3,4,5,1,6,8,2,0] 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdd580a5860 0x7fdd5813eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-i con 0x7fdd580a5860 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd580a6180 0x7fdd5813c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-g con 0x7fdd580a6180 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:40:01.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 0x7fdd5813cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): picked mon.noname-c con 0x7fdd58139f40 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fdd580a0d80 con 0x7fdd580a6180 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdd580a65a0 con 0x7fdd58139f40 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdd580a0f00 con 0x7fdd580a5860 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient(hunting): authenticate will time out at 1684.962872s 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 0x7fdd5813cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 0x7fdd5813cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39636/0 (socket says 172.21.15.105:39636) 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 learned_addr learned my addr 172.21.15.105:0/2102509869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): get_auth_request con 0x7fdd58139f40 auth_method 0 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd580a6180 0x7fdd5813c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdd580a5860 0x7fdd5813eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 10 monclient(hunting): get_auth_request con 0x7fdd580a6180 auth_method 0 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd55d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient(hunting): handle_auth_done global_id 74290 payload 824 2024-09-17T16:40:01.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdd580a5860 msgr2=0x7fdd5813eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdd580a5860 0x7fdd5813eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd580a6180 msgr2=0x7fdd5813c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd580a6180 0x7fdd5813c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 monclient: found mon.noname-c 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd580a3530 con 0x7fdd58139f40 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5659c640 10 monclient: get_auth_request con 0x7fdd580a5860 auth_method 0 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdd580a5860 0x7fdd5813eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd56d9d640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 0x7fdd5813cac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c390 tx=0x7fdd4800c850 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 client.?.objecter ms_handle_connect 0x7fdd58139f40 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:01.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd480047f0 con 0x7fdd58139f40 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: dump: 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.723 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd480049d0 con 0x7fdd58139f40 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 5 monclient: authenticate success, global_id 74290 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _renew_subs 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd58142020 con 0x7fdd58139f40 2024-09-17T16:40:01.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd480133c0 con 0x7fdd58139f40 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd4effd640 10 monclient: dump: 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.725 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd54d99640 4 set_mon_vals no callback set 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 client.74290.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _renew_subs 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd58140060 con 0x7fdd58139f40 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.692+0000 7fdd5ca2b640 1 librados: init done 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5ca2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 0x7fdd5813d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdd58051e80 con 0x7fdd5802e5d0 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 0x7fdd5813d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient: get_auth_request con 0x7fdd5802e5d0 auth_method 0 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdd48011070 con 0x7fdd58139f40 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 10 monclient(hunting): handle_auth_done global_id 74299 payload 824 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdd3c03e580 0x7fdd3c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd5802e9b0 con 0x7fdd5802e5d0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7fdd4804fa40 con 0x7fdd58139f40 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 client.74290.objecter ms_dispatch 0x7fdd58000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 3 client.74290.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 3 client.74290.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 20 client.74290.objecter dump_active .. 0 homeless 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 client.74290.objecter ms_handle_connect 0x7fdd5802e5d0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 client.74290.objecter resend_mon_ops 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 0x7fdd5813d000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd40010790 tx=0x7fdd40010c50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdd3c03e580 0x7fdd3c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd4000f040 con 0x7fdd5802e5d0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 monclient: dump: 2024-09-17T16:40:01.728 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.729 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd55d9b640 10 monclient: get_auth_request con 0x7fdd3c03e580 auth_method 0 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdd4001c400 con 0x7fdd5802e5d0 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 msgr2=0x7fdd5813d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 0x7fdd5813d000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd40010790 tx=0x7fdd40010c50 comp rx=0 tx=0).stop 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 msgr2=0x7fdd5813d000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd5802e5d0 0x7fdd5813d000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd4effd640 10 client.74290.objecter ms_handle_connect 0x7fdd3c03e580 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.696+0000 7fdd55d9b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdd3c03e580 0x7fdd3c040a40 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 crypto rx=0x7fdd500af000 tx=0x7fdd500a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.700+0000 7fdd5ca2b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:01.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.700+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 0x7fdd58064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.700+0000 7fdd5ca2b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.700+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdd580a1d60 con 0x7fdd58063e00 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 0x7fdd58064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient: get_auth_request con 0x7fdd58063e00 auth_method 0 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 10 monclient(hunting): handle_auth_done global_id 74308 payload 824 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 1 -- 172.21.15.105:0/2102509869 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdd5813fcc0 con 0x7fdd58063e00 2024-09-17T16:40:01.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 client.74290.objecter ms_handle_connect 0x7fdd58063e00 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 client.74290.objecter resend_mon_ops 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5659c640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 0x7fdd58064210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd40010790 tx=0x7fdd400046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd40010010 con 0x7fdd58063e00 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 monclient: dump: 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.732 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9431 (secure 0 0 0) 0x7fdd40010010 con 0x7fdd58063e00 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 msgr2=0x7fdd58064210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 0x7fdd58064210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd40010790 tx=0x7fdd400046d0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 msgr2=0x7fdd58064210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd4effd640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58063e00 0x7fdd58064210 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd40010790 tx=0x7fdd400046d0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 41167, 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:01.734 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.735 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.736 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:01.737 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.746 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.748 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.749 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.750 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.751 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.752 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.753 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.755 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.756 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.757 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.758 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:01.758 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:01.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 10 librados: watch_flush enter 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 10 librados: watch_flush exit 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 20 client.74290.objecter shutdown clearing up homeless session... 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 10 client.74290.objecter successfully canceled tick 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdd3c03e580 msgr2=0x7fdd3c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdd3c03e580 0x7fdd3c040a40 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 crypto rx=0x7fdd500af000 tx=0x7fdd500a7070 comp rx=0 tx=0).stop 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 10 monclient: shutdown 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 msgr2=0x7fdd5813cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 --2- 172.21.15.105:0/2102509869 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdd58139f40 0x7fdd5813cac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c390 tx=0x7fdd4800c850 comp rx=0 tx=0).stop 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 reap_dead start 2024-09-17T16:40:01.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd56d9d640 1 -- 172.21.15.105:0/2102509869 reap_dead start 2024-09-17T16:40:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 shutdown_connections 2024-09-17T16:40:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 >> 172.21.15.105:0/2102509869 conn(0x7fdd580aa450 msgr2=0x7fdd580a2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 shutdown_connections 2024-09-17T16:40:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 -- 172.21.15.105:0/2102509869 wait complete. 2024-09-17T16:40:01.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.704+0000 7fdd5ca2b640 1 librados: shutdown 2024-09-17T16:40:01.761 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:01.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-17T16:40:01.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:40:01.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: get_monmap_and_config 2024-09-17T16:40:01.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: monmap: 2024-09-17T16:40:01.879 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:01.876137+0000 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:01.876137+0000 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:01.881 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 Processor -- start 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 -- start start 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: init 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient: _add_conns ranks=[7,1,9,8,2,5,10,3,4,0,6] 2024-09-17T16:40:01.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f54137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-i con 0x7f2f54137830 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2f540a56c0 addr v1:172.21.15.160:6789/0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f540a61f0 0x7f2f5413f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-a con 0x7f2f540a61f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f540a33c0 con 0x7f2f540a56c0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2f540a1d60 con 0x7f2f540a61f0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2f540a1660 con 0x7f2f54137830 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f58e72640 10 monclient(hunting): authenticate will time out at 1685.141837s 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f49d9b640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f2f540a56c0 legacy=0x7f2f540a5ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f540a61f0 0x7f2f5413f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f540a61f0 0x7f2f5413f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42132/0 (socket says 172.21.15.105:42132) 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 -- 172.21.15.105:0/2469969717 learned_addr learned my addr 172.21.15.105:0/2469969717 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): get_auth_request con 0x7f2f540a61f0 auth_method 0 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f5259c640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f54137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f5259c640 10 monclient(hunting): get_auth_request con 0x7f2f54137830 auth_method 0 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_done global_id 36072 payload 274 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 -- 172.21.15.105:0/2469969717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 msgr2=0x7f2f54137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f54137c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 -- 172.21.15.105:0/2469969717 >> v1:172.21.15.160:6789/0 conn(0x7f2f540a56c0 legacy=0x7f2f540a5ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 monclient: found mon.noname-a 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.872+0000 7f2f52d9d640 1 -- 172.21.15.105:0/2469969717 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f540a1ae0 con 0x7f2f540a61f0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f540a61f0 0x7f2f5413f6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7124fa52a4a6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 1 -- 172.21.15.105:0/2469969717 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f4c0090d0 con 0x7f2f540a61f0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: dump: 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.887 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 1 -- 172.21.15.105:0/2469969717 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0a5ac0 con 0x7f2f540a61f0 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 1 -- 172.21.15.105:0/2469969717 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f4c0a5cb0 con 0x7f2f540a61f0 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 5 monclient: authenticate success, global_id 36072 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 4 set_mon_vals no callback set 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f51d9b640 10 monclient: dump: 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.889 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: shutdown 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- 172.21.15.105:0/2469969717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f540a61f0 msgr2=0x7f2f5413f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f540a61f0 0x7f2f5413f6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c01ca10 comp rx=0 tx=0).stop 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- 172.21.15.105:0/2469969717 shutdown_connections 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f540a61f0 0x7f2f5413f6d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- 172.21.15.105:0/2469969717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f54137c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- 172.21.15.105:0/2469969717 >> 172.21.15.105:0/2469969717 conn(0x7f2f540aa390 msgr2=0x7f2f540a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- 172.21.15.105:0/2469969717 shutdown_connections 2024-09-17T16:40:01.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- 172.21.15.105:0/2469969717 wait complete. 2024-09-17T16:40:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: build_initial_monmap 2024-09-17T16:40:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: monmap: 2024-09-17T16:40:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:01.898 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:01.899 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 librados: starting msgr at 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 librados: starting objecter 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 Processor -- start 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- start start 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 librados: setting wanted keys 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 librados: calling monclient init 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: init 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient: _add_conns ranks=[2,1,7,4,0,3,6,5,8] 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f540a56c0 0x7f2f5406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-b con 0x7f2f540a56c0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f540a61f0 0x7f2f5406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-f con 0x7f2f540a61f0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): picked mon.noname-i con 0x7f2f54137830 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541382d0 con 0x7f2f540a61f0 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2f54138150 con 0x7f2f540a56c0 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2f54142430 con 0x7f2f54137830 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f58e72640 10 monclient(hunting): authenticate will time out at 1685.146826s 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f5259c640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f540a56c0 msgr2=0x7f2f5406d220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f5259c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f540a56c0 0x7f2f5406d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f49d9b640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f540a61f0 msgr2=0x7f2f5406d760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f49d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f540a61f0 0x7f2f5406d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:41654/0 (socket says 172.21.15.105:41654) 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 -- 172.21.15.105:0/675489687 learned_addr learned my addr 172.21.15.105:0/675489687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): get_auth_request con 0x7f2f54137830 auth_method 0 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient(hunting): handle_auth_done global_id 64447 payload 824 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f540a56c0 msgr2=0x7f2f5406d220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:01.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f540a56c0 0x7f2f5406d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f540a61f0 msgr2=0x7f2f5406d760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f540a61f0 0x7f2f5406d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 monclient: found mon.noname-i 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.876+0000 7f2f52d9d640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5413d990 con 0x7f2f54137830 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f52d9d640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c01c840 tx=0x7f2f4c0197b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.?.objecter ms_handle_connect 0x7f2f54137830 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:01.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f4c0bb020 con 0x7f2f54137830 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: dump: 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.904 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c003bb0 con 0x7f2f54137830 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 5 monclient: authenticate success, global_id 64447 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 monclient: _renew_subs 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f4c003d60 con 0x7f2f54137830 2024-09-17T16:40:01.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5413f960 con 0x7f2f54137830 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5159a640 4 set_mon_vals no callback set 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: dump: 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.906 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 client.64447.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 monclient: _renew_subs 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5413d6b0 con 0x7f2f54137830 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 1 librados: init done 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 0x7f2f54138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2f5413d330 con 0x7f2f5413c310 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 0x7f2f54138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient: get_auth_request con 0x7f2f5413c310 auth_method 0 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 10 monclient(hunting): handle_auth_done global_id 24729 payload 824 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f54137c10 con 0x7f2f5413c310 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f5259c640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 0x7f2f54138480 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f4400d960 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.64447.objecter ms_handle_connect 0x7f2f5413c310 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.64447.objecter resend_mon_ops 2024-09-17T16:40:01.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2f4c0be020 con 0x7f2f54137830 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2f3403e570 0x7f2f34040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f2f4c0eada0 con 0x7f2f54137830 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.64447.objecter ms_dispatch 0x7f2f54000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 3 client.64447.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 3 client.64447.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 20 client.64447.objecter dump_active .. 0 homeless 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f44012a60 con 0x7f2f5413c310 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: dump: 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.909 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2f44019070 con 0x7f2f5413c310 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:01.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 msgr2=0x7f2f54138480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 0x7f2f54138480 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f4400d960 comp rx=0 tx=0).stop 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 msgr2=0x7f2f54138480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f5413c310 0x7f2f54138480 secure :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f4400d960 comp rx=0 tx=0).stop 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f49d9b640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2f3403e570 0x7f2f34040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f49d9b640 10 monclient: get_auth_request con 0x7f2f3403e570 auth_method 0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f4b7fe640 10 client.64447.objecter ms_handle_connect 0x7f2f3403e570 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.880+0000 7f2f49d9b640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2f3403e570 0x7f2f34040a30 secure :-1 s=READY pgs=3576 cs=0 l=1 rev1=1 crypto rx=0x7f2f5406e860 tx=0x7f2f3c006cc0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 0x7f2f54138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2f54051e80 con 0x7f2f541389c0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 0x7f2f54138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient: get_auth_request con 0x7f2f541389c0 auth_method 0 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:01.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 10 monclient(hunting): handle_auth_done global_id 24738 payload 824 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 1 -- 172.21.15.105:0/675489687 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2f54137290 con 0x7f2f541389c0 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 client.64447.objecter ms_handle_connect 0x7f2f541389c0 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 client.64447.objecter resend_mon_ops 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f5259c640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 0x7f2f54138dd0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f44004740 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f44012a60 con 0x7f2f541389c0 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 monclient: dump: 2024-09-17T16:40:01.912 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:01.913 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f2f440125f0 con 0x7f2f541389c0 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 msgr2=0x7f2f54138dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 0x7f2f54138dd0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f44004740 comp rx=0 tx=0).stop 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 msgr2=0x7f2f54138dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f4b7fe640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2f541389c0 0x7f2f54138dd0 secure :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2f44006f90 tx=0x7f2f44004740 comp rx=0 tx=0).stop 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:01.914 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 749527, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.915 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.916 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.917 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:01.918 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:01.919 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:01.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:01.926 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.927 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.929 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.930 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.931 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.932 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:01.933 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.934 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:01.935 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:01.936 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:01.936 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:01.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 librados: watch_flush enter 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 librados: watch_flush exit 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 20 client.64447.objecter shutdown clearing up homeless session... 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 client.64447.objecter successfully canceled tick 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2f3403e570 msgr2=0x7f2f34040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2f3403e570 0x7f2f34040a30 secure :-1 s=READY pgs=3576 cs=0 l=1 rev1=1 crypto rx=0x7f2f5406e860 tx=0x7f2f3c006cc0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 10 monclient: shutdown 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 msgr2=0x7f2f5406dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f52d9d640 1 -- 172.21.15.105:0/675489687 reap_dead start 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c01c840 tx=0x7f2f4c0197b0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 shutdown_connections 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 --2- 172.21.15.105:0/675489687 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2f54137830 0x7f2f5406dca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:01.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 >> 172.21.15.105:0/675489687 conn(0x7f2f540aa390 msgr2=0x7f2f540a83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 shutdown_connections 2024-09-17T16:40:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 -- 172.21.15.105:0/675489687 wait complete. 2024-09-17T16:40:01.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:01.888+0000 7f2f58e72640 1 librados: shutdown 2024-09-17T16:40:01.939 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:01.939 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-17T16:40:01.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:40:02.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.048+0000 7f394e214640 10 monclient: get_monmap_and_config 2024-09-17T16:40:02.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.048+0000 7f394e214640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.048+0000 7f394e214640 10 monclient: monmap: 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:02.055160+0000 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:02.055160+0000 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.059 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.060 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.060 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:02.060 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:02.060 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 Processor -- start 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 -- start start 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient: init 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient: _add_conns ranks=[4,3,7,1,2,5,9,0,10,6,8] 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): picked mon.noname-c con 0x7f39480a4750 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 0x7f39480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): picked mon.noname-g con 0x7f39480a0360 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3948137510 0x7f3948137950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): picked mon.noname-i con 0x7f3948137510 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f39480a6700 con 0x7f39480a0360 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f39480a55b0 con 0x7f39480a4750 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f394806c810 con 0x7f3948137510 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f394e214640 10 monclient(hunting): authenticate will time out at 1685.321128s 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39650/0 (socket says 172.21.15.105:39650) 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 -- 172.21.15.105:0/4293828860 learned_addr learned my addr 172.21.15.105:0/4293828860 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): get_auth_request con 0x7f39480a4750 auth_method 0 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3947fff640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3948137510 0x7f3948137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3946ffd640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 0x7f39480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3947fff640 10 monclient(hunting): get_auth_request con 0x7f3948137510 auth_method 0 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3947fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3947fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient(hunting): handle_auth_done global_id 74317 payload 274 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 -- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3948137510 msgr2=0x7f3948137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3948137510 0x7f3948137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 -- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 msgr2=0x7f39480a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 0x7f39480a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 monclient: found mon.noname-c 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f39477fe640 1 -- 172.21.15.105:0/4293828860 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39480a6240 con 0x7f39480a4750 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3946ffd640 10 monclient: get_auth_request con 0x7f39480a0360 auth_method 0 2024-09-17T16:40:02.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.052+0000 7f3946ffd640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 0x7f39480a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:40:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39477fe640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39380098a0 tx=0x7f393801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8178d0e13fd1d463 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 1 -- 172.21.15.105:0/4293828860 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39380b0070 con 0x7f39480a4750 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: dump: 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 1 -- 172.21.15.105:0/4293828860 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39380aa810 con 0x7f39480a4750 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 5 monclient: authenticate success, global_id 74317 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 1 -- 172.21.15.105:0/4293828860 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39380a9300 con 0x7f39480a4750 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 4 set_mon_vals no callback set 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39467fc640 10 monclient: dump: 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: shutdown 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/4293828860 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 msgr2=0x7f39480a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39380098a0 tx=0x7f393801c440 comp rx=0 tx=0).stop 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/4293828860 shutdown_connections 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3948137510 0x7f3948137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39480a0360 0x7f39480a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- 172.21.15.105:0/4293828860 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f39480a4750 0x7f39480a4b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/4293828860 >> 172.21.15.105:0/4293828860 conn(0x7f39480a93f0 msgr2=0x7f39480a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/4293828860 shutdown_connections 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/4293828860 wait complete. 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: monmap: 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 librados: starting msgr at 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 librados: starting objecter 2024-09-17T16:40:02.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 Processor -- start 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- start start 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 librados: setting wanted keys 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 librados: calling monclient init 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: init 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _add_conns ranks=[1,6,4,8,5,3,0,2,7] 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39480a0360 0x7f39481d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): picked mon.noname-f con 0x7f39480a0360 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 0x7f39481d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): picked mon.noname-d con 0x7f39480a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3948137510 0x7f39481d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): picked mon.noname-c con 0x7f3948137510 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3948141380 con 0x7f39480a0360 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3948141200 con 0x7f3948137510 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3948141500 con 0x7f39480a4750 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient(hunting): authenticate will time out at 1685.326672s 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 0x7f39481d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39477fe640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39480a0360 msgr2=0x7f39481d1440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f39477fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39480a0360 0x7f39481d1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3948137510 0x7f39481d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 0x7f39481d1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:43140/0 (socket says 172.21.15.105:43140) 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 -- 172.21.15.105:0/2455364848 learned_addr learned my addr 172.21.15.105:0/2455364848 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): get_auth_request con 0x7f39480a4750 auth_method 0 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): get_auth_request con 0x7f3948137510 auth_method 0 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3947fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient(hunting): handle_auth_done global_id 24747 payload 824 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3948137510 msgr2=0x7f39481d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3948137510 0x7f39481d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39480a0360 msgr2=0x7f39481d1440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39480a0360 0x7f39481d1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 monclient: found mon.noname-d 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3948123940 con 0x7f39480a4750 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3946ffd640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 0x7f39481d1980 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c7f0 tx=0x7f393c00ccb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 client.?.objecter ms_handle_connect 0x7f39480a4750 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:02.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f393c0044a0 con 0x7f39480a4750 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: dump: 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.083 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 5 monclient: authenticate success, global_id 24747 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c007830 con 0x7f39480a4750 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _renew_subs 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39480a6d20 con 0x7f39480a4750 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f393c003620 con 0x7f39480a4750 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f3945ffb640 4 set_mon_vals no callback set 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f392ffff640 10 monclient: dump: 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.085 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 client.24747.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _renew_subs 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f394809f7e0 con 0x7f39480a4750 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.056+0000 7f394e214640 1 librados: init done 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f394e214640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f394e214640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 0x7f394809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3948051e80 con 0x7f394813d400 2024-09-17T16:40:02.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 0x7f394809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f393c004030 con 0x7f39480a4750 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient: get_auth_request con 0x7f394813d400 auth_method 0 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f393403e580 0x7f3934040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 10 monclient(hunting): handle_auth_done global_id 54593 payload 824 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3948137950 con 0x7f394813d400 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(449..449 src has 1..449) ==== 6202+0+0 (secure 0 0 0) 0x7f393c050030 con 0x7f39480a4750 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 client.24747.objecter ms_dispatch 0x7f3948000ba0 osd_map(449..449 src has 1..449) 2024-09-17T16:40:02.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 3 client.24747.objecter handle_osd_map got epochs [449,449] > 0 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 3 client.24747.objecter handle_osd_map decoding full epoch 449 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 20 client.24747.objecter dump_active .. 0 homeless 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 client.24747.objecter ms_handle_connect 0x7f394813d400 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 client.24747.objecter resend_mon_ops 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f39477fe640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 0x7f394809da00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393801ac80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f3947fff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f393403e580 0x7f3934040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39380b0040 con 0x7f394813d400 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 monclient: dump: 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.088 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f3947fff640 10 monclient: get_auth_request con 0x7f393403e580 auth_method 0 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f39380aa3a0 con 0x7f394813d400 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 msgr2=0x7f394809da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 0x7f394809da00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393801ac80 comp rx=0 tx=0).stop 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 msgr2=0x7f394809da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394813d400 0x7f394809da00 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393801ac80 comp rx=0 tx=0).stop 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f3947fff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f393403e580 0x7f3934040a40 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 crypto rx=0x7f394813cb20 tx=0x7f3940009780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.060+0000 7f392ffff640 10 client.24747.objecter ms_handle_connect 0x7f393403e580 2024-09-17T16:40:02.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f394e214640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f394e214640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 0x7f394809e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f394e214640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f394809e890 con 0x7f394809df40 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 0x7f394809e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient: get_auth_request con 0x7f394809df40 auth_method 0 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 10 monclient(hunting): handle_auth_done global_id 54602 payload 824 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 1 -- 172.21.15.105:0/2455364848 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3948123c80 con 0x7f394809df40 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f39477fe640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 0x7f394809e350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39380097c0 tx=0x7f39380191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f392ffff640 10 client.24747.objecter ms_handle_connect 0x7f394809df40 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.064+0000 7f392ffff640 10 client.24747.objecter resend_mon_ops 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39380bb040 con 0x7f394809df40 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 10 monclient: dump: 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.093 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9432 (secure 0 0 0) 0x7f39380bc250 con 0x7f394809df40 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:02.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 msgr2=0x7f394809e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 0x7f394809e350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39380097c0 tx=0x7f39380191e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 msgr2=0x7f394809e350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f392ffff640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f394809df40 0x7f394809e350 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39380097c0 tx=0x7f39380191e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 309066, 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:02.095 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.097 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:02.098 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.099 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:02.100 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.101 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:02.102 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.103 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:02.106 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.107 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.108 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:02.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.113 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.116 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.117 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.120 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:02.120 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 10 librados: watch_flush enter 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 10 librados: watch_flush exit 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 20 client.24747.objecter shutdown clearing up homeless session... 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 10 client.24747.objecter successfully canceled tick 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f393403e580 msgr2=0x7f3934040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f393403e580 0x7f3934040a40 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 crypto rx=0x7f394813cb20 tx=0x7f3940009780 comp rx=0 tx=0).stop 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 10 monclient: shutdown 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 msgr2=0x7f39481d1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 --2- 172.21.15.105:0/2455364848 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f39480a4750 0x7f39481d1980 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c7f0 tx=0x7f393c00ccb0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f3947fff640 1 -- 172.21.15.105:0/2455364848 reap_dead start 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f3947fff640 1 -- 172.21.15.105:0/2455364848 reap_dead start 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 shutdown_connections 2024-09-17T16:40:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 >> 172.21.15.105:0/2455364848 conn(0x7f39480a93f0 msgr2=0x7f39480a71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 shutdown_connections 2024-09-17T16:40:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 -- 172.21.15.105:0/2455364848 wait complete. 2024-09-17T16:40:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.068+0000 7f394e214640 1 librados: shutdown 2024-09-17T16:40:02.123 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:02.123 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-17T16:40:02.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:40:02.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: get_monmap_and_config 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: monmap: 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:02.236316+0000 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:02.236316+0000 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:02.240 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:02.241 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 Processor -- start 2024-09-17T16:40:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 -- start start 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: init 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient: _add_conns ranks=[4,1,9,2,0,10,3,8,6,5,7] 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-c con 0x7f451c0a4640 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f451c06c620 addr v1:172.21.15.160:6789/0 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-a con 0x7f451c0a4f60 addr v2:172.21.15.105:3300/0 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c09e190 con 0x7f451c06c620 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f451c09da00 con 0x7f451c0a4f60 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f451c09deb0 con 0x7f451c0a4640 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f45220b5640 10 monclient(hunting): authenticate will time out at 1685.502218s 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451affd640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f451c06c620 legacy=0x7f451c1363f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42140/0 (socket says 172.21.15.105:42140) 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39674/0 (socket says 172.21.15.105:39674) 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 1 -- 172.21.15.105:0/1521403773 learned_addr learned my addr 172.21.15.105:0/1521403773 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): get_auth_request con 0x7f451c0a4f60 auth_method 0 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): get_auth_request con 0x7f451c0a4640 auth_method 0 2024-09-17T16:40:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient(hunting): handle_auth_done global_id 74335 payload 274 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 -- 172.21.15.105:0/1521403773 >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 msgr2=0x7f451c136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 --2- 172.21.15.105:0/1521403773 >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 -- 172.21.15.105:0/1521403773 >> v1:172.21.15.160:6789/0 conn(0x7f451c06c620 legacy=0x7f451c1363f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 monclient: found mon.noname-c 2024-09-17T16:40:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451b7fe640 1 -- 172.21.15.105:0/1521403773 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0a5410 con 0x7f451c0a4640 2024-09-17T16:40:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.232+0000 7f451bfff640 1 --2- 172.21.15.105:0/1521403773 >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451b7fe640 1 --2- 172.21.15.105:0/1521403773 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f450c0097f0 tx=0x7f450c01c210 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2d27c982699eb8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 1 -- 172.21.15.105:0/1521403773 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f450c0b0070 con 0x7f451c0a4640 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: dump: 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.254 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 5 monclient: authenticate success, global_id 74335 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 1 -- 172.21.15.105:0/1521403773 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c0aa810 con 0x7f451c0a4640 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 4 set_mon_vals no callback set 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 1 -- 172.21.15.105:0/1521403773 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f450c0a9300 con 0x7f451c0a4640 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:02.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451a7fc640 10 monclient: dump: 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.257 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: shutdown 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- 172.21.15.105:0/1521403773 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 msgr2=0x7f451c0a4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- 172.21.15.105:0/1521403773 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f450c0097f0 tx=0x7f450c01c210 comp rx=0 tx=0).stop 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- 172.21.15.105:0/1521403773 shutdown_connections 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- 172.21.15.105:0/1521403773 >> v2:172.21.15.105:3300/0 conn(0x7f451c0a4f60 0x7f451c136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- 172.21.15.105:0/1521403773 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f451c0a4640 0x7f451c0a4a20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- 172.21.15.105:0/1521403773 >> 172.21.15.105:0/1521403773 conn(0x7f451c068dd0 msgr2=0x7f451c0691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- 172.21.15.105:0/1521403773 shutdown_connections 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- 172.21.15.105:0/1521403773 wait complete. 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: monmap: 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.259 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 librados: starting msgr at 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 librados: starting objecter 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 Processor -- start 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- start start 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 librados: setting wanted keys 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 librados: calling monclient init 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: init 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient: _add_conns ranks=[8,7,1,6,4,2,5,0,3] 2024-09-17T16:40:02.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 0x7f451c1d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-e con 0x7f451c06c620 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f451c0a4640 0x7f451c1d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-i con 0x7f451c0a4640 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f451c0a4f60 0x7f451c13e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): picked mon.noname-f con 0x7f451c0a4f60 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f451c137250 con 0x7f451c0a4f60 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f451c09f340 con 0x7f451c0a4640 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f451c144370 con 0x7f451c06c620 2024-09-17T16:40:02.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f45220b5640 10 monclient(hunting): authenticate will time out at 1685.507896s 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.236+0000 7f451b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 0x7f451c1d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 0x7f451c1d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:43760/0 (socket says 172.21.15.105:43760) 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 -- 172.21.15.105:0/565937124 learned_addr learned my addr 172.21.15.105:0/565937124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): get_auth_request con 0x7f451c06c620 auth_method 0 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451bfff640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f451c0a4f60 msgr2=0x7f451c13e160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451bfff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f451c0a4f60 0x7f451c13e160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient(hunting): handle_auth_done global_id 54611 payload 824 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f451c0a4640 0x7f451c1d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f451c0a4640 msgr2=0x7f451c1d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f451c0a4640 0x7f451c1d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f451c0a4f60 msgr2=0x7f451c13e160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f451c0a4f60 0x7f451c13e160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 monclient: found mon.noname-e 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c13a610 con 0x7f451c06c620 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451b7fe640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 0x7f451c1d1690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f450c0ae040 tx=0x7f450c018fb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.?.objecter ms_handle_connect 0x7f451c06c620 2024-09-17T16:40:02.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f450c0b0040 con 0x7f451c06c620 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: dump: 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.264 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c0044b0 con 0x7f451c06c620 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f450c0aac90 con 0x7f451c06c620 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: dump: 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.266 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4519ffb640 4 set_mon_vals no callback set 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 5 monclient: authenticate success, global_id 54611 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 monclient: _renew_subs 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c120050 con 0x7f451c06c620 2024-09-17T16:40:02.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 client.54611.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 monclient: _renew_subs 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c0a01d0 con 0x7f451c06c620 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 1 librados: init done 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 0x7f451c13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f451c11f790 con 0x7f451c120290 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f450c004040 con 0x7f451c06c620 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f450803e550 0x7f4508040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 0x7f451c13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(450..450 src has 1..450) ==== 5817+0+0 (secure 0 0 0) 0x7f450c0e98a0 con 0x7f451c06c620 2024-09-17T16:40:02.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.54611.objecter ms_dispatch 0x7f451c000ba0 osd_map(450..450 src has 1..450) 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 3 client.54611.objecter handle_osd_map got epochs [450,450] > 0 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 3 client.54611.objecter handle_osd_map decoding full epoch 450 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 20 client.54611.objecter dump_active .. 0 homeless 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient: get_auth_request con 0x7f451c120290 auth_method 0 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451bfff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f450803e550 0x7f4508040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451bfff640 10 monclient: get_auth_request con 0x7f450803e550 auth_method 0 2024-09-17T16:40:02.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 10 monclient(hunting): handle_auth_done global_id 54669 payload 824 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f451c13fc00 con 0x7f451c120290 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.54611.objecter ms_handle_connect 0x7f451c120290 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.54611.objecter resend_mon_ops 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451affd640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 0x7f451c13e6a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f451c074c40 tx=0x7f451000c8e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 client.54611.objecter ms_handle_connect 0x7f450803e550 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f451bfff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f450803e550 0x7f4508040a10 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 crypto rx=0x7f4514007240 tx=0x7f451400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4510005070 con 0x7f451c120290 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.240+0000 7f4503fff640 10 monclient: dump: 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.270 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4510016070 con 0x7f451c120290 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 msgr2=0x7f451c13e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 0x7f451c13e6a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f451c074c40 tx=0x7f451000c8e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 msgr2=0x7f451c13e6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.244+0000 7f4503fff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c120290 0x7f451c13e6a0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f451c074c40 tx=0x7f451000c8e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f45220b5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f45220b5640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 0x7f451c13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f45220b5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f451c051e80 con 0x7f451c13ebe0 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 0x7f451c13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient: get_auth_request con 0x7f451c13ebe0 auth_method 0 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 10 monclient(hunting): handle_auth_done global_id 54678 payload 824 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.248+0000 7f451affd640 1 -- 172.21.15.105:0/565937124 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f451c0a2fb0 con 0x7f451c13ebe0 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f451affd640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 0x7f451c13eff0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4510015000 tx=0x7f4510005960 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 client.54611.objecter ms_handle_connect 0x7f451c13ebe0 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 client.54611.objecter resend_mon_ops 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f45100057b0 con 0x7f451c13ebe0 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 monclient: dump: 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9265 (secure 0 0 0) 0x7f4510005340 con 0x7f451c13ebe0 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 msgr2=0x7f451c13eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 0x7f451c13eff0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4510015000 tx=0x7f4510005960 comp rx=0 tx=0).stop 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 msgr2=0x7f451c13eff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f4503fff640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f451c13ebe0 0x7f451c13eff0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4510015000 tx=0x7f4510005960 comp rx=0 tx=0).stop 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:02.277 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 309260, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.278 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.279 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.282 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.283 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.284 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.285 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.286 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.287 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.288 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.289 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:02.290 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:02.292 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.293 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.294 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.297 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.298 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:02.299 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 10 librados: watch_flush enter 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 10 librados: watch_flush exit 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 20 client.54611.objecter shutdown clearing up homeless session... 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 10 client.54611.objecter successfully canceled tick 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f450803e550 msgr2=0x7f4508040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f450803e550 0x7f4508040a10 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 crypto rx=0x7f4514007240 tx=0x7f451400b040 comp rx=0 tx=0).stop 2024-09-17T16:40:02.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 10 monclient: shutdown 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 msgr2=0x7f451c1d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 --2- 172.21.15.105:0/565937124 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f451c06c620 0x7f451c1d1690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f450c0ae040 tx=0x7f450c018fb0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f451bfff640 1 -- 172.21.15.105:0/565937124 reap_dead start 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f451bfff640 1 -- 172.21.15.105:0/565937124 reap_dead start 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 shutdown_connections 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 >> 172.21.15.105:0/565937124 conn(0x7f451c068dd0 msgr2=0x7f451c141360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 shutdown_connections 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 -- 172.21.15.105:0/565937124 wait complete. 2024-09-17T16:40:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.252+0000 7f45220b5640 1 librados: shutdown 2024-09-17T16:40:02.302 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:02.302 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-17T16:40:02.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.412+0000 7fdf81e62640 10 monclient: get_monmap_and_config 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.412+0000 7fdf81e62640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.412+0000 7fdf81e62640 10 monclient: monmap: 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:02.419703+0000 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:02.419703+0000 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.423 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:02.424 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 Processor -- start 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 -- start start 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient: init 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient: _add_conns ranks=[10,2,9,5,6,8,4,3,0,7,1] 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 0x7fdf7c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-f con 0x7fdf7c0a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:40:02.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 0x7fdf7c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-b con 0x7fdf7c0a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf7c1397f0 0x7fdf7c139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-a con 0x7fdf7c1397f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c0a7c40 con 0x7fdf7c1397f0 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c0a7890 con 0x7fdf7c0a6a30 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c0a4080 con 0x7fdf7c0a4ab0 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf81e62640 10 monclient(hunting): authenticate will time out at 1685.685488s 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7affd640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 msgr2=0x7fdf7c0a7350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:40:02.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7affd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 0x7fdf7c0a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7b7fe640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 msgr2=0x7fdf7c0a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 0x7fdf7c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf7c1397f0 0x7fdf7c139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf7c1397f0 0x7fdf7c139ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42150/0 (socket says 172.21.15.105:42150) 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2788611877 learned_addr learned my addr 172.21.15.105:0/2788611877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): get_auth_request con 0x7fdf7c1397f0 auth_method 0 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_done global_id 36090 payload 274 2024-09-17T16:40:02.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 msgr2=0x7fdf7c0a7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 0x7fdf7c0a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2788611877 >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 msgr2=0x7fdf7c0a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2788611877 >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 0x7fdf7c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 monclient: found mon.noname-a 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:40:02.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.416+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2788611877 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0a37e0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf7c1397f0 0x7fdf7c139ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0a6570 tx=0x7fdf7401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8925485eb8d19f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 1 -- 172.21.15.105:0/2788611877 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf740090d0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: dump: 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.437 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 1 -- 172.21.15.105:0/2788611877 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf740a5bd0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 1 -- 172.21.15.105:0/2788611877 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf740a5dc0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 5 monclient: authenticate success, global_id 36090 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 4 set_mon_vals no callback set 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7a7fc640 10 monclient: dump: 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.439 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.440 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: shutdown 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf7c1397f0 msgr2=0x7fdf7c139ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf7c1397f0 0x7fdf7c139ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0a6570 tx=0x7fdf7401c8c0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- 172.21.15.105:0/2788611877 shutdown_connections 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf7c0a4ab0 0x7fdf7c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2788611877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf7c1397f0 0x7fdf7c139ca0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2788611877 >> v2:172.21.15.160:3300/0 conn(0x7fdf7c0a6a30 0x7fdf7c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- 172.21.15.105:0/2788611877 >> 172.21.15.105:0/2788611877 conn(0x7fdf7c0a8d00 msgr2=0x7fdf7c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- 172.21.15.105:0/2788611877 shutdown_connections 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- 172.21.15.105:0/2788611877 wait complete. 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: monmap: 2024-09-17T16:40:02.441 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.442 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 librados: starting msgr at 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 librados: starting objecter 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 Processor -- start 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- start start 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 librados: setting wanted keys 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 librados: calling monclient init 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: init 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient: _add_conns ranks=[5,1,4,8,0,7,3,2,6] 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c0a4ab0 0x7fdf7c1d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-h con 0x7fdf7c0a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7c0a6a30 0x7fdf7c1d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-f con 0x7fdf7c0a6a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 0x7fdf7c13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): picked mon.noname-c con 0x7fdf7c1397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c06e770 con 0x7fdf7c0a6a30 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdf7c0a2fc0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fdf7c143260 con 0x7fdf7c0a4ab0 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf81e62640 10 monclient(hunting): authenticate will time out at 1685.691544s 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 0x7fdf7c13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7affd640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7c0a6a30 msgr2=0x7fdf7c1d1b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:02.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 0x7fdf7c13d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39678/0 (socket says 172.21.15.105:39678) 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 learned_addr learned my addr 172.21.15.105:0/2665946654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): get_auth_request con 0x7fdf7c1397f0 auth_method 0 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7affd640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7c0a6a30 0x7fdf7c1d1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c0a4ab0 0x7fdf7c1d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.420+0000 7fdf7bfff640 10 monclient(hunting): handle_auth_done global_id 74344 payload 824 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c0a4ab0 msgr2=0x7fdf7c1d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c0a4ab0 0x7fdf7c1d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7c0a6a30 msgr2=0x7fdf7c1d1b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7c0a6a30 0x7fdf7c1d1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 monclient: found mon.noname-c 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0a9fb0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient: get_auth_request con 0x7fdf7c0a4ab0 auth_method 0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c0a4ab0 0x7fdf7c1d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7bfff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 0x7fdf7c13d440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf7401d160 tx=0x7fdf74019660 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.?.objecter ms_handle_connect 0x7fdf7c1397f0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf740bc030 con 0x7fdf7c1397f0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: dump: 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 5 monclient: authenticate success, global_id 74344 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74003e10 con 0x7fdf7c1397f0 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 monclient: _renew_subs 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c123dd0 con 0x7fdf7c1397f0 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf74004040 con 0x7fdf7c1397f0 2024-09-17T16:40:02.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: dump: 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf79ffb640 4 set_mon_vals no callback set 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 client.74344.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 monclient: _renew_subs 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c13ee40 con 0x7fdf7c1397f0 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 1 librados: init done 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 0x7fdf7c13d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fdf7c0aba60 con 0x7fdf7c123770 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdf740bf070 con 0x7fdf7c1397f0 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdf6803e550 0x7fdf68040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 0x7fdf7c13d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(450..450 src has 1..450) ==== 5817+0+0 (secure 0 0 0) 0x7fdf740bd070 con 0x7fdf7c1397f0 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.74344.objecter ms_dispatch 0x7fdf7c000ba0 osd_map(450..450 src has 1..450) 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 3 client.74344.objecter handle_osd_map got epochs [450,450] > 0 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 3 client.74344.objecter handle_osd_map decoding full epoch 450 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient: get_auth_request con 0x7fdf7c123770 auth_method 0 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 20 client.74344.objecter dump_active .. 0 homeless 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7affd640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdf6803e550 0x7fdf68040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7affd640 10 monclient: get_auth_request con 0x7fdf6803e550 auth_method 0 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_done global_id 74219 payload 824 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf7c139ca0 con 0x7fdf7c123770 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 0x7fdf7c13d980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000c6f0 tx=0x7fdf7000cbb0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.74344.objecter ms_handle_connect 0x7fdf7c123770 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.74344.objecter resend_mon_ops 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 client.74344.objecter ms_handle_connect 0x7fdf6803e550 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf7affd640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdf6803e550 0x7fdf68040a10 secure :-1 s=READY pgs=3579 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c13eba0 tx=0x7fdf6c006d10 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf70003e30 con 0x7fdf7c123770 2024-09-17T16:40:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: dump: 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdf70016070 con 0x7fdf7c123770 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 msgr2=0x7fdf7c13d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 0x7fdf7c13d980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000c6f0 tx=0x7fdf7000cbb0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 msgr2=0x7fdf7c13d980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.424+0000 7fdf63fff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c123770 0x7fdf7c13d980 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000c6f0 tx=0x7fdf7000cbb0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf81e62640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 0x7fdf7c13e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf81e62640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fdf7c051e80 con 0x7fdf7c13dec0 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 0x7fdf7c13e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient: get_auth_request con 0x7fdf7c13dec0 auth_method 0 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 10 monclient(hunting): handle_auth_done global_id 74228 payload 824 2024-09-17T16:40:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 1 -- 172.21.15.105:0/2665946654 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdf7c13e7e0 con 0x7fdf7c13dec0 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf7b7fe640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 0x7fdf7c13e2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000afd0 tx=0x7fdf700059e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 10 client.74344.objecter ms_handle_connect 0x7fdf7c13dec0 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 10 client.74344.objecter resend_mon_ops 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7001ce00 con 0x7fdf7c13dec0 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.432+0000 7fdf63fff640 10 monclient: dump: 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9264 (secure 0 0 0) 0x7fdf7001ce00 con 0x7fdf7c13dec0 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 msgr2=0x7fdf7c13e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 0x7fdf7c13e2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000afd0 tx=0x7fdf700059e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 msgr2=0x7fdf7c13e2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf63fff640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdf7c13dec0 0x7fdf7c13e2a0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000afd0 tx=0x7fdf700059e0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 41907, 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:02.460 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.462 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.462 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.462 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.462 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.463 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.463 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.463 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.464 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.465 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.466 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.468 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.470 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:02.472 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.473 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:02.474 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.475 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.476 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.477 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.478 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.480 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:02.482 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.483 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.484 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:02.485 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:02.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 10 librados: watch_flush enter 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 10 librados: watch_flush exit 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 20 client.74344.objecter shutdown clearing up homeless session... 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 10 client.74344.objecter successfully canceled tick 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdf6803e550 msgr2=0x7fdf68040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdf6803e550 0x7fdf68040a10 secure :-1 s=READY pgs=3579 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c13eba0 tx=0x7fdf6c006d10 comp rx=0 tx=0).stop 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 10 monclient: shutdown 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 msgr2=0x7fdf7c13d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 --2- 172.21.15.105:0/2665946654 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdf7c1397f0 0x7fdf7c13d440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf7401d160 tx=0x7fdf74019660 comp rx=0 tx=0).stop 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 reap_dead start 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf7bfff640 1 -- 172.21.15.105:0/2665946654 reap_dead start 2024-09-17T16:40:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 shutdown_connections 2024-09-17T16:40:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 >> 172.21.15.105:0/2665946654 conn(0x7fdf7c0a8d00 msgr2=0x7fdf7c0a4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 shutdown_connections 2024-09-17T16:40:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 -- 172.21.15.105:0/2665946654 wait complete. 2024-09-17T16:40:02.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.436+0000 7fdf81e62640 1 librados: shutdown 2024-09-17T16:40:02.488 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:02.488 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-17T16:40:02.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:40:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 10 monclient: get_monmap_and_config 2024-09-17T16:40:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 10 monclient: monmap: 2024-09-17T16:40:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.609 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:40:02.606071+0000 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:40:02.606071+0000 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:40:02.611 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:40:02.611 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 1 Processor -- start 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 1 -- start start 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.600+0000 7fc7ab3ea640 10 monclient: init 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient: _add_conns ranks=[7,3,4,9,1,0,8,2,6,5,10] 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a4750 0x7fc7a40a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-i con 0x7fc7a40a4750 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 0x7fc7a40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-g con 0x7fc7a40a0ae0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-c con 0x7fc7a409e130 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a409e5b0 con 0x7fc7a40a0ae0 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fc7a40a55b0 con 0x7fc7a409e130 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc7a40a62b0 con 0x7fc7a40a4750 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient(hunting): authenticate will time out at 1685.872679s 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:39692/0 (socket says 172.21.15.105:39692) 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 -- 172.21.15.105:0/17091182 learned_addr learned my addr 172.21.15.105:0/17091182 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): get_auth_request con 0x7fc7a409e130 auth_method 0 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a8985640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 0x7fc7a40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient(hunting): handle_auth_done global_id 74353 payload 274 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 -- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a4750 msgr2=0x7fc7a40a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a4750 0x7fc7a40a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 -- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 msgr2=0x7fc7a40a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 0x7fc7a40a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 monclient: found mon.noname-c 2024-09-17T16:40:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:40:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 -- 172.21.15.105:0/17091182 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a409e7c0 con 0x7fc7a409e130 2024-09-17T16:40:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a8985640 10 monclient: get_auth_request con 0x7fc7a40a0ae0 auth_method 0 2024-09-17T16:40:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a8985640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 0x7fc7a40a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7a9987640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d1b0 tx=0x7fc79c01e7f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1f8f3d026ba04c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 1 -- 172.21.15.105:0/17091182 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c0ae020 con 0x7fc7a409e130 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: dump: 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 1 -- 172.21.15.105:0/17091182 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c009600 con 0x7fc7a409e130 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 1 -- 172.21.15.105:0/17091182 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c009ca0 con 0x7fc7a409e130 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 5 monclient: authenticate success, global_id 74353 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 10 monclient: get_monmap_and_config success 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc7ab3ea640 4 set_mon_vals no callback set 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.604+0000 7fc79bfff640 10 monclient: dump: 2024-09-17T16:40:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: shutdown 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/17091182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 msgr2=0x7fc7a41365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d1b0 tx=0x7fc79c01e7f0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/17091182 shutdown_connections 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fc7a409e130 0x7fc7a41365e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a40a0ae0 0x7fc7a40a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/17091182 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a4750 0x7fc7a40a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/17091182 >> 172.21.15.105:0/17091182 conn(0x7fc7a40a93f0 msgr2=0x7fc7a40a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/17091182 shutdown_connections 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/17091182 wait complete. 2024-09-17T16:40:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: build_initial_monmap 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: monmap: 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:40:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 librados: starting msgr at 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 librados: starting objecter 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 Processor -- start 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- start start 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 librados: setting wanted keys 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 librados: calling monclient init 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: init 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _reopen_session rank -1 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _add_conns ranks=[7,2,1,0,3,5,8,4,6] 2024-09-17T16:40:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 0x7fc7a41cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-i con 0x7fc7a409e130 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40a4750 0x7fc7a41cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a40a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a413a190 0x7fc7a413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): picked mon.noname-f con 0x7fc7a413a190 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a406ccb0 con 0x7fc7a413a190 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a406cb30 con 0x7fc7a40a4750 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc7a406ce30 con 0x7fc7a409e130 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): _renew_subs 2024-09-17T16:40:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient(hunting): authenticate will time out at 1685.878003s 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a8985640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40a4750 msgr2=0x7fc7a41cde20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a8985640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40a4750 0x7fc7a41cde20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9987640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a413a190 msgr2=0x7fc7a413a5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9987640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a413a190 0x7fc7a413a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 0x7fc7a41cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 0x7fc7a41cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:41702/0 (socket says 172.21.15.105:41702) 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 -- 172.21.15.105:0/700506238 learned_addr learned my addr 172.21.15.105:0/700506238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): get_auth_request con 0x7fc7a409e130 auth_method 0 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient(hunting): handle_auth_done global_id 64465 payload 824 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40a4750 msgr2=0x7fc7a41cde20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a40a4750 0x7fc7a41cde20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a413a190 msgr2=0x7fc7a413a5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7a413a190 0x7fc7a413a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient: _finish_hunting 0 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 monclient: found mon.noname-i 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a413c0d0 con 0x7fc7a409e130 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7a9186640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 0x7fc7a41cd8e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000b410 tx=0x7fc7a000b8d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 client.?.objecter ms_handle_connect 0x7fc7a409e130 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7a0015020 con 0x7fc7a409e130 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: dump: 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.633 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: _finish_auth 0 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 5 monclient: authenticate success, global_id 64465 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _renew_subs 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:40:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a4120ae0 con 0x7fc7a409e130 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a0011690 con 0x7fc7a409e130 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 client.64465.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _renew_subs 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a40a87b0 con 0x7fc7a409e130 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc7ab3ea640 1 librados: init done 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc79b7fe640 4 set_mon_vals no callback set 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7a0011cb0 con 0x7fc7a409e130 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.608+0000 7fc799ffb640 10 monclient: dump: 2024-09-17T16:40:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.636 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7ab3ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 0x7fc7a4120480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc7a4051e80 con 0x7fc7a40a0a70 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 0x7fc7a4120480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient: get_auth_request con 0x7fc7a40a0a70 auth_method 0 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc7a0018020 con 0x7fc7a409e130 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc78803e550 0x7fc788040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(450..450 src has 1..450) ==== 5817+0+0 (secure 0 0 0) 0x7fc7a004fca0 con 0x7fc7a409e130 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 client.64465.objecter ms_dispatch 0x7fc7a4000ba0 osd_map(450..450 src has 1..450) 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 3 client.64465.objecter handle_osd_map got epochs [450,450] > 0 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 3 client.64465.objecter handle_osd_map decoding full epoch 450 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 20 client.64465.objecter dump_active .. 0 homeless 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a9987640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc78803e550 0x7fc788040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a9987640 10 monclient: get_auth_request con 0x7fc78803e550 auth_method 0 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 10 monclient(hunting): handle_auth_done global_id 64474 payload 824 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7a41365e0 con 0x7fc7a40a0a70 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a8985640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 0x7fc7a4120480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7a413c860 tx=0x7fc79400cb80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 client.64465.objecter ms_handle_connect 0x7fc7a40a0a70 2024-09-17T16:40:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 client.64465.objecter resend_mon_ops 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 client.64465.objecter ms_handle_connect 0x7fc78803e550 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc7a9987640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc78803e550 0x7fc788040a10 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a9000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc794015070 con 0x7fc7a40a0a70 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 monclient: dump: 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc794007660 con 0x7fc7a40a0a70 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 msgr2=0x7fc7a4120480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 0x7fc7a4120480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7a413c860 tx=0x7fc79400cb80 comp rx=0 tx=0).stop 2024-09-17T16:40:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 msgr2=0x7fc7a4120480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.612+0000 7fc799ffb640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a40a0a70 0x7fc7a4120480 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7a413c860 tx=0x7fc79400cb80 comp rx=0 tx=0).stop 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 0x7fc7a413af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 monclient(hunting): start opening mon connection 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fc7a40008e0 con 0x7fc7a413ab30 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 0x7fc7a413af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient: get_auth_request con 0x7fc7a413ab30 auth_method 0 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:40:02.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 10 monclient(hunting): handle_auth_done global_id 64483 payload 824 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 1 -- 172.21.15.105:0/700506238 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7a4120f00 con 0x7fc7a413ab30 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7a8985640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 0x7fc7a413af40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc794016000 tx=0x7fc794004740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 client.64465.objecter ms_handle_connect 0x7fc7a413ab30 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 client.64465.objecter resend_mon_ops 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7940233f0 con 0x7fc7a413ab30 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 monclient: dump: 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:40:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:40:02.645 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9265 (secure 0 0 0) 0x7fc7940233f0 con 0x7fc7a413ab30 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 msgr2=0x7fc7a413af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 0x7fc7a413af40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc794016000 tx=0x7fc794004740 comp rx=0 tx=0).stop 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 msgr2=0x7fc7a413af40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc799ffb640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a413ab30 0x7fc7a413af40 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc794016000 tx=0x7fc794004740 comp rx=0 tx=0).stop 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:40:02.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 80, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 175726, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.647 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.648 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.649 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:40:02.650 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:40:02.651 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.652 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.653 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.655 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.658 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.660 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.661 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:40:02.663 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:40:02.666 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:40:02.667 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:40:02.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:40:02.669 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:40:02.669 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:40:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 librados: watch_flush enter 2024-09-17T16:40:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 librados: watch_flush exit 2024-09-17T16:40:02.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 20 client.64465.objecter shutdown clearing up homeless session... 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 client.64465.objecter successfully canceled tick 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc78803e550 msgr2=0x7fc788040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc78803e550 0x7fc788040a10 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c0a9000 comp rx=0 tx=0).stop 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 10 monclient: shutdown 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 msgr2=0x7fc7a41cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.620+0000 7fc7ab3ea640 1 --2- 172.21.15.105:0/700506238 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fc7a409e130 0x7fc7a41cd8e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000b410 tx=0x7fc7a000b8d0 comp rx=0 tx=0).stop 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7a9987640 1 -- 172.21.15.105:0/700506238 reap_dead start 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7a9987640 1 -- 172.21.15.105:0/700506238 reap_dead start 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 shutdown_connections 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 >> 172.21.15.105:0/700506238 conn(0x7fc7a40a93f0 msgr2=0x7fc7a40a2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 shutdown_connections 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7ab3ea640 1 -- 172.21.15.105:0/700506238 wait complete. 2024-09-17T16:40:02.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:40:02.624+0000 7fc7ab3ea640 1 librados: shutdown 2024-09-17T16:40:02.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-17T16:40:02.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:41:32.673 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-17T16:41:32.673 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-17T16:41:32.674 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-17T16:41:32.677 INFO:tasks.ceph.mon.f:Started 2024-09-17T16:41:32.677 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-17T16:41:32.677 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T16:41:32.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T16:41:32.680 INFO:tasks.ceph.mon.b:Started 2024-09-17T16:41:32.680 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:41:32.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:41:32.797 INFO:tasks.ceph.mon.b.smithi105.stderr:2024-09-17T16:41:32.788+0000 7fb04b8c3e00 -1 mon.b@-1(???).log v0 log_external_backlog rewinding external_log_to from 480 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:41:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.788+0000 7fd8cee4b640 10 monclient: get_monmap_and_config 2024-09-17T16:41:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.788+0000 7fd8cee4b640 10 monclient: build_initial_monmap 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.788+0000 7fd8cee4b640 10 monclient: monmap: 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:32.795284+0000 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:32.795284+0000 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:32.800 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 Processor -- start 2024-09-17T16:41:32.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 -- start start 2024-09-17T16:41:32.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient: init 2024-09-17T16:41:32.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient: _add_conns ranks=[1,9,3,2,5,6,10,7,0,4,8] 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd8c8139d20 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd8c80a5da0 0x7fd8c813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-a con 0x7fd8c80a5da0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c8141570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-g con 0x7fd8c813f180 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c80a61a0 con 0x7fd8c8139d20 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c80a4110 con 0x7fd8c80a5da0 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c813a810 con 0x7fd8c813f180 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cee4b640 10 monclient(hunting): authenticate will time out at 1776.061321s 2024-09-17T16:41:32.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8ccbe7640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd8c8139d20 0x7fd8c813a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:60018/0 (socket says 172.21.15.105:60018) 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8ccbe7640 1 -- 172.21.15.105:0/375988813 learned_addr learned my addr 172.21.15.105:0/375988813 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/375988813 >> v2:172.21.15.105:3300/0 conn(0x7fd8c80a5da0 0x7fd8c813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 --2- 172.21.15.105:0/375988813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c8141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): get_auth_request con 0x7fd8c813f180 auth_method 0 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): get_auth_request con 0x7fd8c80a5da0 auth_method 0 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:32.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_done global_id 54714 payload 274 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 -- 172.21.15.105:0/375988813 >> v2:172.21.15.105:3300/0 conn(0x7fd8c80a5da0 msgr2=0x7fd8c813ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 --2- 172.21.15.105:0/375988813 >> v2:172.21.15.105:3300/0 conn(0x7fd8c80a5da0 0x7fd8c813ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 -- 172.21.15.105:0/375988813 >> v1:172.21.15.160:6789/0 conn(0x7fd8c8139d20 legacy=0x7fd8c813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient: _finish_hunting 0 2024-09-17T16:41:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 monclient: found mon.noname-g 2024-09-17T16:41:32.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:32.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:41:32.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 -- 172.21.15.105:0/375988813 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c80745d0 con 0x7fd8c813f180 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.792+0000 7fd8cd3e8640 1 --2- 172.21.15.105:0/375988813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c8141570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009d90 tx=0x7fd8c001c830 comp rx=0 tx=0).ready entity=mon.3 client_cookie=482e2e7d7f24f96a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 1 -- 172.21.15.105:0/375988813 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8c00b2070 con 0x7fd8c813f180 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: dump: 2024-09-17T16:41:32.815 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:32.816 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 1 -- 172.21.15.105:0/375988813 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c00a5a50 con 0x7fd8c813f180 2024-09-17T16:41:32.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 5 monclient: authenticate success, global_id 54714 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 4 set_mon_vals no callback set 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 1 -- 172.21.15.105:0/375988813 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8c00a5c80 con 0x7fd8c813f180 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c77fe640 10 monclient: dump: 2024-09-17T16:41:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: shutdown 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/375988813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 msgr2=0x7fd8c8141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/375988813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c8141570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009d90 tx=0x7fd8c001c830 comp rx=0 tx=0).stop 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/375988813 shutdown_connections 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/375988813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c8141570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/375988813 >> v2:172.21.15.105:3300/0 conn(0x7fd8c80a5da0 0x7fd8c813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/375988813 >> 172.21.15.105:0/375988813 conn(0x7fd8c80aa250 msgr2=0x7fd8c80aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/375988813 shutdown_connections 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/375988813 wait complete. 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: build_initial_monmap 2024-09-17T16:41:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: monmap: 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 librados: starting msgr at 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 librados: starting objecter 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 Processor -- start 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- start start 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 librados: setting wanted keys 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 librados: calling monclient init 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: init 2024-09-17T16:41:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _add_conns ranks=[6,7,3,1,0,5,8,4,2] 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-d con 0x7fd8c80a5da0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8c8139d20 0x7fd8c813af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-i con 0x7fd8c8139d20 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c813b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): picked mon.noname-g con 0x7fd8c813f180 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c806da30 con 0x7fd8c813f180 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd8c80a0c50 con 0x7fd8c80a5da0 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fd8c81417a0 con 0x7fd8c8139d20 2024-09-17T16:41:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient(hunting): authenticate will time out at 1776.066764s 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58290/0 (socket says 172.21.15.105:58290) 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 -- 172.21.15.105:0/1358369054 learned_addr learned my addr 172.21.15.105:0/1358369054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): get_auth_request con 0x7fd8c80a5da0 auth_method 0 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8c8139d20 0x7fd8c813af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c813b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): get_auth_request con 0x7fd8c8139d20 auth_method 0 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): get_auth_request con 0x7fd8c813f180 auth_method 0 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cd3e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient(hunting): handle_auth_done global_id 24783 payload 824 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 -- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8c8139d20 msgr2=0x7fd8c813af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8c8139d20 0x7fd8c813af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 -- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 msgr2=0x7fd8c813b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c813b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient: _finish_hunting 0 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 monclient: found mon.noname-d 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 -- 172.21.15.105:0/1358369054 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c8073b60 con 0x7fd8c80a5da0 2024-09-17T16:41:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8ccbe7640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c480 tx=0x7fd8bc00c940 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 client.?.objecter ms_handle_connect 0x7fd8c80a5da0 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8bc017070 con 0x7fd8c80a5da0 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: dump: 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 5 monclient: authenticate success, global_id 24783 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8bc007070 con 0x7fd8c80a5da0 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _renew_subs 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c813d460 con 0x7fd8c80a5da0 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8bc007710 con 0x7fd8c80a5da0 2024-09-17T16:41:32.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c6ffd640 4 set_mon_vals no callback set 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 client.24783.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8c57fa640 10 monclient: dump: 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _renew_subs 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c813bc60 con 0x7fd8c80a5da0 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.796+0000 7fd8cee4b640 1 librados: init done 2024-09-17T16:41:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8cee4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8cee4b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8cee4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c813eb60 con 0x7fd8c80a5da0 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd8bc0072a0 con 0x7fd8c80a5da0 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 0x7fd8ac040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(540..540 src has 1..540) ==== 6657+0+0 (secure 0 0 0) 0x7fd8bc04f260 con 0x7fd8c80a5da0 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 10 client.24783.objecter ms_dispatch 0x7fd8c8000ba0 osd_map(540..540 src has 1..540) 2024-09-17T16:41:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 3 client.24783.objecter handle_osd_map got epochs [540,540] > 0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 3 client.24783.objecter handle_osd_map decoding full epoch 540 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 20 client.24783.objecter dump_active .. 0 homeless 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 0x7fd8ac040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c7fff640 10 monclient: get_auth_request con 0x7fd8ac03e5b0 auth_method 0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c57fa640 10 client.24783.objecter ms_handle_connect 0x7fd8ac03e5b0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.800+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 0x7fd8ac040a70 secure :-1 s=READY pgs=3587 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8073080 tx=0x7fd8b4009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.804+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8bc01ad50 con 0x7fd8c80a5da0 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.804+0000 7fd8c57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:32.804+0000 7fd8c57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8cee4b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:41:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8cee4b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8cee4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd8c813a100 con 0x7fd8c80a5da0 2024-09-17T16:41:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8c57fa640 1 -- 172.21.15.105:0/1358369054 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3201 (secure 0 0 0) 0x7fd8bc005cd0 con 0x7fd8c80a5da0 2024-09-17T16:41:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8c57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.068+0000 7fd8c57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:33.080 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":80,"quorum":[0,3,4,5,6,7,8],"quorum_names":["a","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":92,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:41:33.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 10 librados: watch_flush enter 2024-09-17T16:41:33.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 10 librados: watch_flush exit 2024-09-17T16:41:33.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 20 client.24783.objecter shutdown clearing up homeless session... 2024-09-17T16:41:33.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 10 client.24783.objecter successfully canceled tick 2024-09-17T16:41:33.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 msgr2=0x7fd8ac040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:33.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 0x7fd8ac040a70 secure :-1 s=READY pgs=3587 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8073080 tx=0x7fd8b4009190 comp rx=0 tx=0).stop 2024-09-17T16:41:33.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 10 monclient: shutdown 2024-09-17T16:41:33.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:33.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 msgr2=0x7fd8c813aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c480 tx=0x7fd8bc00c940 comp rx=0 tx=0).stop 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 shutdown_connections 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8ac03e5b0 0x7fd8ac040a70 unknown :-1 s=CLOSED pgs=3587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8c813f180 0x7fd8c813b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fd8c8139d20 0x7fd8c813af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 --2- 172.21.15.105:0/1358369054 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8c80a5da0 0x7fd8c813aa10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:33.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 >> 172.21.15.105:0/1358369054 conn(0x7fd8c80aa250 msgr2=0x7fd8c806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:33.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 shutdown_connections 2024-09-17T16:41:33.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 -- 172.21.15.105:0/1358369054 wait complete. 2024-09-17T16:41:33.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:33.072+0000 7fd8cee4b640 1 librados: shutdown 2024-09-17T16:41:36.100 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:41:36.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: get_monmap_and_config 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: build_initial_monmap 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: monmap: 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:36.225163+0000 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:36.225163+0000 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:36.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 Processor -- start 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 -- start start 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: init 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient: _add_conns ranks=[9,8,4,0,10,6,5,3,7,1,2] 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-a con 0x7f01ac0a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01ac0a4910 0x7f01ac0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-e con 0x7f01ac0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac1396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac0a3fc0 con 0x7f01ac0a68e0 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f01ac0a63c0 con 0x7f01ac1396a0 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f01ac0a7740 con 0x7f01ac0a4910 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01b13b6640 10 monclient(hunting): authenticate will time out at 1779.490896s 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55340/0 (socket says 172.21.15.105:55340) 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 -- 172.21.15.105:0/1110547820 learned_addr learned my addr 172.21.15.105:0/1110547820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:41152/0 (socket says 172.21.15.105:41152) 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): get_auth_request con 0x7f01ac0a68e0 auth_method 0 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): get_auth_request con 0x7f01ac1396a0 auth_method 0 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01ac0a4910 0x7f01ac0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): get_auth_request con 0x7f01ac0a4910 auth_method 0 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient(hunting): handle_auth_done global_id 36117 payload 274 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 -- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01ac0a4910 msgr2=0x7f01ac0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01ac0a4910 0x7f01ac0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 -- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 msgr2=0x7f01ac139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 monclient: found mon.noname-a 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01aaffd640 1 -- 172.21.15.105:0/1110547820 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac06ebc0 con 0x7f01ac0a68e0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.220+0000 7f01ab7fe640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01aaffd640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f019c009870 tx=0x7f019c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfae08e1771573d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 1 -- 172.21.15.105:0/1110547820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f019c0b2070 con 0x7f01ac0a68e0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: dump: 2024-09-17T16:41:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:36.238 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:41:36.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 5 monclient: authenticate success, global_id 36117 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 1 -- 172.21.15.105:0/1110547820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c0ab7b0 con 0x7f01ac0a68e0 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 4 set_mon_vals no callback set 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 1 -- 172.21.15.105:0/1110547820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f019c0abe50 con 0x7f01ac0a68e0 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01a9ffb640 10 monclient: dump: 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:36.242 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: shutdown 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 msgr2=0x7f01ac0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f019c009870 tx=0x7f019c01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- 172.21.15.105:0/1110547820 shutdown_connections 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac1396a0 0x7f01ac139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f01ac0a4910 0x7f01ac0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- 172.21.15.105:0/1110547820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac0a6cc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- 172.21.15.105:0/1110547820 >> 172.21.15.105:0/1110547820 conn(0x7f01ac068ba0 msgr2=0x7f01ac068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- 172.21.15.105:0/1110547820 shutdown_connections 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- 172.21.15.105:0/1110547820 wait complete. 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: build_initial_monmap 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: monmap: 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:36.248 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 librados: starting msgr at 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 librados: starting objecter 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 Processor -- start 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- start start 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 librados: setting wanted keys 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 librados: calling monclient init 2024-09-17T16:41:36.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: init 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient: _add_conns ranks=[4,0,2,5,3,6,1,7,8] 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac0a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac1d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-a con 0x7f01ac0a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): picked mon.noname-b con 0x7f01ac1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac06e630 con 0x7f01ac0a68e0 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac0a7af0 con 0x7f01ac1396a0 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:36.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f01ac143320 con 0x7f01ac0a4910 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.224+0000 7f01b13b6640 10 monclient(hunting): authenticate will time out at 1779.495946s 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:41164/0 (socket says 172.21.15.105:41164) 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 -- 172.21.15.105:0/3839783536 learned_addr learned my addr 172.21.15.105:0/3839783536 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): get_auth_request con 0x7f01ac0a4910 auth_method 0 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 10 monclient(hunting): get_auth_request con 0x7f01ac1396a0 auth_method 0 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac1d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): get_auth_request con 0x7f01ac0a68e0 auth_method 0 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:36.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 msgr2=0x7f01ac13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 msgr2=0x7f01ac13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01ab7fe640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient(hunting): handle_auth_done global_id 74407 payload 824 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 msgr2=0x7f01ac13d720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 msgr2=0x7f01ac1d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac1d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 monclient: found mon.noname-c 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 -- 172.21.15.105:0/3839783536 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac13fc40 con 0x7f01ac0a4910 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac1d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aaffd640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f019c0097c0 tx=0x7f019c019190 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 client.?.objecter ms_handle_connect 0x7f01ac0a4910 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f019c0b2070 con 0x7f01ac0a4910 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:36.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: dump: 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:36.255 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c0abdd0 con 0x7f01ac0a4910 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 5 monclient: authenticate success, global_id 74407 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _renew_subs 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac0a9fe0 con 0x7f01ac0a4910 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f019c003a90 con 0x7f01ac0a4910 2024-09-17T16:41:36.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 monclient: dump: 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:36.257 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01a97fa640 4 set_mon_vals no callback set 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 client.74407.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _renew_subs 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac077b80 con 0x7f01ac0a4910 2024-09-17T16:41:36.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 1 librados: init done 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac139b50 con 0x7f01ac0a4910 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f019c0bf060 con 0x7f01ac0a4910 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 0x7f0198040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f019c0e9f20 con 0x7f01ac0a4910 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 client.74407.objecter ms_dispatch 0x7f01ac000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 3 client.74407.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 3 client.74407.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:36.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 20 client.74407.objecter dump_active .. 0 homeless 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 0x7f0198040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 10 monclient: get_auth_request con 0x7f019803e540 auth_method 0 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01937fe640 10 client.74407.objecter ms_handle_connect 0x7f019803e540 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.228+0000 7f01aa7fc640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 0x7f0198040a00 secure :-1 s=READY pgs=3589 cs=0 l=1 rev1=1 crypto rx=0x7f01ac13d6b0 tx=0x7f01a0009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.232+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f01ac0a9fe0 con 0x7f01ac0a4910 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.232+0000 7f01937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:36.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.232+0000 7f01937fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:36.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:41:36.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:36.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f01ac0746d0 con 0x7f01ac0a4910 2024-09-17T16:41:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01937fe640 1 -- 172.21.15.105:0/3839783536 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3201 (secure 0 0 0) 0x7f01ac13fc40 con 0x7f01ac0a4910 2024-09-17T16:41:36.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:36.472 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":80,"quorum":[0,3,4,5,6,7,8],"quorum_names":["a","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":95,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:41:36.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01937fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 librados: watch_flush enter 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 librados: watch_flush exit 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 20 client.74407.objecter shutdown clearing up homeless session... 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 client.74407.objecter successfully canceled tick 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 msgr2=0x7f0198040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 0x7f0198040a00 secure :-1 s=READY pgs=3589 cs=0 l=1 rev1=1 crypto rx=0x7f01ac13d6b0 tx=0x7f01a0009190 comp rx=0 tx=0).stop 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 10 monclient: shutdown 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 msgr2=0x7f01ac1d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f019c0097c0 tx=0x7f019c019190 comp rx=0 tx=0).stop 2024-09-17T16:41:36.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 shutdown_connections 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f019803e540 0x7f0198040a00 unknown :-1 s=CLOSED pgs=3589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f01ac1396a0 0x7f01ac13d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac0a68e0 0x7f01ac1d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 --2- 172.21.15.105:0/3839783536 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f01ac0a4910 0x7f01ac1d14a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 >> 172.21.15.105:0/3839783536 conn(0x7f01ac068ba0 msgr2=0x7f01ac0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:36.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 shutdown_connections 2024-09-17T16:41:36.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 -- 172.21.15.105:0/3839783536 wait complete. 2024-09-17T16:41:36.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:36.464+0000 7f01b13b6640 1 librados: shutdown 2024-09-17T16:41:39.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:41:39.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: get_monmap_and_config 2024-09-17T16:41:39.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: build_initial_monmap 2024-09-17T16:41:39.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: monmap: 2024-09-17T16:41:39.617 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:39.613804+0000 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:39.613804+0000 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:39.619 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 Processor -- start 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 -- start start 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: init 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient: _add_conns ranks=[9,8,2,10,4,0,7,6,1,5,3] 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-a con 0x7f3c800a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-e con 0x7f3c800a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-b con 0x7f3c801397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:39.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3c800a7c40 con 0x7f3c800a6a30 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800a7890 con 0x7f3c801397f0 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3c800a4080 con 0x7f3c800a4ab0 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c87186640 10 monclient(hunting): authenticate will time out at 1782.879505s 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59482/0 (socket says 172.21.15.105:59482) 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 learned_addr learned my addr 172.21.15.105:0/2236307364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c800a4ab0 auth_method 0 2024-09-17T16:41:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c801397f0 auth_method 0 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c800a6a30 auth_method 0 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.608+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:39.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c801397f0 auth_method 0 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c800a4ab0 auth_method 0 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:39.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:39.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.812+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:39.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:39.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c800a6a30 auth_method 0 2024-09-17T16:41:39.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:39.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:39.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:39.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:39.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:39.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:39.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:39.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c801397f0 auth_method 0 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c800a4ab0 auth_method 0 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.216+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c800a6a30 auth_method 0 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:40.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c801397f0 auth_method 0 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c800a4ab0 auth_method 0 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.020+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c800a6a30 auth_method 0 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:41.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): tick 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 msgr2=0x7f3c800a7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:42.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c800a4ab0 0x7f3c800a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 msgr2=0x7f3c80139ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c801397f0 0x7f3c80139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 msgr2=0x7f3c800a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.105:3300/0 conn(0x7f3c800a6a30 0x7f3c800a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient: _add_conns ranks=[3,4,6,1,0,10,7,5] 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-g con 0x7f3c68003830 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-c con 0x7f3c68006240 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-d con 0x7f3c68008dc0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:42.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800a7c40 con 0x7f3c68003830 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f3c800a7890 con 0x7f3c68006240 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3c800a4080 con 0x7f3c68008dc0 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.608+0000 7f3c767fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c68008dc0 auth_method 0 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c68006240 auth_method 0 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c68003830 auth_method 0 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c68008dc0 auth_method 0 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c68006240 auth_method 0 2024-09-17T16:41:42.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c68003830 auth_method 0 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:42.816+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:41:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c68006240 auth_method 0 2024-09-17T16:41:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:43.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c68008dc0 auth_method 0 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:43.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c68003830 auth_method 0 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:43.220+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c68006240 auth_method 0 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c68008dc0 auth_method 0 2024-09-17T16:41:44.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c85723640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c68003830 auth_method 0 2024-09-17T16:41:44.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:44.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:44.024+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:41:45.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): tick 2024-09-17T16:41:45.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:41:45.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 monclient(hunting): continuing hunt 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 msgr2=0x7f3c6800b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c68008dc0 0x7f3c6800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 msgr2=0x7f3c680086d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f3c68006240 0x7f3c680086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 msgr2=0x7f3c68005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c85723640 1 -- 172.21.15.105:0/2236307364 reap_dead start 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient: _add_conns ranks=[10,7,0,5,1] 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-f con 0x7f3c680066c0 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:45.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 0x7f3c68009b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-i con 0x7f3c68009700 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c6800a0e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c800a37e0 con 0x7f3c6800a0e0 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3c800a7c40 con 0x7f3c680066c0 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 1 -- 172.21.15.105:0/2236307364 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3c800a7890 con 0x7f3c68009700 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c767fc640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 0x7f3c68009b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c680066c0 auth_method 0 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c68009700 auth_method 0 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 msgr2=0x7f3c680091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:41:45.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 msgr2=0x7f3c680091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c84f22640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 1 -- 172.21.15.105:0/2236307364 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244009243 0 0) 0x7f3c800a37e0 con 0x7f3c6800a0e0 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 10 monclient(hunting): _negotiate 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 10 monclient(hunting): my global_id is 36135 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c777fe640 1 -- 172.21.15.105:0/2236307364 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c6c003530 con 0x7f3c6800a0e0 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient(hunting): handle_auth_done global_id 64519 payload 274 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 msgr2=0x7f3c680091c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:45.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 >> v1:172.21.15.105:6789/0 conn(0x7f3c6800a0e0 legacy=0x7f3c6800a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 monclient: found mon.noname-i 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:41:45.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.612+0000 7f3c77fff640 1 -- 172.21.15.105:0/2236307364 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6800aeb0 con 0x7f3c68009700 2024-09-17T16:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.616+0000 7f3c77fff640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 0x7f3c68009b70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c800a7e50 tx=0x7f3c7001ddc0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=35d84321bade885c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.616+0000 7f3c777fe640 1 -- 172.21.15.105:0/2236307364 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c700b3070 con 0x7f3c68009700 2024-09-17T16:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.616+0000 7f3c777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.616+0000 7f3c777fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.616+0000 7f3c777fe640 10 monclient: dump: 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 1 -- 172.21.15.105:0/2236307364 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c700173e0 con 0x7f3c68009700 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 5 monclient: authenticate success, global_id 64519 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 4 set_mon_vals no callback set 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 1 -- 172.21.15.105:0/2236307364 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c7000f040 con 0x7f3c68009700 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c777fe640 10 monclient: dump: 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 10 monclient: shutdown 2024-09-17T16:41:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 -- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 msgr2=0x7f3c68009b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 0x7f3c68009b70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c800a7e50 tx=0x7f3c7001ddc0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 -- 172.21.15.105:0/2236307364 shutdown_connections 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f3c68009700 0x7f3c68009b70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 --2- 172.21.15.105:0/2236307364 >> v2:172.21.15.160:3300/0 conn(0x7f3c680066c0 0x7f3c680091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 --2- 172.21.15.105:0/2236307364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3c68003830 0x7f3c68005c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 -- 172.21.15.105:0/2236307364 >> 172.21.15.105:0/2236307364 conn(0x7f3c800a8d00 msgr2=0x7f3c800a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 -- 172.21.15.105:0/2236307364 shutdown_connections 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 1 -- 172.21.15.105:0/2236307364 wait complete. 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 10 monclient: build_initial_monmap 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.620+0000 7f3c87186640 10 monclient: monmap: 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 librados: starting msgr at 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 librados: starting objecter 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 Processor -- start 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- start start 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 librados: setting wanted keys 2024-09-17T16:41:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 librados: calling monclient init 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: init 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _add_conns ranks=[8,6,0,2,1,4,3,5,7] 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-e con 0x7f3c68003830 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 0x7f3c80122bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-d con 0x7f3c680066c0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68009700 0x7f3c80123100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): picked mon.noname-a con 0x7f3c68009700 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8006e740 con 0x7f3c68009700 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f3c80139c00 con 0x7f3c680066c0 2024-09-17T16:41:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f3c80124cd0 con 0x7f3c68003830 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient(hunting): authenticate will time out at 1788.894076s 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 0x7f3c80122bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59520/0 (socket says 172.21.15.105:59520) 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 -- 172.21.15.105:0/3846432720 learned_addr learned my addr 172.21.15.105:0/3846432720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): get_auth_request con 0x7f3c68003830 auth_method 0 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68009700 0x7f3c80123100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): get_auth_request con 0x7f3c680066c0 auth_method 0 2024-09-17T16:41:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): get_auth_request con 0x7f3c68009700 auth_method 0 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c85723640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient(hunting): handle_auth_done global_id 54638 payload 824 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 -- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 msgr2=0x7f3c80122bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 0x7f3c80122bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 -- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68009700 msgr2=0x7f3c80123100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68009700 0x7f3c80123100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient: _finish_hunting 0 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 monclient: found mon.noname-e 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 -- 172.21.15.105:0/3846432720 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c801249f0 con 0x7f3c68003830 2024-09-17T16:41:45.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c77fff640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 0x7f3c80122bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c84f22640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c016a70 tx=0x7f3c7c016f30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 client.?.objecter ms_handle_connect 0x7f3c68003830 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c7c00f040 con 0x7f3c68003830 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: dump: 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:45.654 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:45.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c01a280 con 0x7f3c68003830 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c7c0188c0 con 0x7f3c68003830 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c76ffd640 4 set_mon_vals no callback set 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c757fa640 10 monclient: dump: 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:45.656 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 5 monclient: authenticate success, global_id 54638 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _renew_subs 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c80124fb0 con 0x7f3c68003830 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 client.54638.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _renew_subs 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c80138dc0 con 0x7f3c68003830 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.624+0000 7f3c87186640 1 librados: init done 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c87186640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c87186640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c87186640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c80138870 con 0x7f3c68003830 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3c7c018450 con 0x7f3c68003830 2024-09-17T16:41:45.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 0x7f3c68042850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f3c7c020070 con 0x7f3c68003830 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 10 client.54638.objecter ms_dispatch 0x7f3c80000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 3 client.54638.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 3 client.54638.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 20 client.54638.objecter dump_active .. 0 homeless 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c77fff640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 0x7f3c68042850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c77fff640 10 monclient: get_auth_request con 0x7f3c68040390 auth_method 0 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c757fa640 10 client.54638.objecter ms_handle_connect 0x7f3c68040390 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.628+0000 7f3c77fff640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 0x7f3c68042850 secure :-1 s=READY pgs=3590 cs=0 l=1 rev1=1 crypto rx=0x7f3c700b0000 tx=0x7f3c700a8070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.632+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3c7c02bd50 con 0x7f3c68003830 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.632+0000 7f3c757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:45.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.632+0000 7f3c757fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:45.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.868+0000 7f3c87186640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:41:45.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.868+0000 7f3c87186640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:45.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.868+0000 7f3c87186640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:45.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.868+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3c80139f90 con 0x7f3c68003830 2024-09-17T16:41:45.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c757fa640 1 -- 172.21.15.105:0/3846432720 <== mon.8 v2:172.21.15.105:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f3c7c02d440 con 0x7f3c68003830 2024-09-17T16:41:45.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:45.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c757fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:45.881 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":84,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:41:45.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 10 librados: watch_flush enter 2024-09-17T16:41:45.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 10 librados: watch_flush exit 2024-09-17T16:41:45.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 20 client.54638.objecter shutdown clearing up homeless session... 2024-09-17T16:41:45.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 10 client.54638.objecter successfully canceled tick 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 msgr2=0x7f3c68042850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 0x7f3c68042850 secure :-1 s=READY pgs=3590 cs=0 l=1 rev1=1 crypto rx=0x7f3c700b0000 tx=0x7f3c700a8070 comp rx=0 tx=0).stop 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 10 monclient: shutdown 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 msgr2=0x7f3c80122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:45.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c016a70 tx=0x7f3c7c016f30 comp rx=0 tx=0).stop 2024-09-17T16:41:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 shutdown_connections 2024-09-17T16:41:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f3c68040390 0x7f3c68042850 unknown :-1 s=CLOSED pgs=3590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68009700 0x7f3c80123100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f3c680066c0 0x7f3c80122bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 --2- 172.21.15.105:0/3846432720 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f3c68003830 0x7f3c80122680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 >> 172.21.15.105:0/3846432720 conn(0x7f3c800a8d00 msgr2=0x7f3c800a9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 shutdown_connections 2024-09-17T16:41:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 -- 172.21.15.105:0/3846432720 wait complete. 2024-09-17T16:41:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:45.872+0000 7f3c87186640 1 librados: shutdown 2024-09-17T16:41:45.900 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:41:45.901 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:41:45.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:41:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: get_monmap_and_config 2024-09-17T16:41:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: monmap: 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:46.016712+0000 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:46.016712+0000 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:46.025 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 Processor -- start 2024-09-17T16:41:46.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 -- start start 2024-09-17T16:41:46.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: init 2024-09-17T16:41:46.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient: _add_conns ranks=[1,3,10,5,8,9,2,7,0,4,6] 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff8240a6a30 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 0x7ff82413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-g con 0x7ff8240a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-f con 0x7ff8240a7520 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8240a4080 con 0x7ff8240a6a30 2024-09-17T16:41:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ff8240a7c40 con 0x7ff8240a7520 2024-09-17T16:41:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff8240a37e0 con 0x7ff8240a4ab0 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff82a58b640 10 monclient(hunting): authenticate will time out at 1789.282459s 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 0x7ff82413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:33826/0 (socket says 172.21.15.105:33826) 2024-09-17T16:41:46.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 -- 172.21.15.105:0/4093664089 learned_addr learned my addr 172.21.15.105:0/4093664089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 10 monclient(hunting): get_auth_request con 0x7ff8240a7520 auth_method 0 2024-09-17T16:41:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): get_auth_request con 0x7ff8240a4ab0 auth_method 0 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 -- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 msgr2=0x7ff824140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 -- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 msgr2=0x7ff824140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 --2- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff828b28640 1 --2- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient(hunting): handle_auth_done global_id 54732 payload 274 2024-09-17T16:41:46.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 -- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 msgr2=0x7ff824140f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:41:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 --2- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 -- 172.21.15.105:0/4093664089 >> v1:172.21.15.160:6789/0 conn(0x7ff8240a6a30 legacy=0x7ff8240a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 monclient: found mon.noname-g 2024-09-17T16:41:46.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:41:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.012+0000 7ff8237fe640 1 -- 172.21.15.105:0/4093664089 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8240a3ac0 con 0x7ff8240a4ab0 2024-09-17T16:41:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.016+0000 7ff8237fe640 1 --2- 172.21.15.105:0/4093664089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 0x7ff82413ea30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097c0 tx=0x7ff80c01c320 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6bab4056416e518b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 1 -- 172.21.15.105:0/4093664089 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff80c0b2070 con 0x7ff8240a4ab0 2024-09-17T16:41:46.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: dump: 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.055 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.058 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 1 -- 172.21.15.105:0/4093664089 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80c0ab7f0 con 0x7ff8240a4ab0 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 1 -- 172.21.15.105:0/4093664089 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff80c0abe50 con 0x7ff8240a4ab0 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 5 monclient: authenticate success, global_id 54732 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 4 set_mon_vals no callback set 2024-09-17T16:41:46.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff822ffd640 10 monclient: dump: 2024-09-17T16:41:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.060 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.061 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 10 monclient: shutdown 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 -- 172.21.15.105:0/4093664089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 msgr2=0x7ff82413ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 --2- 172.21.15.105:0/4093664089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 0x7ff82413ea30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097c0 tx=0x7ff80c01c320 comp rx=0 tx=0).stop 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 -- 172.21.15.105:0/4093664089 shutdown_connections 2024-09-17T16:41:46.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 --2- 172.21.15.105:0/4093664089 >> v2:172.21.15.160:3300/0 conn(0x7ff8240a7520 0x7ff824140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 --2- 172.21.15.105:0/4093664089 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8240a4ab0 0x7ff82413ea30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 -- 172.21.15.105:0/4093664089 >> 172.21.15.105:0/4093664089 conn(0x7ff8240a8d00 msgr2=0x7ff8240a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 -- 172.21.15.105:0/4093664089 shutdown_connections 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.020+0000 7ff82a58b640 1 -- 172.21.15.105:0/4093664089 wait complete. 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: monmap: 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:46.063 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 librados: starting msgr at 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 librados: starting objecter 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 Processor -- start 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- start start 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 librados: setting wanted keys 2024-09-17T16:41:46.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 librados: calling monclient init 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: init 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _add_conns ranks=[1,6,7,5,2,4,8,0,3] 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8240a4ab0 0x7ff8241d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-f con 0x7ff8240a4ab0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 0x7ff8241d1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-d con 0x7ff8240a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff8240a7520 0x7ff8241265d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): picked mon.noname-i con 0x7ff8240a7520 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff824143e60 con 0x7ff8240a4ab0 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff824143ce0 con 0x7ff8240a6a30 2024-09-17T16:41:46.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff824143fe0 con 0x7ff8240a7520 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): authenticate will time out at 1789.293944s 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 0x7ff8241d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 0x7ff8241d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:60400/0 (socket says 172.21.15.105:60400) 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 -- 172.21.15.105:0/985406117 learned_addr learned my addr 172.21.15.105:0/985406117 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): get_auth_request con 0x7ff8240a6a30 auth_method 0 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8240a4ab0 0x7ff8241d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): get_auth_request con 0x7ff8240a4ab0 auth_method 0 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff828b28640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff8240a7520 0x7ff8241265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient(hunting): handle_auth_done global_id 24801 payload 824 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff8240a7520 msgr2=0x7ff8241265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff8240a7520 0x7ff8241265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8240a4ab0 msgr2=0x7ff8241d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8240a4ab0 0x7ff8241d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 monclient: found mon.noname-d 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824129f30 con 0x7ff8240a6a30 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff828b28640 10 monclient: get_auth_request con 0x7ff8240a7520 auth_method 0 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8240a4ab0 0x7ff8241d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff828b28640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff8240a7520 0x7ff8241265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8237fe640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 0x7ff8241d1730 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097c0 tx=0x7ff80c0190c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 client.?.objecter ms_handle_connect 0x7ff8240a6a30 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff80c0b2040 con 0x7ff8240a6a30 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: dump: 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.068 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.069 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80c0bb070 con 0x7ff8240a6a30 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff80c0044b0 con 0x7ff8240a6a30 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff8227fc640 4 set_mon_vals no callback set 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 10 monclient: dump: 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.070 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 5 monclient: authenticate success, global_id 24801 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _renew_subs 2024-09-17T16:41:46.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82412a210 con 0x7ff8240a6a30 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 client.24801.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _renew_subs 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff824140c90 con 0x7ff8240a6a30 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 librados: init done 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 0x7ff824126ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff824128c30 con 0x7ff824126b10 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 0x7ff824126ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient: get_auth_request con 0x7ff824126b10 auth_method 0 2024-09-17T16:41:46.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff823fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.024+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff80c004040 con 0x7ff8240a6a30 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 10 monclient(hunting): handle_auth_done global_id 36153 payload 824 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff80403e550 0x7ff804040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8240a6e10 con 0x7ff824126b10 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7ff80c0e9dd0 con 0x7ff8240a6a30 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 client.24801.objecter ms_dispatch 0x7ff824000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 3 client.24801.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 3 client.24801.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:46.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 20 client.24801.objecter dump_active .. 0 homeless 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 0x7ff824126ef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eaa0 tx=0x7ff81c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 client.24801.objecter ms_handle_connect 0x7ff824126b10 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 client.24801.objecter resend_mon_ops 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff828b28640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff80403e550 0x7ff804040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff81c0050d0 con 0x7ff824126b10 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 monclient: dump: 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.074 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff828b28640 10 monclient: get_auth_request con 0x7ff80403e550 auth_method 0 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff81c018070 con 0x7ff824126b10 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 msgr2=0x7ff824126ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 0x7ff824126ef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eaa0 tx=0x7ff81c00ef60 comp rx=0 tx=0).stop 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 msgr2=0x7ff824126ef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824126b10 0x7ff824126ef0 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eaa0 tx=0x7ff81c00ef60 comp rx=0 tx=0).stop 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff820ff9640 10 client.24801.objecter ms_handle_connect 0x7ff80403e550 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.028+0000 7ff828b28640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff80403e550 0x7ff804040a10 secure :-1 s=READY pgs=3591 cs=0 l=1 rev1=1 crypto rx=0x7ff824127cb0 tx=0x7ff814008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff82a58b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff82a58b640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 0x7ff824122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff82a58b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff824051e80 con 0x7ff824127430 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 0x7ff824122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient: get_auth_request con 0x7ff824127430 auth_method 0 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 10 monclient(hunting): handle_auth_done global_id 36162 payload 824 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 1 -- 172.21.15.105:0/985406117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff824128770 con 0x7ff824127430 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff823fff640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 0x7ff824122680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009010 tx=0x7ff81c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 10 client.24801.objecter ms_handle_connect 0x7ff824127430 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 10 client.24801.objecter resend_mon_ops 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff81c006e60 con 0x7ff824127430 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.032+0000 7ff820ff9640 10 monclient: dump: 2024-09-17T16:41:46.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.078 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7ff81c0050d0 con 0x7ff824127430 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 msgr2=0x7ff824122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 0x7ff824122680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009010 tx=0x7ff81c001260 comp rx=0 tx=0).stop 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 msgr2=0x7ff824122680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff820ff9640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff824127430 0x7ff824122680 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009010 tx=0x7ff81c001260 comp rx=0 tx=0).stop 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:46.079 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 696835, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.081 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.082 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:46.083 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.084 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.085 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.086 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.088 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.089 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:46.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:46.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.096 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.098 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.100 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:46.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.102 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.102 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:46.102 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:46.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 10 librados: watch_flush enter 2024-09-17T16:41:46.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 10 librados: watch_flush exit 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 20 client.24801.objecter shutdown clearing up homeless session... 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 10 client.24801.objecter successfully canceled tick 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff80403e550 msgr2=0x7ff804040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff80403e550 0x7ff804040a10 secure :-1 s=READY pgs=3591 cs=0 l=1 rev1=1 crypto rx=0x7ff824127cb0 tx=0x7ff814008040 comp rx=0 tx=0).stop 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 10 monclient: shutdown 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 msgr2=0x7ff8241d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 --2- 172.21.15.105:0/985406117 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8240a6a30 0x7ff8241d1730 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097c0 tx=0x7ff80c0190c0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff828b28640 1 -- 172.21.15.105:0/985406117 reap_dead start 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff828b28640 1 -- 172.21.15.105:0/985406117 reap_dead start 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 shutdown_connections 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 >> 172.21.15.105:0/985406117 conn(0x7ff8240a8d00 msgr2=0x7ff8240aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 shutdown_connections 2024-09-17T16:41:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 -- 172.21.15.105:0/985406117 wait complete. 2024-09-17T16:41:46.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.036+0000 7ff82a58b640 1 librados: shutdown 2024-09-17T16:41:46.105 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:46.105 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:41:46.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:41:46.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: get_monmap_and_config 2024-09-17T16:41:46.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: monmap: 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:46.227932+0000 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:46.227932+0000 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.232 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:46.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 Processor -- start 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 -- start start 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: init 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient: _add_conns ranks=[8,4,10,5,6,3,0,9,1,2,7] 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-e con 0x7efe480a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-c con 0x7efe480a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 0x7efe48139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-f con 0x7efe481396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7efe480a3fc0 con 0x7efe481396a0 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7efe480a63c0 con 0x7efe480a4910 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7efe480a7740 con 0x7efe480a68e0 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4f879640 10 monclient(hunting): authenticate will time out at 1789.493680s 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:60594/0 (socket says 172.21.15.105:60594) 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59532/0 (socket says 172.21.15.105:59532) 2024-09-17T16:41:46.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 -- 172.21.15.105:0/2006161683 learned_addr learned my addr 172.21.15.105:0/2006161683 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): get_auth_request con 0x7efe480a4910 auth_method 0 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): get_auth_request con 0x7efe480a68e0 auth_method 0 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 1 --2- 172.21.15.105:0/2006161683 >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 0x7efe48139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 10 monclient(hunting): get_auth_request con 0x7efe481396a0 auth_method 0 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient(hunting): handle_auth_done global_id 74416 payload 274 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 -- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 msgr2=0x7efe480a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 -- 172.21.15.105:0/2006161683 >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 msgr2=0x7efe48139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 --2- 172.21.15.105:0/2006161683 >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 0x7efe48139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 monclient: found mon.noname-c 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:46.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4ce14640 1 -- 172.21.15.105:0/2006161683 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4806ebc0 con 0x7efe480a4910 2024-09-17T16:41:46.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4d615640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:46.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.224+0000 7efe4de16640 1 --2- 172.21.15.105:0/2006161683 >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 0x7efe48139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe4401c3f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8598e4135ae9fb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 1 -- 172.21.15.105:0/2006161683 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440b0070 con 0x7efe480a4910 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: dump: 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.246 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 5 monclient: authenticate success, global_id 74416 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:46.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 1 -- 172.21.15.105:0/2006161683 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440aa810 con 0x7efe480a4910 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 4 set_mon_vals no callback set 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 1 -- 172.21.15.105:0/2006161683 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440a9300 con 0x7efe480a4910 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe3ffff640 10 monclient: dump: 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.248 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: shutdown 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 msgr2=0x7efe480a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe4401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- 172.21.15.105:0/2006161683 shutdown_connections 2024-09-17T16:41:46.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- 172.21.15.105:0/2006161683 >> v2:172.21.15.160:3300/0 conn(0x7efe481396a0 0x7efe48139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efe480a4910 0x7efe480a7200 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- 172.21.15.105:0/2006161683 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efe480a68e0 0x7efe480a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- 172.21.15.105:0/2006161683 >> 172.21.15.105:0/2006161683 conn(0x7efe48068ba0 msgr2=0x7efe48068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- 172.21.15.105:0/2006161683 shutdown_connections 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- 172.21.15.105:0/2006161683 wait complete. 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: monmap: 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:46.250 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 librados: starting msgr at 2024-09-17T16:41:46.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 librados: starting objecter 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 Processor -- start 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- start start 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 librados: setting wanted keys 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 librados: calling monclient init 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: init 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient: _add_conns ranks=[0,2,7,1,3,5,4,6,8] 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480a4910 0x7efe48138070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-a con 0x7efe480a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe480a68e0 0x7efe481385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-b con 0x7efe480a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efe481396a0 0x7efe4813d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): picked mon.noname-i con 0x7efe481396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:46.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efe4806e630 con 0x7efe480a4910 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efe480a7af0 con 0x7efe480a68e0 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7efe48143320 con 0x7efe481396a0 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4f879640 10 monclient(hunting): authenticate will time out at 1789.499720s 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480a4910 0x7efe48138070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe480a68e0 0x7efe481385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480a4910 0x7efe48138070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33218/0 (socket says 172.21.15.105:33218) 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 1 -- 172.21.15.105:0/1422814027 learned_addr learned my addr 172.21.15.105:0/1422814027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 10 monclient(hunting): get_auth_request con 0x7efe480a4910 auth_method 0 2024-09-17T16:41:46.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4d615640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): get_auth_request con 0x7efe480a68e0 auth_method 0 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.228+0000 7efe4ce14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4de16640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efe481396a0 0x7efe4813d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 10 monclient(hunting): handle_auth_done global_id 84107 payload 824 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efe481396a0 msgr2=0x7efe4813d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7efe481396a0 0x7efe4813d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480a4910 msgr2=0x7efe48138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe480a4910 0x7efe48138070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 monclient: found mon.noname-b 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48123b70 con 0x7efe480a68e0 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4ce14640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe480a68e0 0x7efe481385b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe480a4380 tx=0x7efe44019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.?.objecter ms_handle_connect 0x7efe480a68e0 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440b0040 con 0x7efe480a68e0 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: dump: 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.256 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440044b0 con 0x7efe480a68e0 2024-09-17T16:41:46.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 5 monclient: authenticate success, global_id 84107 2024-09-17T16:41:46.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440aacd0 con 0x7efe480a68e0 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3effd640 4 set_mon_vals no callback set 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: dump: 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.259 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 monclient: _renew_subs 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48139250 con 0x7efe480a68e0 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 client.84107.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:46.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 monclient: _renew_subs 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4813fc40 con 0x7efe480a68e0 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 1 librados: init done 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 0x7efe4813dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efe4813f1a0 con 0x7efe4813dc10 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 0x7efe4813dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient: get_auth_request con 0x7efe4813dc10 auth_method 0 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efe44004040 con 0x7efe480a68e0 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 10 monclient(hunting): handle_auth_done global_id 84116 payload 824 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe480aa050 con 0x7efe4813dc10 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efe2c03e550 0x7efe2c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7efe440e9550 con 0x7efe480a68e0 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.84107.objecter ms_dispatch 0x7efe48000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:46.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 3 client.84107.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 3 client.84107.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4d615640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 0x7efe4813dff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe38011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 20 client.84107.objecter dump_active .. 0 homeless 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.84107.objecter ms_handle_connect 0x7efe4813dc10 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.84107.objecter resend_mon_ops 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe38014440 con 0x7efe4813dc10 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: dump: 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.263 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.264 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.265 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.265 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.265 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.266 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.266 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4de16640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efe2c03e550 0x7efe2c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4de16640 10 monclient: get_auth_request con 0x7efe2c03e550 auth_method 0 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efe3800fb60 con 0x7efe4813dc10 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 msgr2=0x7efe4813dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 0x7efe4813dff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe38011e90 comp rx=0 tx=0).stop 2024-09-17T16:41:46.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 msgr2=0x7efe4813dff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813dc10 0x7efe4813dff0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe38011e90 comp rx=0 tx=0).stop 2024-09-17T16:41:46.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe4de16640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efe2c03e550 0x7efe2c040a10 secure :-1 s=READY pgs=3592 cs=0 l=1 rev1=1 crypto rx=0x7efe40009e50 tx=0x7efe4000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.232+0000 7efe3dffb640 10 client.84107.objecter ms_handle_connect 0x7efe2c03e550 2024-09-17T16:41:46.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:46.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 0x7efe480a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efe48051e80 con 0x7efe4813e530 2024-09-17T16:41:46.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 0x7efe480a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient: get_auth_request con 0x7efe4813e530 auth_method 0 2024-09-17T16:41:46.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 10 monclient(hunting): handle_auth_done global_id 84125 payload 824 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 1 -- 172.21.15.105:0/1422814027 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efe48077c10 con 0x7efe4813e530 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 client.84107.objecter ms_handle_connect 0x7efe4813e530 2024-09-17T16:41:46.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 client.84107.objecter resend_mon_ops 2024-09-17T16:41:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4d615640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 0x7efe480a0a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe380046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe380148b0 con 0x7efe4813e530 2024-09-17T16:41:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:46.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 monclient: dump: 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.277 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.277 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.277 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.277 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.277 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.278 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.278 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.278 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.278 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.278 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7efe3801d400 con 0x7efe4813e530 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 msgr2=0x7efe480a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 0x7efe480a0a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe380046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 msgr2=0x7efe480a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe3dffb640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe4813e530 0x7efe480a0a20 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe380119d0 tx=0x7efe380046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.280 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:46.280 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.281 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.281 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:46.281 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:46.282 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 13475, 2024-09-17T16:41:46.282 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:46.282 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:46.282 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:46.282 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:46.283 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.284 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.285 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:46.287 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:46.288 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.289 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.290 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:46.292 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.293 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.294 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.297 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:46.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.299 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.302 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.305 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:46.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.309 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:46.310 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 librados: watch_flush enter 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 librados: watch_flush exit 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 20 client.84107.objecter shutdown clearing up homeless session... 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 client.84107.objecter successfully canceled tick 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efe2c03e550 msgr2=0x7efe2c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efe2c03e550 0x7efe2c040a10 secure :-1 s=READY pgs=3592 cs=0 l=1 rev1=1 crypto rx=0x7efe40009e50 tx=0x7efe4000b040 comp rx=0 tx=0).stop 2024-09-17T16:41:46.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 10 monclient: shutdown 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe480a68e0 msgr2=0x7efe481385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 --2- 172.21.15.105:0/1422814027 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe480a68e0 0x7efe481385b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe480a4380 tx=0x7efe44019190 comp rx=0 tx=0).stop 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4de16640 1 -- 172.21.15.105:0/1422814027 reap_dead start 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4de16640 1 -- 172.21.15.105:0/1422814027 reap_dead start 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 shutdown_connections 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 >> 172.21.15.105:0/1422814027 conn(0x7efe48068ba0 msgr2=0x7efe480a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 shutdown_connections 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 -- 172.21.15.105:0/1422814027 wait complete. 2024-09-17T16:41:46.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.240+0000 7efe4f879640 1 librados: shutdown 2024-09-17T16:41:46.314 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:46.314 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:41:46.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: get_monmap_and_config 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: monmap: 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:46.432546+0000 2024-09-17T16:41:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:46.432546+0000 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:46.437 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 Processor -- start 2024-09-17T16:41:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 -- start start 2024-09-17T16:41:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: init 2024-09-17T16:41:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient: _add_conns ranks=[6,2,3,0,7,9,8,5,10,1,4] 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-d con 0x7ff28c0a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-b con 0x7ff28c0a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff28c1397f0 0x7ff28c139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-g con 0x7ff28c1397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0a7c40 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c0a7890 con 0x7ff28c1397f0 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff28c0a4080 con 0x7ff28c0a6a30 2024-09-17T16:41:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff290f3a640 10 monclient(hunting): authenticate will time out at 1789.698299s 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:34830/0 (socket says 172.21.15.105:34830) 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 1 -- 172.21.15.105:0/2617683482 learned_addr learned my addr 172.21.15.105:0/2617683482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): get_auth_request con 0x7ff28c0a4ab0 auth_method 0 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:60408/0 (socket says 172.21.15.105:60408) 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): get_auth_request con 0x7ff28c0a6a30 auth_method 0 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28ad9d640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff28c1397f0 0x7ff28c139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28ad9d640 10 monclient(hunting): get_auth_request con 0x7ff28c1397f0 auth_method 0 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient(hunting): handle_auth_done global_id 24810 payload 274 2024-09-17T16:41:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 -- 172.21.15.105:0/2617683482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff28c1397f0 msgr2=0x7ff28c139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff28c1397f0 0x7ff28c139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 -- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 msgr2=0x7ff28c0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 monclient: found mon.noname-d 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff28a59c640 1 -- 172.21.15.105:0/2617683482 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c0a37e0 con 0x7ff28c0a6a30 2024-09-17T16:41:46.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.428+0000 7ff289d9b640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28a59c640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0097f0 tx=0x7ff27c01c370 comp rx=0 tx=0).ready entity=mon.6 client_cookie=5a9f3a442cd948db server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 1 -- 172.21.15.105:0/2617683482 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff27c0b0070 con 0x7ff28c0a6a30 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: dump: 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.451 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 1 -- 172.21.15.105:0/2617683482 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c0aa810 con 0x7ff28c0a6a30 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 5 monclient: authenticate success, global_id 24810 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 1 -- 172.21.15.105:0/2617683482 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff27c0a9300 con 0x7ff28c0a6a30 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 4 set_mon_vals no callback set 2024-09-17T16:41:46.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff28959a640 10 monclient: dump: 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.454 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 10 monclient: shutdown 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 -- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 msgr2=0x7ff28c0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0097f0 tx=0x7ff27c01c370 comp rx=0 tx=0).stop 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 -- 172.21.15.105:0/2617683482 shutdown_connections 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff28c1397f0 0x7ff28c139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c0a4ab0 0x7ff28c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 --2- 172.21.15.105:0/2617683482 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff28c0a6a30 0x7ff28c0a6e10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 -- 172.21.15.105:0/2617683482 >> 172.21.15.105:0/2617683482 conn(0x7ff28c0a8d00 msgr2=0x7ff28c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 -- 172.21.15.105:0/2617683482 shutdown_connections 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.432+0000 7ff290f3a640 1 -- 172.21.15.105:0/2617683482 wait complete. 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: monmap: 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.456 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 librados: starting msgr at 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 librados: starting objecter 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 Processor -- start 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- start start 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 librados: setting wanted keys 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 librados: calling monclient init 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: init 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _add_conns ranks=[0,7,2,1,5,6,4,8,3] 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 0x7ff28c138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-a con 0x7ff28c0a4ab0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff28c0a6a30 0x7ff28c1387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-i con 0x7ff28c0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c1397f0 0x7ff28c13d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): picked mon.noname-b con 0x7ff28c1397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c06e770 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c13a5a0 con 0x7ff28c1397f0 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff28c143260 con 0x7ff28c0a6a30 2024-09-17T16:41:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): authenticate will time out at 1789.705313s 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 0x7ff28c138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c1397f0 0x7ff28c13d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 0x7ff28c138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33228/0 (socket says 172.21.15.105:33228) 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 -- 172.21.15.105:0/3289921777 learned_addr learned my addr 172.21.15.105:0/3289921777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): get_auth_request con 0x7ff28c0a4ab0 auth_method 0 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 10 monclient(hunting): get_auth_request con 0x7ff28c1397f0 auth_method 0 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff28c0a6a30 0x7ff28c1387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): get_auth_request con 0x7ff28c0a6a30 auth_method 0 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient(hunting): handle_auth_done global_id 36180 payload 824 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff28c0a6a30 msgr2=0x7ff28c1387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff28c0a6a30 0x7ff28c1387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c1397f0 msgr2=0x7ff28c13d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c1397f0 0x7ff28c13d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 monclient: found mon.noname-a 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c123fd0 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28ad9d640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff28c1397f0 0x7ff28c13d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff28a59c640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 0x7ff28c138260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009710 tx=0x7ff27c019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 client.?.objecter ms_handle_connect 0x7ff28c0a4ab0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff27c0b0040 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: dump: 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 5 monclient: authenticate success, global_id 36180 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27c0b9070 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _renew_subs 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c0aa100 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff27c0044b0 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 client.36180.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 monclient: dump: 2024-09-17T16:41:46.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27bfff640 4 set_mon_vals no callback set 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _renew_subs 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1393d0 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 librados: init done 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 0x7ff28c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff28c051e80 con 0x7ff28c13e360 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 0x7ff28c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient: get_auth_request con 0x7ff28c13e360 auth_method 0 2024-09-17T16:41:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff27c0aac90 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 10 monclient(hunting): handle_auth_done global_id 74425 payload 824 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff28c13f1b0 con 0x7ff28c13e360 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff26c03e580 0x7ff26c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff289d9b640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 0x7ff28c02d450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff28c13f7c0 tx=0x7ff280010590 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 client.36180.objecter ms_handle_connect 0x7ff28c13e360 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 client.36180.objecter resend_mon_ops 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7ff27c0e9ef0 con 0x7ff28c0a4ab0 2024-09-17T16:41:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 10 client.36180.objecter ms_dispatch 0x7ff28c000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 3 client.36180.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.436+0000 7ff27affd640 3 client.36180.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 20 client.36180.objecter dump_active .. 0 homeless 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff280003e10 con 0x7ff28c13e360 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 monclient: dump: 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff28ad9d640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff26c03e580 0x7ff26c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff280010c60 con 0x7ff28c13e360 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 msgr2=0x7ff28c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 0x7ff28c02d450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff28c13f7c0 tx=0x7ff280010590 comp rx=0 tx=0).stop 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 msgr2=0x7ff28c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e360 0x7ff28c02d450 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff28c13f7c0 tx=0x7ff280010590 comp rx=0 tx=0).stop 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff28ad9d640 10 monclient: get_auth_request con 0x7ff26c03e580 auth_method 0 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff27affd640 10 client.36180.objecter ms_handle_connect 0x7ff26c03e580 2024-09-17T16:41:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.440+0000 7ff28ad9d640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff26c03e580 0x7ff26c040a40 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7ff28c06eab0 tx=0x7ff284008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 0x7ff28c02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff28c06a200 con 0x7ff28c13e740 2024-09-17T16:41:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 0x7ff28c02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient: get_auth_request con 0x7ff28c13e740 auth_method 0 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 10 monclient(hunting): handle_auth_done global_id 74434 payload 824 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 1 -- 172.21.15.105:0/3289921777 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff28c123900 con 0x7ff28c13e740 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 client.36180.objecter ms_handle_connect 0x7ff28c13e740 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 client.36180.objecter resend_mon_ops 2024-09-17T16:41:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff289d9b640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 0x7ff28c02d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff28001a000 tx=0x7ff280004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2800046b0 con 0x7ff28c13e740 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 monclient: dump: 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.474 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7ff280004240 con 0x7ff28c13e740 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:46.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 msgr2=0x7ff28c02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 0x7ff28c02d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff28001a000 tx=0x7ff280004a50 comp rx=0 tx=0).stop 2024-09-17T16:41:46.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 msgr2=0x7ff28c02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff27affd640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff28c13e740 0x7ff28c02d990 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff28001a000 tx=0x7ff280004a50 comp rx=0 tx=0).stop 2024-09-17T16:41:46.478 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:46.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.478 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.478 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:46.478 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 145906, 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:46.479 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.480 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.481 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:46.483 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.484 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.487 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.489 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:46.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.494 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.498 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.499 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:46.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.503 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.507 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.508 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.509 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.510 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:46.512 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 librados: watch_flush enter 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 librados: watch_flush exit 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 20 client.36180.objecter shutdown clearing up homeless session... 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 client.36180.objecter successfully canceled tick 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff26c03e580 msgr2=0x7ff26c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff26c03e580 0x7ff26c040a40 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7ff28c06eab0 tx=0x7ff284008040 comp rx=0 tx=0).stop 2024-09-17T16:41:46.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 10 monclient: shutdown 2024-09-17T16:41:46.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 msgr2=0x7ff28c138260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 --2- 172.21.15.105:0/3289921777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff28c0a4ab0 0x7ff28c138260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009710 tx=0x7ff27c019110 comp rx=0 tx=0).stop 2024-09-17T16:41:46.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff28ad9d640 1 -- 172.21.15.105:0/3289921777 reap_dead start 2024-09-17T16:41:46.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff28ad9d640 1 -- 172.21.15.105:0/3289921777 reap_dead start 2024-09-17T16:41:46.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 shutdown_connections 2024-09-17T16:41:46.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 >> 172.21.15.105:0/3289921777 conn(0x7ff28c0a8d00 msgr2=0x7ff28c13fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 shutdown_connections 2024-09-17T16:41:46.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.444+0000 7ff290f3a640 1 -- 172.21.15.105:0/3289921777 wait complete. 2024-09-17T16:41:46.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.448+0000 7ff290f3a640 1 librados: shutdown 2024-09-17T16:41:46.518 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:46.519 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:41:46.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: get_monmap_and_config 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: monmap: 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.644 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:46.640422+0000 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:46.640422+0000 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:46.645 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:46.646 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 Processor -- start 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 -- start start 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: init 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient: _add_conns ranks=[4,7,10,2,1,9,8,6,3,0,5] 2024-09-17T16:41:46.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-c con 0x7f00080a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00080a6cc0 0x7f00080a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-i con 0x7f00080a6cc0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 0x7f0008139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-f con 0x7f00081396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f00080a2ea0 con 0x7f00081396a0 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0008069ab0 con 0x7f00080a68e0 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f00080a87d0 con 0x7f00080a6cc0 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000ec43640 10 monclient(hunting): authenticate will time out at 1789.906169s 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:60638/0 (socket says 172.21.15.105:60638) 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 -- 172.21.15.105:0/1267443486 learned_addr learned my addr 172.21.15.105:0/1267443486 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): get_auth_request con 0x7f00080a68e0 auth_method 0 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f0007fff640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00080a6cc0 0x7f00080a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000d1e0640 1 --2- 172.21.15.105:0/1267443486 >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 0x7f0008139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f0007fff640 10 monclient(hunting): get_auth_request con 0x7f00080a6cc0 auth_method 0 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f0007fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f0007fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f0007fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient(hunting): handle_auth_done global_id 74443 payload 274 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 -- 172.21.15.105:0/1267443486 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00080a6cc0 msgr2=0x7f00080a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00080a6cc0 0x7f00080a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 -- 172.21.15.105:0/1267443486 >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 msgr2=0x7f0008139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 --2- 172.21.15.105:0/1267443486 >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 0x7f0008139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 monclient: found mon.noname-c 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:46.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000c9df640 1 -- 172.21.15.105:0/1267443486 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00080a7b80 con 0x7f00080a68e0 2024-09-17T16:41:46.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000d1e0640 10 monclient: get_auth_request con 0x7f00081396a0 auth_method 0 2024-09-17T16:41:46.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.636+0000 7f000d1e0640 1 --2- 172.21.15.105:0/1267443486 >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 0x7f0008139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000c9df640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0000009870 tx=0x7f000001c3f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2e99bc2523b804cb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 1 -- 172.21.15.105:0/1267443486 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00000b0070 con 0x7f00080a68e0 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: dump: 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 1 -- 172.21.15.105:0/1267443486 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00000aa810 con 0x7f00080a68e0 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 5 monclient: authenticate success, global_id 74443 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 1 -- 172.21.15.105:0/1267443486 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00000a9300 con 0x7f00080a68e0 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f00077fe640 10 monclient: dump: 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.666 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 4 set_mon_vals no callback set 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 10 monclient: shutdown 2024-09-17T16:41:46.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 -- 172.21.15.105:0/1267443486 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 msgr2=0x7f00080a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0000009870 tx=0x7f000001c3f0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 -- 172.21.15.105:0/1267443486 shutdown_connections 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 --2- 172.21.15.105:0/1267443486 >> v2:172.21.15.160:3300/0 conn(0x7f00081396a0 0x7f0008139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f00080a6cc0 0x7f00080a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 --2- 172.21.15.105:0/1267443486 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00080a68e0 0x7f00080a48a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 -- 172.21.15.105:0/1267443486 >> 172.21.15.105:0/1267443486 conn(0x7f00080a96e0 msgr2=0x7f0008068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 -- 172.21.15.105:0/1267443486 shutdown_connections 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.640+0000 7f000ec43640 1 -- 172.21.15.105:0/1267443486 wait complete. 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: monmap: 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:46.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 librados: starting msgr at 2024-09-17T16:41:46.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 librados: starting objecter 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 Processor -- start 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- start start 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 librados: setting wanted keys 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 librados: calling monclient init 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: init 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _add_conns ranks=[2,4,8,5,1,0,3,7,6] 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 0x7f0008138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-b con 0x7f00080a6cc0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 0x7f0008138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-c con 0x7f00081396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 0x7f000813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): picked mon.noname-e con 0x7f000813d890 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:46.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f00080a75e0 con 0x7f00080a6cc0 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0008069500 con 0x7f00081396a0 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f00081239f0 con 0x7f000813d890 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): authenticate will time out at 1789.913378s 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 0x7f0008138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 0x7f0008138a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:60646/0 (socket says 172.21.15.105:60646) 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 0x7f000813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 0x7f0008138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 0x7f0008138510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:34862/0 (socket says 172.21.15.105:34862) 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 -- 172.21.15.105:0/611167417 learned_addr learned my addr 172.21.15.105:0/611167417 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 0x7f000813dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59534/0 (socket says 172.21.15.105:59534) 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): get_auth_request con 0x7f00080a6cc0 auth_method 0 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): get_auth_request con 0x7f00081396a0 auth_method 0 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): get_auth_request con 0x7f000813d890 auth_method 0 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient(hunting): handle_auth_done global_id 74452 payload 824 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 msgr2=0x7f000813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 0x7f000813dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 msgr2=0x7f0008138510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 0x7f0008138510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 monclient: found mon.noname-c 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00080abd10 con 0x7f00081396a0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00080a6cc0 0x7f0008138510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000d1e0640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f000813d890 0x7f000813dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0007fff640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 0x7f0008138a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efff0007e60 tx=0x7efff000e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 client.?.objecter ms_handle_connect 0x7f00081396a0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff0014890 con 0x7f00081396a0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: dump: 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.674 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.675 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff0019070 con 0x7f00081396a0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff0013380 con 0x7f00081396a0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 5 monclient: authenticate success, global_id 74452 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _renew_subs 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0008123710 con 0x7f00081396a0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f0006ffd640 4 set_mon_vals no callback set 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:46.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: dump: 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.677 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 client.74452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _renew_subs 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f00080a4ad0 con 0x7f00081396a0 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 librados: init done 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 0x7f000813e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0008123f10 con 0x7f00080a96e0 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 0x7f000813e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient: get_auth_request con 0x7f00080a96e0 auth_method 0 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efff0014420 con 0x7f00081396a0 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 10 monclient(hunting): handle_auth_done global_id 24819 payload 824 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0008122ac0 con 0x7f00080a96e0 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f000c9df640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 0x7f000813e230 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f000001c220 tx=0x7f0000019190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7effe803e580 0x7effe8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 client.74452.objecter ms_handle_connect 0x7f00080a96e0 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 client.74452.objecter resend_mon_ops 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7efff0050600 con 0x7f00081396a0 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 client.74452.objecter ms_dispatch 0x7f0008000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:46.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 3 client.74452.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 3 client.74452.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 20 client.74452.objecter dump_active .. 0 homeless 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00000b0040 con 0x7f00080a96e0 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.644+0000 7f00057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 10 monclient: dump: 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.680 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f000d1e0640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7effe803e580 0x7effe8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f00000aa890 con 0x7f00080a96e0 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:46.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 msgr2=0x7f000813e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 0x7f000813e230 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f000001c220 tx=0x7f0000019190 comp rx=0 tx=0).stop 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 msgr2=0x7f000813e230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f00080a96e0 0x7f000813e230 secure :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f000001c220 tx=0x7f0000019190 comp rx=0 tx=0).stop 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f000d1e0640 10 monclient: get_auth_request con 0x7effe803e580 auth_method 0 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f000d1e0640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7effe803e580 0x7effe8040a40 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f000813f760 tx=0x7efff8008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.648+0000 7f00057fa640 10 client.74452.objecter ms_handle_connect 0x7effe803e580 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000ec43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000ec43640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 0x7f00080a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000ec43640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f00080a7760 con 0x7f000813e770 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 0x7f00080a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient: get_auth_request con 0x7f000813e770 auth_method 0 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 10 monclient(hunting): handle_auth_done global_id 24828 payload 824 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 1 -- 172.21.15.105:0/611167417 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f00080a70a0 con 0x7f000813e770 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 client.74452.objecter ms_handle_connect 0x7f000813e770 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 client.74452.objecter resend_mon_ops 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000c9df640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 0x7f00080a0a20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c10 tx=0x7f00000a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00000b0030 con 0x7f000813e770 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 monclient: dump: 2024-09-17T16:41:46.683 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.684 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f0000003bd0 con 0x7f000813e770 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 msgr2=0x7f00080a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 0x7f00080a0a20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c10 tx=0x7f00000a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 msgr2=0x7f00080a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f00057fa640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f000813e770 0x7f00080a0a20 secure :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c10 tx=0x7f00000a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:46.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 854291, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:46.686 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.687 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.688 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:46.689 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.690 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.692 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.697 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:46.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.705 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.707 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:46.708 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.652+0000 7f000ec43640 10 librados: watch_flush enter 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 10 librados: watch_flush exit 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 20 client.74452.objecter shutdown clearing up homeless session... 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 10 client.74452.objecter successfully canceled tick 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7effe803e580 msgr2=0x7effe8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7effe803e580 0x7effe8040a40 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f000813f760 tx=0x7efff8008040 comp rx=0 tx=0).stop 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 10 monclient: shutdown 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 msgr2=0x7f0008138a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 --2- 172.21.15.105:0/611167417 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f00081396a0 0x7f0008138a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efff0007e60 tx=0x7efff000e4b0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000d1e0640 1 -- 172.21.15.105:0/611167417 reap_dead start 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000d1e0640 1 -- 172.21.15.105:0/611167417 reap_dead start 2024-09-17T16:41:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 shutdown_connections 2024-09-17T16:41:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 >> 172.21.15.105:0/611167417 conn(0x7f00080a68e0 msgr2=0x7f00080a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 shutdown_connections 2024-09-17T16:41:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 -- 172.21.15.105:0/611167417 wait complete. 2024-09-17T16:41:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.656+0000 7f000ec43640 1 librados: shutdown 2024-09-17T16:41:46.711 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:46.711 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:41:46.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:41:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: get_monmap_and_config 2024-09-17T16:41:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: monmap: 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:46.827897+0000 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:46.827897+0000 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:46.833 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 Processor -- start 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 -- start start 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: init 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient: _add_conns ranks=[3,0,6,4,10,5,1,2,7,8,9] 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedb40a68e0 0x7fedb40a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-g con 0x7fedb40a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fedb40a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:46.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-d con 0x7fedb40a7200 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb406ebc0 con 0x7fedb40a4910 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40a3fc0 con 0x7fedb40a68e0 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fedb40a63c0 con 0x7fedb40a7200 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb8f67640 10 monclient(hunting): authenticate will time out at 1790.093575s 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:60436/0 (socket says 172.21.15.105:60436) 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 -- 172.21.15.105:0/97700716 learned_addr learned my addr 172.21.15.105:0/97700716 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): get_auth_request con 0x7fedb40a7200 auth_method 0 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb1d9b640 1 --1- 172.21.15.105:0/97700716 >> v1:172.21.15.105:6789/0 conn(0x7fedb40a4910 0x7fedb41396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48838/0 (socket says 172.21.15.105:48838) 2024-09-17T16:41:46.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_done global_id 24837 payload 274 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb259c640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedb40a68e0 0x7fedb40a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 -- 172.21.15.105:0/97700716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedb40a68e0 msgr2=0x7fedb40a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb159a640 1 -- 172.21.15.105:0/97700716 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103291170 0 0) 0x7fedb406ebc0 con 0x7fedb40a4910 2024-09-17T16:41:46.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedb40a68e0 0x7fedb40a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 -- 172.21.15.105:0/97700716 >> v1:172.21.15.105:6789/0 conn(0x7fedb40a4910 legacy=0x7fedb41396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 monclient: found mon.noname-d 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb2d9d640 1 -- 172.21.15.105:0/97700716 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40a3690 con 0x7fedb40a7200 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.824+0000 7fedb159a640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fedac009e80 tx=0x7fedac01c8e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3d7ca4298e09b483 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 1 -- 172.21.15.105:0/97700716 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fedac0090d0 con 0x7fedb40a7200 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:46.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: dump: 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 1 -- 172.21.15.105:0/97700716 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac0a5bd0 con 0x7fedb40a7200 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 5 monclient: authenticate success, global_id 24837 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 4 set_mon_vals no callback set 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 1 -- 172.21.15.105:0/97700716 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fedac0a5d80 con 0x7fedb40a7200 2024-09-17T16:41:46.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb159a640 10 monclient: dump: 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: shutdown 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/97700716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 msgr2=0x7fedb4139db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fedac009e80 tx=0x7fedac01c8e0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/97700716 shutdown_connections 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fedb40a7200 0x7fedb4139db0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- 172.21.15.105:0/97700716 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedb40a68e0 0x7fedb40a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/97700716 >> 172.21.15.105:0/97700716 conn(0x7fedb4068ba0 msgr2=0x7fedb4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/97700716 shutdown_connections 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/97700716 wait complete. 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: build_initial_monmap 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: monmap: 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:46.845 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 librados: starting msgr at 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 librados: starting objecter 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 Processor -- start 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- start start 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 librados: setting wanted keys 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 librados: calling monclient init 2024-09-17T16:41:46.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: init 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _add_conns ranks=[5,2,8,7,0,6,3,4,1] 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fedb40a4910 0x7fedb413de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-h con 0x7fedb40a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedb40a68e0 0x7fedb413e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-b con 0x7fedb40a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 0x7fedb413e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): picked mon.noname-e con 0x7fedb40a7200 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fedb413a2f0 con 0x7fedb40a68e0 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fedb406e630 con 0x7fedb40a4910 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fedb413a470 con 0x7fedb40a7200 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient(hunting): authenticate will time out at 1790.098581s 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 0x7fedb413e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 0x7fedb413e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59548/0 (socket says 172.21.15.105:59548) 2024-09-17T16:41:46.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 learned_addr learned my addr 172.21.15.105:0/3035233918 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): get_auth_request con 0x7fedb40a7200 auth_method 0 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedb40a68e0 0x7fedb413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): get_auth_request con 0x7fedb40a68e0 auth_method 0 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fedb40a4910 0x7fedb413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient(hunting): handle_auth_done global_id 54665 payload 824 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fedb40a4910 msgr2=0x7fedb413de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fedb40a4910 0x7fedb413de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedb40a68e0 msgr2=0x7fedb413e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedb40a68e0 0x7fedb413e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 monclient: found mon.noname-e 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb4122740 con 0x7fedb40a7200 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb1d9b640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fedb40a68e0 0x7fedb413e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:46.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb259c640 10 monclient: get_auth_request con 0x7fedb40a4910 auth_method 0 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fedb40a4910 0x7fedb413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb2d9d640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 0x7fedb413e890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fedac0af040 tx=0x7fedac019680 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 client.?.objecter ms_handle_connect 0x7fedb40a7200 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fedac0bc020 con 0x7fedb40a7200 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: dump: 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.857 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:41:46.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac003bb0 con 0x7fedb40a7200 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 5 monclient: authenticate success, global_id 54665 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _renew_subs 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb4123580 con 0x7fedb40a7200 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fedac003d60 con 0x7fedb40a7200 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7feda2ffd640 10 monclient: dump: 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.859 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.860 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 client.54665.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _renew_subs 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb0d99640 4 set_mon_vals no callback set 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb406dd90 con 0x7fedb40a7200 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.828+0000 7fedb8f67640 1 librados: init done 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb8f67640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb8f67640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 0x7fedb413aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fedb4051e80 con 0x7fedb413a8c0 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 0x7fedb413aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient: get_auth_request con 0x7fedb413a8c0 auth_method 0 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fedac0bf020 con 0x7fedb40a7200 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 10 monclient(hunting): handle_auth_done global_id 54674 payload 824 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedb40a6ff0 con 0x7fedb413a8c0 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fed9403e4b0 0x7fed94040970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7fedac0b3040 con 0x7fedb40a7200 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 client.54665.objecter ms_dispatch 0x7fedb4000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 3 client.54665.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:46.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 3 client.54665.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 0x7fedb413aca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feda800d790 tx=0x7feda800dc50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 20 client.54665.objecter dump_active .. 0 homeless 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 client.54665.objecter ms_handle_connect 0x7fedb413a8c0 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 client.54665.objecter resend_mon_ops 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feda8016070 con 0x7fedb413a8c0 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 monclient: dump: 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.863 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb1d9b640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fed9403e4b0 0x7fed94040970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb1d9b640 10 monclient: get_auth_request con 0x7fed9403e4b0 auth_method 0 2024-09-17T16:41:46.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feda8008700 con 0x7fedb413a8c0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 msgr2=0x7fedb413aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 0x7fedb413aca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feda800d790 tx=0x7feda800dc50 comp rx=0 tx=0).stop 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 msgr2=0x7fedb413aca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413a8c0 0x7fedb413aca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7fedb1d9b640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fed9403e4b0 0x7fed94040970 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7feda4007f40 tx=0x7feda4007ed0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.832+0000 7feda2ffd640 10 client.54665.objecter ms_handle_connect 0x7fed9403e4b0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb8f67640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb8f67640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 0x7fedb41db110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb8f67640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fedb40a6cc0 con 0x7fedb413b1e0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 0x7fedb41db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:46.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient: get_auth_request con 0x7fedb413b1e0 auth_method 0 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 10 monclient(hunting): handle_auth_done global_id 54683 payload 824 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.836+0000 7fedb259c640 1 -- 172.21.15.105:0/3035233918 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fedb413f900 con 0x7fedb413b1e0 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 client.54665.objecter ms_handle_connect 0x7fedb413b1e0 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 client.54665.objecter resend_mon_ops 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb259c640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 0x7fedb41db110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feda8000c10 tx=0x7feda80046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feda8029020 con 0x7fedb413b1e0 2024-09-17T16:41:46.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 monclient: dump: 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:46.867 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7feda80113c0 con 0x7fedb413b1e0 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 msgr2=0x7fedb41db110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 0x7fedb41db110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feda8000c10 tx=0x7feda80046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 msgr2=0x7fedb41db110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:46.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7feda2ffd640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb413b1e0 0x7fedb41db110 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feda8000c10 tx=0x7feda80046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 413837, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:46.869 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.870 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.871 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:46.873 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:46.874 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:46.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.878 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:46.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.885 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.886 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.887 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.888 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.889 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:46.890 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:46.891 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:46.892 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:46.893 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 10 librados: watch_flush enter 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 10 librados: watch_flush exit 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 20 client.54665.objecter shutdown clearing up homeless session... 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 10 client.54665.objecter successfully canceled tick 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fed9403e4b0 msgr2=0x7fed94040970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fed9403e4b0 0x7fed94040970 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7feda4007f40 tx=0x7feda4007ed0 comp rx=0 tx=0).stop 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 10 monclient: shutdown 2024-09-17T16:41:46.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 msgr2=0x7fedb413e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 --2- 172.21.15.105:0/3035233918 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fedb40a7200 0x7fedb413e890 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fedac0af040 tx=0x7fedac019680 comp rx=0 tx=0).stop 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 reap_dead start 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb2d9d640 1 -- 172.21.15.105:0/3035233918 reap_dead start 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 shutdown_connections 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 >> 172.21.15.105:0/3035233918 conn(0x7fedb4068ba0 msgr2=0x7fedb40aa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 shutdown_connections 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 -- 172.21.15.105:0/3035233918 wait complete. 2024-09-17T16:41:46.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:46.840+0000 7fedb8f67640 1 librados: shutdown 2024-09-17T16:41:46.896 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:46.897 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:41:46.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:41:47.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.028+0000 7f0c764b3640 10 monclient: get_monmap_and_config 2024-09-17T16:41:47.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.028+0000 7f0c764b3640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient: monmap: 2024-09-17T16:41:47.039 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:47.035750+0000 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:47.035750+0000 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.040 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.041 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.041 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.041 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:47.041 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:47.041 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 Processor -- start 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 -- start start 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient: init 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient: _add_conns ranks=[9,10,5,0,7,4,8,1,3,6,2] 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c700a4750 0x7f0c700a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-a con 0x7f0c700a4750 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:47.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0c700a0ae0 0x7f0c700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-f con 0x7f0c700a0ae0 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c7009e130 0x7f0c701365e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-h con 0x7f0c7009e130 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0c7009e5b0 con 0x7f0c700a4750 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f0c700a55b0 con 0x7f0c700a0ae0 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0c700a62b0 con 0x7f0c7009e130 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c764b3640 10 monclient(hunting): authenticate will time out at 1790.301339s 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c700a4750 0x7f0c700a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c700a4750 0x7f0c700a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33234/0 (socket says 172.21.15.105:33234) 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 -- 172.21.15.105:0/640391036 learned_addr learned my addr 172.21.15.105:0/640391036 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): get_auth_request con 0x7f0c700a4750 auth_method 0 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c67fff640 1 --2- 172.21.15.105:0/640391036 >> v2:172.21.15.160:3300/0 conn(0x7f0c700a0ae0 0x7f0c700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c74a50640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c7009e130 0x7f0c701365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c67fff640 10 monclient(hunting): get_auth_request con 0x7f0c700a0ae0 auth_method 0 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c74a50640 10 monclient(hunting): get_auth_request con 0x7f0c7009e130 auth_method 0 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c74a50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c74a50640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c74a50640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_done global_id 36198 payload 274 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 -- 172.21.15.105:0/640391036 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c7009e130 msgr2=0x7f0c701365e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c7009e130 0x7f0c701365e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 -- 172.21.15.105:0/640391036 >> v2:172.21.15.160:3300/0 conn(0x7f0c700a0ae0 msgr2=0x7f0c700a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/640391036 >> v2:172.21.15.160:3300/0 conn(0x7f0c700a0ae0 0x7f0c700a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 monclient: found mon.noname-a 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.032+0000 7f0c6ffff640 1 -- 172.21.15.105:0/640391036 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7009e7c0 con 0x7f0c700a4750 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c700a4b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a6700 tx=0x7f0c6001c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9426eddcc177fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 1 -- 172.21.15.105:0/640391036 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c600b1070 con 0x7f0c700a4750 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: dump: 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.049 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 1 -- 172.21.15.105:0/640391036 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c600a4c90 con 0x7f0c700a4750 2024-09-17T16:41:47.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 1 -- 172.21.15.105:0/640391036 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c600aac20 con 0x7f0c700a4750 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 5 monclient: authenticate success, global_id 36198 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 4 set_mon_vals no callback set 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6f7fe640 10 monclient: dump: 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.051 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: shutdown 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- 172.21.15.105:0/640391036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 msgr2=0x7f0c700a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c700a4b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a6700 tx=0x7f0c6001c120 comp rx=0 tx=0).stop 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- 172.21.15.105:0/640391036 shutdown_connections 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c7009e130 0x7f0c701365e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- 172.21.15.105:0/640391036 >> v2:172.21.15.160:3300/0 conn(0x7f0c700a0ae0 0x7f0c700a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- 172.21.15.105:0/640391036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c700a4b30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- 172.21.15.105:0/640391036 >> 172.21.15.105:0/640391036 conn(0x7f0c700a93f0 msgr2=0x7f0c700a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- 172.21.15.105:0/640391036 shutdown_connections 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- 172.21.15.105:0/640391036 wait complete. 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: monmap: 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:47.053 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 librados: starting msgr at 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 librados: starting objecter 2024-09-17T16:41:47.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 Processor -- start 2024-09-17T16:41:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- start start 2024-09-17T16:41:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 librados: setting wanted keys 2024-09-17T16:41:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 librados: calling monclient init 2024-09-17T16:41:47.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: init 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient: _add_conns ranks=[4,0,1,2,6,7,3,5,8] 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-c con 0x7f0c7009e130 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c70134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-a con 0x7f0c700a4750 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013a4a0 0x7f0c7013a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): picked mon.noname-f con 0x7f0c7013a4a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c701402d0 con 0x7f0c700a4750 2024-09-17T16:41:47.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7009ea50 con 0x7f0c7013a4a0 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f0c70140450 con 0x7f0c7009e130 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 10 monclient(hunting): authenticate will time out at 1790.306636s 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c70134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:60654/0 (socket says 172.21.15.105:60654) 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 -- 172.21.15.105:0/3288952606 learned_addr learned my addr 172.21.15.105:0/3288952606 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): get_auth_request con 0x7f0c7009e130 auth_method 0 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): get_auth_request con 0x7f0c700a4750 auth_method 0 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c74a50640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013a4a0 0x7f0c7013a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c74a50640 10 monclient(hunting): get_auth_request con 0x7f0c7013a4a0 auth_method 0 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c74a50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c74a50640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c74a50640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient(hunting): handle_auth_done global_id 74461 payload 824 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013a4a0 msgr2=0x7f0c7013a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013a4a0 0x7f0c7013a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 msgr2=0x7f0c70134500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c70134500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 monclient: found mon.noname-c 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7013c5d0 con 0x7f0c7009e130 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c67fff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700a4750 0x7f0c70134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:47.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6ffff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c6001a7d0 tx=0x7f0c60018ec0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 client.?.objecter ms_handle_connect 0x7f0c7009e130 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c600bc070 con 0x7f0c7009e130 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: dump: 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.066 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c600ac8b0 con 0x7f0c7009e130 2024-09-17T16:41:47.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c764b3640 5 monclient: authenticate success, global_id 74461 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c600b1070 con 0x7f0c7009e130 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6effd640 4 set_mon_vals no callback set 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.036+0000 7f0c6d7fa640 10 monclient: dump: 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.068 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 monclient: _renew_subs 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c701209d0 con 0x7f0c7009e130 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 client.74461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:47.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 monclient: _renew_subs 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7013cac0 con 0x7f0c7009e130 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 1 librados: init done 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 0x7f0c7013b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7011f4b0 con 0x7f0c7013ae40 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0c600b5070 con 0x7f0c7009e130 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 0x7f0c7013b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c5003e550 0x7f0c50040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient: get_auth_request con 0x7f0c7013ae40 auth_method 0 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f0c600e99e0 con 0x7f0c7009e130 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 client.74461.objecter ms_dispatch 0x7f0c70000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 3 client.74461.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 3 client.74461.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 20 client.74461.objecter dump_active .. 0 homeless 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c74a50640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c5003e550 0x7f0c50040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c74a50640 10 monclient: get_auth_request con 0x7f0c5003e550 auth_method 0 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 10 monclient(hunting): handle_auth_done global_id 84142 payload 824 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c70072610 con 0x7f0c7013ae40 2024-09-17T16:41:47.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 client.74461.objecter ms_handle_connect 0x7f0c7013ae40 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 client.74461.objecter resend_mon_ops 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c67fff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 0x7f0c7013b220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a5810 tx=0x7f0c58007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c58018070 con 0x7f0c7013ae40 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 monclient: dump: 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.072 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.073 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0c5800eca0 con 0x7f0c7013ae40 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 msgr2=0x7f0c7013b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 0x7f0c7013b220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a5810 tx=0x7f0c58007f60 comp rx=0 tx=0).stop 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 msgr2=0x7f0c7013b220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c7013ae40 0x7f0c7013b220 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a5810 tx=0x7f0c58007f60 comp rx=0 tx=0).stop 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c6d7fa640 10 client.74461.objecter ms_handle_connect 0x7f0c5003e550 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.040+0000 7f0c74a50640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c5003e550 0x7f0c50040a10 secure :-1 s=READY pgs=3596 cs=0 l=1 rev1=1 crypto rx=0x7f0c7009e540 tx=0x7f0c6800b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.048+0000 7f0c764b3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.048+0000 7f0c764b3640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 0x7f0c700a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.048+0000 7f0c764b3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.048+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70051e80 con 0x7f0c700a0a70 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 0x7f0c700a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient: get_auth_request con 0x7f0c700a0a70 auth_method 0 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 10 monclient(hunting): handle_auth_done global_id 84151 payload 824 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 1 -- 172.21.15.105:0/3288952606 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0c70074a30 con 0x7f0c700a0a70 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c67fff640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 0x7f0c700a0e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a5810 tx=0x7f0c580046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 client.74461.objecter ms_handle_connect 0x7f0c700a0a70 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 client.74461.objecter resend_mon_ops 2024-09-17T16:41:47.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c580273f0 con 0x7f0c700a0a70 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 monclient: dump: 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.076 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f0c58014ab0 con 0x7f0c700a0a70 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 msgr2=0x7f0c700a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 0x7f0c700a0e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c700a5810 tx=0x7f0c580046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 msgr2=0x7f0c700a0e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c6d7fa640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c700a0a70 0x7f0c700a0e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 14287, 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:47.078 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.079 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.080 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:47.081 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:47.082 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.083 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.085 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.086 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.089 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.090 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.091 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:47.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.094 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.095 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.096 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:47.097 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:47.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.099 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:47.100 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 10 librados: watch_flush enter 2024-09-17T16:41:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 10 librados: watch_flush exit 2024-09-17T16:41:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 20 client.74461.objecter shutdown clearing up homeless session... 2024-09-17T16:41:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 10 client.74461.objecter successfully canceled tick 2024-09-17T16:41:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c5003e550 msgr2=0x7f0c50040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c5003e550 0x7f0c50040a10 secure :-1 s=READY pgs=3596 cs=0 l=1 rev1=1 crypto rx=0x7f0c7009e540 tx=0x7f0c6800b040 comp rx=0 tx=0).stop 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 10 monclient: shutdown 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 msgr2=0x7f0c70133fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c74a50640 1 -- 172.21.15.105:0/3288952606 reap_dead start 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c6001a7d0 tx=0x7f0c60018ec0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 shutdown_connections 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 --2- 172.21.15.105:0/3288952606 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f0c7009e130 0x7f0c70133fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 >> 172.21.15.105:0/3288952606 conn(0x7f0c700a93f0 msgr2=0x7f0c70072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 shutdown_connections 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 -- 172.21.15.105:0/3288952606 wait complete. 2024-09-17T16:41:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.052+0000 7f0c764b3640 1 librados: shutdown 2024-09-17T16:41:47.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:47.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:41:47.103 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:41:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: get_monmap_and_config 2024-09-17T16:41:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: monmap: 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:47.215895+0000 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:47.215895+0000 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.220 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:47.221 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:47.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 Processor -- start 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 -- start start 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: init 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient: _add_conns ranks=[0,3,8,6,9,10,5,2,7,4,1] 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d0c0a4750 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c0a5240 0x7f7d0c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-g con 0x7f7d0c0a5240 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 0x7f7d0c13f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-e con 0x7f7d0c13cd70 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c0a0b90 con 0x7f7d0c0a4750 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c0a08b0 con 0x7f7d0c0a5240 2024-09-17T16:41:47.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f7d0c0a0e70 con 0x7f7d0c13cd70 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d11efc640 10 monclient(hunting): authenticate will time out at 1790.481806s 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7d0c0a4750 0x7f7d0c0a4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48852/0 (socket says 172.21.15.105:48852) 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0b7fe640 1 -- 172.21.15.105:0/163340568 learned_addr learned my addr 172.21.15.105:0/163340568 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 0x7f7d0c13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): get_auth_request con 0x7f7d0c13cd70 auth_method 0 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c0a5240 0x7f7d0c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): get_auth_request con 0x7f7d0c0a5240 auth_method 0 2024-09-17T16:41:47.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 1 -- 172.21.15.105:0/163340568 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760106081 0 0) 0x7f7d040014a0 con 0x7f7d0c0a4750 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 10 monclient(hunting): _negotiate 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 10 monclient(hunting): my global_id is 36216 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0a7fc640 1 -- 172.21.15.105:0/163340568 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf4003530 con 0x7f7d0c0a4750 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_done global_id 54692 payload 274 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 -- 172.21.15.105:0/163340568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c0a5240 msgr2=0x7f7d0c13c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c0a5240 0x7f7d0c13c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 -- 172.21.15.105:0/163340568 >> v1:172.21.15.105:6789/0 conn(0x7f7d0c0a4750 legacy=0x7f7d0c0a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 monclient: found mon.noname-e 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.212+0000 7f7d0bfff640 1 -- 172.21.15.105:0/163340568 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c09fca0 con 0x7f7d0c13cd70 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 0x7f7d0c13f160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc01c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=770ef12e0fbe03aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 1 -- 172.21.15.105:0/163340568 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7cfc0b0070 con 0x7f7d0c13cd70 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:47.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: dump: 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 1 -- 172.21.15.105:0/163340568 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0aa810 con 0x7f7d0c13cd70 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 5 monclient: authenticate success, global_id 54692 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 1 -- 172.21.15.105:0/163340568 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7cfc0a9300 con 0x7f7d0c13cd70 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 4 set_mon_vals no callback set 2024-09-17T16:41:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0a7fc640 10 monclient: dump: 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.231 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: shutdown 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- 172.21.15.105:0/163340568 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 msgr2=0x7f7d0c13f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 0x7f7d0c13f160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097f0 tx=0x7f7cfc01c3a0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- 172.21.15.105:0/163340568 shutdown_connections 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c13cd70 0x7f7d0c13f160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- 172.21.15.105:0/163340568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c0a5240 0x7f7d0c13c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- 172.21.15.105:0/163340568 >> 172.21.15.105:0/163340568 conn(0x7f7d0c0a5840 msgr2=0x7f7d0c0a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- 172.21.15.105:0/163340568 shutdown_connections 2024-09-17T16:41:47.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- 172.21.15.105:0/163340568 wait complete. 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: monmap: 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:47.241 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.242 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 librados: starting msgr at 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 librados: starting objecter 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 Processor -- start 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- start start 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 librados: setting wanted keys 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 librados: calling monclient init 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: init 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient: _add_conns ranks=[2,8,3,0,7,1,6,4,5] 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d0c0a4750 0x7f7d0c131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-b con 0x7f7d0c0a4750 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-e con 0x7f7d0c0a5240 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c13cd70 0x7f7d0c09da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): picked mon.noname-g con 0x7f7d0c13cd70 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c142040 con 0x7f7d0c0a4750 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c141ec0 con 0x7f7d0c13cd70 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f7d0c1421c0 con 0x7f7d0c0a5240 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d11efc640 10 monclient(hunting): authenticate will time out at 1790.486952s 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d0c0a4750 0x7f7d0c131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:59570/0 (socket says 172.21.15.105:59570) 2024-09-17T16:41:47.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 learned_addr learned my addr 172.21.15.105:0/3721875043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): get_auth_request con 0x7f7d0c0a5240 auth_method 0 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request con 0x7f7d0c0a4750 auth_method 0 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient(hunting): handle_auth_done global_id 54701 payload 824 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c13cd70 msgr2=0x7f7d0c09da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c13cd70 0x7f7d0c09da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c13cd70 0x7f7d0c09da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d0c0a4750 msgr2=0x7f7d0c131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d0c0a4750 0x7f7d0c131c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 monclient: found mon.noname-e 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c0a3f20 con 0x7f7d0c0a5240 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7d0c0a4750 0x7f7d0c131c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000e930 tx=0x7f7d0000edf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 client.?.objecter ms_handle_connect 0x7f7d0c0a5240 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 msgr2=0x7f7d0c1321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 msgr2=0x7f7d0c1321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000e930 tx=0x7f7d0000edf0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000e930 tx=0x7f7d0000edf0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 msgr2=0x7f7d0c1321c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f7d0c0a5240 0x7f7d0c1321c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient: _add_conns ranks=[5,7,1,6,0,4] 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7cf8004f70 0x7f7cf80073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): picked mon.noname-h con 0x7f7cf8004f70 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7cf8007f10 0x7f7cf800a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): picked mon.noname-i con 0x7f7cf8007f10 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 0x7f7cf800d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): picked mon.noname-f con 0x7f7cf800af40 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800dff0 con 0x7f7cf800af40 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f7cf800e170 con 0x7f7cf8004f70 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f7cf800e370 con 0x7f7cf8007f10 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.216+0000 7f7cf3fff640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7cf8004f70 0x7f7cf80073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 0x7f7cf800d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request con 0x7f7cf8004f70 auth_method 0 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7cf8007f10 0x7f7cf800a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): get_auth_request con 0x7f7cf800af40 auth_method 0 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): get_auth_request con 0x7f7cf8007f10 auth_method 0 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:41:47.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient(hunting): handle_auth_done global_id 54701 payload 293 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7cf8007f10 msgr2=0x7f7cf800a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f7cf8007f10 0x7f7cf800a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7cf8004f70 msgr2=0x7f7cf80073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7cf8004f70 0x7f7cf80073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 monclient: found mon.noname-f 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:41:47.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf800e630 con 0x7f7cf800af40 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 -- 172.21.15.105:0/3721875043 reap_dead start 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 client.?.objecter ms_handle_connect 0x7f7cf800af40 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0bfff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 0x7f7cf800d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0ab000 tx=0x7f7d00009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7cfc0b0040 con 0x7f7cf800af40 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: dump: 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.251 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 5 monclient: authenticate success, global_id 54701 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 monclient: _renew_subs 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c09f310 con 0x7f7cf800af40 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0aa520 con 0x7f7cf800af40 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d09ffb640 4 set_mon_vals no callback set 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7cfc0aae30 con 0x7f7cf800af40 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 client.54701.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: dump: 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.253 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 monclient: _renew_subs 2024-09-17T16:41:47.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c13b530 con 0x7f7cf800af40 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 1 librados: init done 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c051e80 con 0x7f7d0c131c80 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient: get_auth_request con 0x7f7d0c131c80 auth_method 0 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7cfc004040 con 0x7f7cf800af40 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_done global_id 54759 payload 824 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 0x7f7cf8045070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d0c09f9f0 con 0x7f7d0c131c80 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f7cfc0e8120 con 0x7f7cf800af40 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 client.54701.objecter ms_dispatch 0x7f7d0c000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 3 client.54701.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 3 client.54701.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 20 client.54701.objecter dump_active .. 0 homeless 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 0x7f7cf8045070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 client.54701.objecter ms_handle_connect 0x7f7d0c131c80 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 client.54701.objecter resend_mon_ops 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d04010470 tx=0x7f7d04010930 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7d0affd640 10 monclient: get_auth_request con 0x7f7cf8042bb0 auth_method 0 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7d04019020 con 0x7f7d0c131c80 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.220+0000 7f7cf3fff640 10 monclient: dump: 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.257 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 10 client.54701.objecter ms_handle_connect 0x7f7cf8042bb0 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7d0affd640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 0x7f7cf8045070 secure :-1 s=READY pgs=3597 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0a51c0 tx=0x7f7d00008030 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7d040072e0 con 0x7f7d0c131c80 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 msgr2=0x7f7d0c132060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d04010470 tx=0x7f7d04010930 comp rx=0 tx=0).stop 2024-09-17T16:41:47.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 msgr2=0x7f7d0c132060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.224+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d04010470 tx=0x7f7d04010930 comp rx=0 tx=0).stop 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d11efc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d11efc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c0a4b30 con 0x7f7d0c1325a0 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient: get_auth_request con 0x7f7d0c1325a0 auth_method 0 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 10 monclient(hunting): handle_auth_done global_id 54768 payload 824 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 1 -- 172.21.15.105:0/3721875043 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7d0c06e230 con 0x7f7d0c1325a0 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7d0b7fe640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d0401a000 tx=0x7f7d04001aa0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 10 client.54701.objecter ms_handle_connect 0x7f7d0c1325a0 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 10 client.54701.objecter resend_mon_ops 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7d04019040 con 0x7f7d0c1325a0 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.228+0000 7f7cf3fff640 10 monclient: dump: 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.260 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.261 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f7d04015650 con 0x7f7d0c1325a0 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 msgr2=0x7f7d0c0720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d0401a000 tx=0x7f7d04001aa0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 msgr2=0x7f7d0c0720e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7cf3fff640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d0401a000 tx=0x7f7d04001aa0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 414239, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:47.263 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.264 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.265 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:47.266 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.267 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:47.268 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.269 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.270 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.272 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:47.273 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.274 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.275 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.276 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.277 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.278 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.279 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.280 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:47.281 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.282 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.283 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:47.284 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 10 librados: watch_flush enter 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 10 librados: watch_flush exit 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 20 client.54701.objecter shutdown clearing up homeless session... 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 10 client.54701.objecter successfully canceled tick 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 msgr2=0x7f7cf8045070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 0x7f7cf8045070 secure :-1 s=READY pgs=3597 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0a51c0 tx=0x7f7d00008030 comp rx=0 tx=0).stop 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 10 monclient: shutdown 2024-09-17T16:41:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 msgr2=0x7f7cf800d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 0x7f7cf800d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0ab000 tx=0x7f7d00009190 comp rx=0 tx=0).stop 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 shutdown_connections 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f7cf8042bb0 0x7f7cf8045070 unknown :-1 s=CLOSED pgs=3597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf800af40 0x7f7cf800d400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c131c80 0x7f7d0c132060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 --2- 172.21.15.105:0/3721875043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d0c1325a0 0x7f7d0c0720e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 >> 172.21.15.105:0/3721875043 conn(0x7f7d0c0a5840 msgr2=0x7f7d0c0a7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 shutdown_connections 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 -- 172.21.15.105:0/3721875043 wait complete. 2024-09-17T16:41:47.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.232+0000 7f7d11efc640 1 librados: shutdown 2024-09-17T16:41:47.288 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:47.288 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:41:47.288 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:41:47.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: get_monmap_and_config 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: monmap: 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:47.404492+0000 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:47.404492+0000 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.409 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 Processor -- start 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 -- start start 2024-09-17T16:41:47.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: init 2024-09-17T16:41:47.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient: _add_conns ranks=[3,2,9,10,8,0,1,4,5,7,6] 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 0x7fc1ec0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-g con 0x7fc1ec0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-b con 0x7fc1ec0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-a con 0x7fc1ec1396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ec0a3fc0 con 0x7fc1ec1396a0 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0a63c0 con 0x7fc1ec0a4910 2024-09-17T16:41:47.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0a7740 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f45b6640 10 monclient(hunting): authenticate will time out at 1790.670215s 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33258/0 (socket says 172.21.15.105:33258) 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:34882/0 (socket says 172.21.15.105:34882) 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 1 -- 172.21.15.105:0/69039397 learned_addr learned my addr 172.21.15.105:0/69039397 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): get_auth_request con 0x7fc1ec1396a0 auth_method 0 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): get_auth_request con 0x7fc1ec0a4910 auth_method 0 2024-09-17T16:41:47.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2352640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 0x7fc1ec0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_done global_id 84179 payload 274 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 -- 172.21.15.105:0/69039397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 msgr2=0x7fc1ec0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 0x7fc1ec0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 -- 172.21.15.105:0/69039397 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 msgr2=0x7fc1ec139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/69039397 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 monclient: found mon.noname-b 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f1b51640 1 -- 172.21.15.105:0/69039397 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec06ebc0 con 0x7fc1ec0a4910 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2352640 10 monclient: get_auth_request con 0x7fc1ec0a68e0 auth_method 0 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2352640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 0x7fc1ec0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.400+0000 7fc1f2b53640 1 --2- 172.21.15.105:0/69039397 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009d90 tx=0x7fc1e001c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26127ed6f8415980 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 1 -- 172.21.15.105:0/69039397 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e00b2070 con 0x7fc1ec0a4910 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:47.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: dump: 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.418 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 1 -- 172.21.15.105:0/69039397 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e00a5ab0 con 0x7fc1ec0a4910 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 5 monclient: authenticate success, global_id 84179 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:47.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 1 -- 172.21.15.105:0/69039397 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e00a5ce0 con 0x7fc1ec0a4910 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 4 set_mon_vals no callback set 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1350640 10 monclient: dump: 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.420 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: shutdown 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- 172.21.15.105:0/69039397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 msgr2=0x7fc1ec0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009d90 tx=0x7fc1e001c800 comp rx=0 tx=0).stop 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- 172.21.15.105:0/69039397 shutdown_connections 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/69039397 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec1396a0 0x7fc1ec139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec0a7200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/69039397 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1ec0a68e0 0x7fc1ec0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- 172.21.15.105:0/69039397 >> 172.21.15.105:0/69039397 conn(0x7fc1ec068ba0 msgr2=0x7fc1ec068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- 172.21.15.105:0/69039397 shutdown_connections 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- 172.21.15.105:0/69039397 wait complete. 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: monmap: 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:47.422 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 librados: starting msgr at 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 librados: starting objecter 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 Processor -- start 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- start start 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 librados: setting wanted keys 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 librados: calling monclient init 2024-09-17T16:41:47.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: init 2024-09-17T16:41:47.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient: _add_conns ranks=[2,0,6,5,3,1,8,7,4] 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-b con 0x7fc1ec0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 0x7fc1ec138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-a con 0x7fc1ec0a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1ec1396a0 0x7fc1ec13d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): picked mon.noname-d con 0x7fc1ec1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec06e630 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0a7af0 con 0x7fc1ec0a4910 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fc1ec143320 con 0x7fc1ec1396a0 2024-09-17T16:41:47.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f45b6640 10 monclient(hunting): authenticate will time out at 1790.675326s 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 0x7fc1ec138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 0x7fc1ec138500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33262/0 (socket says 172.21.15.105:33262) 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 -- 172.21.15.105:0/29824063 learned_addr learned my addr 172.21.15.105:0/29824063 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): get_auth_request con 0x7fc1ec0a68e0 auth_method 0 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): get_auth_request con 0x7fc1ec0a4910 auth_method 0 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1ec1396a0 0x7fc1ec13d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): get_auth_request con 0x7fc1ec1396a0 auth_method 0 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2b53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient(hunting): handle_auth_done global_id 36234 payload 824 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1ec1396a0 msgr2=0x7fc1ec13d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fc1ec1396a0 0x7fc1ec13d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 msgr2=0x7fc1ec137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 monclient: found mon.noname-a 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f1b51640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec13fcd0 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.404+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0a4910 0x7fc1ec137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f1b51640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 0x7fc1ec138500 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc1e001c630 tx=0x7fc1e00195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.?.objecter ms_handle_connect 0x7fc1ec0a68e0 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e00b2050 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: dump: 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.435 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e00be070 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 5 monclient: authenticate success, global_id 36234 2024-09-17T16:41:47.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0003bb0 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f0b4f640 4 set_mon_vals no callback set 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: dump: 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.437 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 monclient: _renew_subs 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec123b70 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 client.36234.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 monclient: _renew_subs 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec077c10 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 1 librados: init done 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 0x7fc1ec13e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc1ec122650 con 0x7fc1ec13dd00 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc1e0004040 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 0x7fc1ec13e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient: get_auth_request con 0x7fc1ec13dd00 auth_method 0 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7fc1e00eb200 con 0x7fc1ec0a68e0 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.36234.objecter ms_dispatch 0x7fc1ec000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:47.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 3 client.36234.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 3 client.36234.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 20 client.36234.objecter dump_active .. 0 homeless 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2b53640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2b53640 10 monclient: get_auth_request con 0x7fc1d803e550 auth_method 0 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 10 monclient(hunting): handle_auth_done global_id 74255 payload 824 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1ec0aa070 con 0x7fc1ec13dd00 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.36234.objecter ms_handle_connect 0x7fc1ec13dd00 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.36234.objecter resend_mon_ops 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 0x7fc1ec13e0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00fc70 tx=0x7fc1dc00d590 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1dc00dd10 con 0x7fc1ec13dd00 2024-09-17T16:41:47.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1f2b53640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 secure :-1 s=READY pgs=3598 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40096b0 tx=0x7fc1e4009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: dump: 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.441 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 client.36234.objecter ms_handle_connect 0x7fc1d803e550 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc1dc00dd10 con 0x7fc1ec13dd00 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:47.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 msgr2=0x7fc1ec13e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 0x7fc1ec13e0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00fc70 tx=0x7fc1dc00d590 comp rx=0 tx=0).stop 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 msgr2=0x7fc1ec13e0e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.408+0000 7fc1eaffd640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13dd00 0x7fc1ec13e0e0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00fc70 tx=0x7fc1dc00d590 comp rx=0 tx=0).stop 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f45b6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 0x7fc1ec0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f45b6640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc1ec051e80 con 0x7fc1ec13e620 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 0x7fc1ec0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient: get_auth_request con 0x7fc1ec13e620 auth_method 0 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 10 monclient(hunting): handle_auth_done global_id 74264 payload 824 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 1 -- 172.21.15.105:0/29824063 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc1ec13f080 con 0x7fc1ec13e620 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1f2352640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 0x7fc1ec0a0a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00afd0 tx=0x7fc1dc004740 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 client.36234.objecter ms_handle_connect 0x7fc1ec13e620 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 client.36234.objecter resend_mon_ops 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc1dc006760 con 0x7fc1ec13e620 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 monclient: dump: 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.445 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fc1dc006760 con 0x7fc1ec13e620 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 msgr2=0x7fc1ec0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 0x7fc1ec0a0a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00afd0 tx=0x7fc1dc004740 comp rx=0 tx=0).stop 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 msgr2=0x7fc1ec0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.416+0000 7fc1eaffd640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc1ec13e620 0x7fc1ec0a0a20 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00afd0 tx=0x7fc1dc004740 comp rx=0 tx=0).stop 2024-09-17T16:41:47.446 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 146890, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:47.447 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.448 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.449 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.450 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.451 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.452 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.453 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.454 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.455 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:47.457 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.461 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:47.463 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.464 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.467 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.468 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:47.468 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 10 librados: watch_flush enter 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 10 librados: watch_flush exit 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 20 client.36234.objecter shutdown clearing up homeless session... 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 10 client.36234.objecter successfully canceled tick 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 msgr2=0x7fc1d8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc1d803e550 0x7fc1d8040a10 secure :-1 s=READY pgs=3598 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40096b0 tx=0x7fc1e4009240 comp rx=0 tx=0).stop 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 10 monclient: shutdown 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 msgr2=0x7fc1ec138500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 --2- 172.21.15.105:0/29824063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0a68e0 0x7fc1ec138500 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc1e001c630 tx=0x7fc1e00195a0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f2b53640 1 -- 172.21.15.105:0/29824063 reap_dead start 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f2b53640 1 -- 172.21.15.105:0/29824063 reap_dead start 2024-09-17T16:41:47.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 shutdown_connections 2024-09-17T16:41:47.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 >> 172.21.15.105:0/29824063 conn(0x7fc1ec068ba0 msgr2=0x7fc1ec0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 shutdown_connections 2024-09-17T16:41:47.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 -- 172.21.15.105:0/29824063 wait complete. 2024-09-17T16:41:47.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.420+0000 7fc1f45b6640 1 librados: shutdown 2024-09-17T16:41:47.471 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:47.471 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:41:47.471 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: get_monmap_and_config 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: monmap: 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:47.584293+0000 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:47.584293+0000 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.588 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:47.589 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 Processor -- start 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 -- start start 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: init 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient: _add_conns ranks=[1,9,3,7,2,0,8,6,5,4,10] 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f101c137830 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f101c13a6a0 0x7f101c13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-a con 0x7f101c13a6a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c13cfd0 0x7f101c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-g con 0x7f101c13cfd0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f101c0a33c0 con 0x7f101c137830 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f101c0a1d60 con 0x7f101c13a6a0 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f101c0a1660 con 0x7f101c13cfd0 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f1021ba0640 10 monclient(hunting): authenticate will time out at 1790.850064s 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f101c13a6a0 0x7f101c13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f101c13a6a0 0x7f101c13ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33268/0 (socket says 172.21.15.105:33268) 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 -- 172.21.15.105:0/794551424 learned_addr learned my addr 172.21.15.105:0/794551424 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): get_auth_request con 0x7f101c13a6a0 auth_method 0 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101bfff640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c13cfd0 0x7f101c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101bfff640 10 monclient(hunting): get_auth_request con 0x7f101c13cfd0 auth_method 0 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient(hunting): handle_auth_done global_id 36243 payload 274 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 -- 172.21.15.105:0/794551424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c13cfd0 msgr2=0x7f101c13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c13cfd0 0x7f101c13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 -- 172.21.15.105:0/794551424 >> v1:172.21.15.160:6789/0 conn(0x7f101c137830 legacy=0x7f101c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 monclient: found mon.noname-a 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.580+0000 7f101affd640 1 -- 172.21.15.105:0/794551424 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c0a1ae0 con 0x7f101c13a6a0 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101affd640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c13a6a0 0x7f101c13ca90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1014009870 tx=0x7f101401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04c03853bf97e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 1 -- 172.21.15.105:0/794551424 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10140b0070 con 0x7f101c13a6a0 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: dump: 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.596 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 1 -- 172.21.15.105:0/794551424 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10140aa810 con 0x7f101c13a6a0 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 5 monclient: authenticate success, global_id 36243 2024-09-17T16:41:47.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 1 -- 172.21.15.105:0/794551424 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10140a9300 con 0x7f101c13a6a0 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101a7fc640 10 monclient: dump: 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.598 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 4 set_mon_vals no callback set 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: shutdown 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- 172.21.15.105:0/794551424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c13a6a0 msgr2=0x7f101c13ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c13a6a0 0x7f101c13ca90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1014009870 tx=0x7f101401c540 comp rx=0 tx=0).stop 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- 172.21.15.105:0/794551424 shutdown_connections 2024-09-17T16:41:47.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c13cfd0 0x7f101c13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- 172.21.15.105:0/794551424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f101c13a6a0 0x7f101c13ca90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- 172.21.15.105:0/794551424 >> 172.21.15.105:0/794551424 conn(0x7f101c0aa390 msgr2=0x7f101c0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- 172.21.15.105:0/794551424 shutdown_connections 2024-09-17T16:41:47.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- 172.21.15.105:0/794551424 wait complete. 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: monmap: 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:47.606 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 librados: starting msgr at 2024-09-17T16:41:47.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 librados: starting objecter 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 Processor -- start 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- start start 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 librados: setting wanted keys 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 librados: calling monclient init 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: init 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient: _add_conns ranks=[3,5,2,1,7,0,4,6,8] 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c137830 0x7f101c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-g con 0x7f101c137830 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f101c13a6a0 0x7f101c06d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-h con 0x7f101c13a6a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): picked mon.noname-b con 0x7f101c13cfd0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:47.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f101c1422a0 con 0x7f101c13cfd0 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f101c142120 con 0x7f101c137830 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f101c142420 con 0x7f101c13a6a0 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f1021ba0640 10 monclient(hunting): authenticate will time out at 1790.855346s 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:34904/0 (socket says 172.21.15.105:34904) 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 -- 172.21.15.105:0/2048733434 learned_addr learned my addr 172.21.15.105:0/2048733434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): get_auth_request con 0x7f101c13cfd0 auth_method 0 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101affd640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f101c13a6a0 0x7f101c06d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c137830 0x7f101c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient(hunting): handle_auth_done global_id 84197 payload 824 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f101c13a6a0 msgr2=0x7f101c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f101c13a6a0 0x7f101c06d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c137830 msgr2=0x7f101c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c137830 0x7f101c06d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 monclient: found mon.noname-b 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:47.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101bfff640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c072250 con 0x7f101c13cfd0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101b7fe640 10 monclient: get_auth_request con 0x7f101c137830 auth_method 0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101affd640 10 monclient: get_auth_request con 0x7f101c13a6a0 auth_method 0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f101c137830 0x7f101c06d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.584+0000 7f101affd640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f101c13a6a0 0x7f101c06d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101bfff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f100c007d40 tx=0x7f100c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.?.objecter ms_handle_connect 0x7f101c13cfd0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f100c00edd0 con 0x7f101c13cfd0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: dump: 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.611 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c019070 con 0x7f101c13cfd0 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f100c014d00 con 0x7f101c13cfd0 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: dump: 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.613 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 5 monclient: authenticate success, global_id 84197 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 monclient: _renew_subs 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1019ffb640 4 set_mon_vals no callback set 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c06edc0 con 0x7f101c13cfd0 2024-09-17T16:41:47.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 client.84197.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 monclient: _renew_subs 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c072530 con 0x7f101c13cfd0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 1 librados: init done 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 0x7f101c1d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f101c051e80 con 0x7f101c0a05a0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f100c014420 con 0x7f101c13cfd0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f100803e550 0x7f1008040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f100c0509e0 con 0x7f101c13cfd0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.84197.objecter ms_dispatch 0x7f101c000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 3 client.84197.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:47.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 3 client.84197.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 0x7f101c1d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 20 client.84197.objecter dump_active .. 0 homeless 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient: get_auth_request con 0x7f101c0a05a0 auth_method 0 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101affd640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f100803e550 0x7f1008040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101affd640 10 monclient: get_auth_request con 0x7f100803e550 auth_method 0 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 10 monclient(hunting): handle_auth_done global_id 64546 payload 824 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f101c139e40 con 0x7f101c0a05a0 2024-09-17T16:41:47.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.84197.objecter ms_handle_connect 0x7f101c0a05a0 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 0x7f101c1d1b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f101000d480 tx=0x7f101000d940 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.84197.objecter resend_mon_ops 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 client.84197.objecter ms_handle_connect 0x7f100803e550 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f101affd640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f100803e550 0x7f1008040a10 secure :-1 s=READY pgs=3599 cs=0 l=1 rev1=1 crypto rx=0x7f10140ad000 tx=0x7f10140a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10100074a0 con 0x7f101c0a05a0 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: dump: 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.617 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1010007030 con 0x7f101c0a05a0 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:47.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 msgr2=0x7f101c1d1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 0x7f101c1d1b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f101000d480 tx=0x7f101000d940 comp rx=0 tx=0).stop 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 msgr2=0x7f101c1d1b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.588+0000 7f1003fff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c0a05a0 0x7f101c1d1b10 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f101000d480 tx=0x7f101000d940 comp rx=0 tx=0).stop 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1021ba0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1021ba0640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 0x7f101c02d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1021ba0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f101c0a5d50 con 0x7f101c02d450 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 0x7f101c02d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient: get_auth_request con 0x7f101c02d450 auth_method 0 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 10 monclient(hunting): handle_auth_done global_id 64555 payload 824 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 1 -- 172.21.15.105:0/2048733434 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f101c137c10 con 0x7f101c02d450 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f101b7fe640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 0x7f101c02d830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1010017000 tx=0x7f1010003770 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 client.84197.objecter ms_handle_connect 0x7f101c02d450 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 client.84197.objecter resend_mon_ops 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1010028020 con 0x7f101c02d450 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 monclient: dump: 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.620 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.621 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f10100128f0 con 0x7f101c02d450 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 msgr2=0x7f101c02d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 0x7f101c02d830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1010017000 tx=0x7f1010003770 comp rx=0 tx=0).stop 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 msgr2=0x7f101c02d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:47.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.596+0000 7f1003fff640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f101c02d450 0x7f101c02d830 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1010017000 tx=0x7f1010003770 comp rx=0 tx=0).stop 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 280701, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:47.623 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.624 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.625 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:47.626 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:47.627 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.628 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.629 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:47.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.632 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.635 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.639 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.641 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.642 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.643 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:47.644 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:47.645 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:47.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 10 librados: watch_flush enter 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 10 librados: watch_flush exit 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 20 client.84197.objecter shutdown clearing up homeless session... 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 10 client.84197.objecter successfully canceled tick 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f100803e550 msgr2=0x7f1008040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f100803e550 0x7f1008040a10 secure :-1 s=READY pgs=3599 cs=0 l=1 rev1=1 crypto rx=0x7f10140ad000 tx=0x7f10140a6000 comp rx=0 tx=0).stop 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 10 monclient: shutdown 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 msgr2=0x7f101c06dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f101bfff640 1 -- 172.21.15.105:0/2048733434 reap_dead start 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f100c007d40 tx=0x7f100c00e4b0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 shutdown_connections 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 --2- 172.21.15.105:0/2048733434 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f101c13cfd0 0x7f101c06dca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 >> 172.21.15.105:0/2048733434 conn(0x7f101c0aa390 msgr2=0x7f101c13d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 shutdown_connections 2024-09-17T16:41:47.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 -- 172.21.15.105:0/2048733434 wait complete. 2024-09-17T16:41:47.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.600+0000 7f1021ba0640 1 librados: shutdown 2024-09-17T16:41:47.648 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:47.648 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:41:47.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 10 monclient: get_monmap_and_config 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 10 monclient: monmap: 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:47.758442+0000 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:47.758442+0000 2024-09-17T16:41:47.762 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:47.763 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 1 Processor -- start 2024-09-17T16:41:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 1 -- start start 2024-09-17T16:41:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.752+0000 7f32bdc31640 10 monclient: init 2024-09-17T16:41:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient: _add_conns ranks=[9,4,0,7,6,1,10,5,8,2,3] 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32b80a68e0 0x7f32b80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-a con 0x7f32b80a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 0x7f32b80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-c con 0x7f32b80a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32b81396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b806ebc0 con 0x7f32b81396a0 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f32b80a3fc0 con 0x7f32b80a68e0 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f32b80a63c0 con 0x7f32b80a4910 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient(hunting): authenticate will time out at 1791.024208s 2024-09-17T16:41:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32b80a68e0 0x7f32b80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 0x7f32b80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f32b81396a0 0x7f32b8139b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55168/0 (socket says 172.21.15.105:55168) 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b7fff640 1 -- 172.21.15.105:0/1730569770 learned_addr learned my addr 172.21.15.105:0/1730569770 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): get_auth_request con 0x7f32b80a68e0 auth_method 0 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): get_auth_request con 0x7f32b80a4910 auth_method 0 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient(hunting): handle_auth_done global_id 74470 payload 274 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 -- 172.21.15.105:0/1730569770 >> v2:172.21.15.105:3300/0 conn(0x7f32b80a68e0 msgr2=0x7f32b80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 --2- 172.21.15.105:0/1730569770 >> v2:172.21.15.105:3300/0 conn(0x7f32b80a68e0 0x7f32b80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 -- 172.21.15.105:0/1730569770 >> v1:172.21.15.105:6789/0 conn(0x7f32b81396a0 legacy=0x7f32b8139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 1 -- 172.21.15.105:0/1730569770 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257183624 0 0) 0x7f32b806ebc0 con 0x7f32b81396a0 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 monclient: found mon.noname-c 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 -- 172.21.15.105:0/1730569770 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80a3690 con 0x7f32b80a4910 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b6ffd640 1 --2- 172.21.15.105:0/1730569770 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 0x7f32b80a7200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32a80097c0 tx=0x7f32a801c360 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ed47904eb58ba7b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 1 -- 172.21.15.105:0/1730569770 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32a80b0070 con 0x7f32b80a4910 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: dump: 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.776 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 1 -- 172.21.15.105:0/1730569770 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a80aa810 con 0x7f32b80a4910 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 5 monclient: authenticate success, global_id 74470 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 4 set_mon_vals no callback set 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 1 -- 172.21.15.105:0/1730569770 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32a80a9300 con 0x7f32b80a4910 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32b67fc640 10 monclient: dump: 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.778 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 10 monclient: shutdown 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- 172.21.15.105:0/1730569770 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 msgr2=0x7f32b80a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 --2- 172.21.15.105:0/1730569770 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 0x7f32b80a7200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32a80097c0 tx=0x7f32a801c360 comp rx=0 tx=0).stop 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- 172.21.15.105:0/1730569770 shutdown_connections 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 --2- 172.21.15.105:0/1730569770 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f32b80a4910 0x7f32b80a7200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 --2- 172.21.15.105:0/1730569770 >> v2:172.21.15.105:3300/0 conn(0x7f32b80a68e0 0x7f32b80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- 172.21.15.105:0/1730569770 >> 172.21.15.105:0/1730569770 conn(0x7f32b8068ba0 msgr2=0x7f32b8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- 172.21.15.105:0/1730569770 shutdown_connections 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.756+0000 7f32bdc31640 1 -- 172.21.15.105:0/1730569770 wait complete. 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: build_initial_monmap 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: monmap: 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:47.780 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:47.781 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 librados: starting msgr at 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 librados: starting objecter 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 Processor -- start 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- start start 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 librados: setting wanted keys 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 librados: calling monclient init 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: init 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _add_conns ranks=[7,1,0,3,4,2,8,5,6] 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f32b80a4910 0x7f32b80a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-i con 0x7f32b80a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32b80a68e0 0x7f32b809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-f con 0x7f32b80a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): picked mon.noname-a con 0x7f32b81396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81444b0 con 0x7f32b81396a0 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8144330 con 0x7f32b80a68e0 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f32b8144630 con 0x7f32b80a4910 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient(hunting): authenticate will time out at 1791.029390s 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46122/0 (socket says 172.21.15.105:46122) 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 -- 172.21.15.105:0/2261669142 learned_addr learned my addr 172.21.15.105:0/2261669142 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:47.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): get_auth_request con 0x7f32b81396a0 auth_method 0 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b6ffd640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32b80a68e0 0x7f32b809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b77fe640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f32b80a4910 0x7f32b80a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b6ffd640 10 monclient(hunting): get_auth_request con 0x7f32b80a68e0 auth_method 0 2024-09-17T16:41:47.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b77fe640 10 monclient(hunting): get_auth_request con 0x7f32b80a4910 auth_method 0 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient(hunting): handle_auth_done global_id 36270 payload 824 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 -- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f32b80a4910 msgr2=0x7f32b80a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f32b80a4910 0x7f32b80a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 -- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32b80a68e0 msgr2=0x7f32b809da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32b80a68e0 0x7f32b809da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:47.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 monclient: found mon.noname-a 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 -- 172.21.15.105:0/2261669142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80a9ee0 con 0x7f32b81396a0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b7fff640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c6b0 tx=0x7f32b000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 client.?.objecter ms_handle_connect 0x7f32b81396a0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32b0018070 con 0x7f32b81396a0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: dump: 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.786 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: _finish_auth 0 2024-09-17T16:41:47.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b000a640 con 0x7f32b81396a0 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32b000ace0 con 0x7f32b81396a0 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 5 monclient: authenticate success, global_id 36270 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32b5ffb640 4 set_mon_vals no callback set 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 monclient: dump: 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.788 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _renew_subs 2024-09-17T16:41:47.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b80a0a10 con 0x7f32b81396a0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 client.36270.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _renew_subs 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b80a29a0 con 0x7f32b81396a0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 librados: init done 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f32b000a870 con 0x7f32b81396a0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8139b50 con 0x7f32b81396a0 2024-09-17T16:41:47.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 0x7f32a4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(541..541 src has 1..541) ==== 6657+0+0 (secure 0 0 0) 0x7f32b004f9c0 con 0x7f32b81396a0 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 10 client.36270.objecter ms_dispatch 0x7f32b8000ba0 osd_map(541..541 src has 1..541) 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 3 client.36270.objecter handle_osd_map got epochs [541,541] > 0 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 3 client.36270.objecter handle_osd_map decoding full epoch 541 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.760+0000 7f329ffff640 20 client.36270.objecter dump_active .. 0 homeless 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.764+0000 7f32b77fe640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 0x7f32a4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.764+0000 7f32b77fe640 10 monclient: get_auth_request con 0x7f32a403e580 auth_method 0 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.764+0000 7f329ffff640 10 client.36270.objecter ms_handle_connect 0x7f32a403e580 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.764+0000 7f32b77fe640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 0x7f32a4040a40 secure :-1 s=READY pgs=3600 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009af0 tx=0x7f32ac008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:47.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.768+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32b0011360 con 0x7f32b81396a0 2024-09-17T16:41:47.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.768+0000 7f329ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:47.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.768+0000 7f329ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:47.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.984+0000 7f32bdc31640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:41:47.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.984+0000 7f32bdc31640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:41:47.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.984+0000 7f32bdc31640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:47.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.984+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f32b80a0be0 con 0x7f32b81396a0 2024-09-17T16:41:47.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.988+0000 7f329ffff640 1 -- 172.21.15.105:0/2261669142 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f32b00070d0 con 0x7f32b81396a0 2024-09-17T16:41:47.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.988+0000 7f329ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:41:47.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.988+0000 7f329ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:47.999 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:47.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.988+0000 7f32bdc31640 10 librados: watch_flush enter 2024-09-17T16:41:48.000 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.000 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=17,config=2,health=10,kv=10,logm=61} crc {auth=2732920997,config=692619927,health=3777914565,kv=2863082231,logm=693355958}) 2024-09-17T16:41:48.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 10 librados: watch_flush exit 2024-09-17T16:41:48.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 20 client.36270.objecter shutdown clearing up homeless session... 2024-09-17T16:41:48.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 10 client.36270.objecter successfully canceled tick 2024-09-17T16:41:48.001 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.001 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.002 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=1035615830,config=692619927,health=286278900,kv=865020949,logm=529276082}) 2024-09-17T16:41:48.002 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.002 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 msgr2=0x7f32a4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:48.002 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=10,logm=60} crc {auth=798211198,config=692619927,health=161364307,kv=2863082231,logm=1427742084}) 2024-09-17T16:41:48.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 0x7f32a4040a40 secure :-1 s=READY pgs=3600 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009af0 tx=0x7f32ac008040 comp rx=0 tx=0).stop 2024-09-17T16:41:48.003 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 10 monclient: shutdown 2024-09-17T16:41:48.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:48.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 msgr2=0x7f32b809df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:48.004 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c6b0 tx=0x7f32b000cb70 comp rx=0 tx=0).stop 2024-09-17T16:41:48.004 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=11,kv=10,logm=60} crc {auth=1035615830,config=692619927,health=3011492542,kv=2863082231,logm=4030210391}) 2024-09-17T16:41:48.004 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=10,logm=60} crc {auth=1863349889,config=692619927,health=4234991974,kv=2863082231,logm=1279908184}) 2024-09-17T16:41:48.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 shutdown_connections 2024-09-17T16:41:48.005 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f32a403e580 0x7f32a4040a40 unknown :-1 s=CLOSED pgs=3600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32b81396a0 0x7f32b809df40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32b80a68e0 0x7f32b809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 --2- 172.21.15.105:0/2261669142 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f32b80a4910 0x7f32b80a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:48.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 >> 172.21.15.105:0/2261669142 conn(0x7f32b8068ba0 msgr2=0x7f32b80a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:48.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=15,config=2,health=11,kv=10,logm=62} crc {auth=1149944946,config=692619927,health=2183865940,kv=2863082231,logm=2484702992}) 2024-09-17T16:41:48.007 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=16,config=2,health=11,kv=10,logm=61} crc {auth=523134951,config=692619927,health=3426394697,kv=2863082231,logm=3061886804}) 2024-09-17T16:41:48.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=1,health=11,kv=10,logm=63} crc {auth=1469686169,config=3678701417,health=4053104945,kv=2863082231,logm=469895466}) 2024-09-17T16:41:48.008 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=16,config=2,health=11,kv=9,logm=62} crc {auth=3902169342,config=692619927,health=1824742390,kv=2502644681,logm=1265861972}) 2024-09-17T16:41:48.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 shutdown_connections 2024-09-17T16:41:48.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 -- 172.21.15.105:0/2261669142 wait complete. 2024-09-17T16:41:48.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:47.992+0000 7f32bdc31640 1 librados: shutdown 2024-09-17T16:41:48.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=7309201}) 2024-09-17T16:41:48.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.012 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3682893980}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3903331413}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2836989120}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.013 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1226360413}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3775007503}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=23216821}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2551248590}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:47.996+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2767178376}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=86883258}) 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.014 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=177760623}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=460865089}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=506344483}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2547642283}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3704511003}) 2024-09-17T16:41:48.015 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.016 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.016 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3595858463}) 2024-09-17T16:41:48.016 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.016 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949678042}) 2024-09-17T16:41:48.016 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.000+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3476736146}) 2024-09-17T16:41:48.018 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.018 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.018 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529776029}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3781723311}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3788276881}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=757215321}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=509046121}) 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.019 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2953686672}) 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2368035788}) 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=462980815}) 2024-09-17T16:41:48.020 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.008+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1626816221}) 2024-09-17T16:41:48.022 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:41:48.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.024 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1234605769}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1025784686}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558871807}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.025 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2953573662}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2200195763}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2153304069}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1987363004}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580815607}) 2024-09-17T16:41:48.026 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.012+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3072666133}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341172151}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2724193508}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3032262940}) 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.031 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2258141843}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=987703200}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2407502806}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1055288732}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927252933}) 2024-09-17T16:41:48.032 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.016+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3278819656}) 2024-09-17T16:41:48.033 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.033 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.033 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3830263051}) 2024-09-17T16:41:48.033 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274532362}) 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1514563162}) 2024-09-17T16:41:48.034 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2641495479}) 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3313526799}) 2024-09-17T16:41:48.035 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182542381}) 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2956194900}) 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255457038}) 2024-09-17T16:41:48.036 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.020+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=564103694}) 2024-09-17T16:41:48.046 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.046 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.046 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307214703}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220486410}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1977183448}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1709785730}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=86798354}) 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.047 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2341250237}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1681335977}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757607881}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.024+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1415006413}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3920011539}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4065024539}) 2024-09-17T16:41:48.048 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1625786992}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1549950260}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1523074939}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1443621891}) 2024-09-17T16:41:48.049 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1372412334}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084445351}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.028+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=821000196}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2920734134}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1931436930}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1951481133}) 2024-09-17T16:41:48.050 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=82298510}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=893438663}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=475922504}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=902982184}) 2024-09-17T16:41:48.051 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2461877425}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.032+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1509485743}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3133030825}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1046758053}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=378674878}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.052 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1937617545}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3437021505}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=973352338}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2561352830}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1503617618}) 2024-09-17T16:41:48.053 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.036+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1080448908}) 2024-09-17T16:41:48.063 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1763589055}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3791723772}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1447330614}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1850980303}) 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.064 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=424042870}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=309848781}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3111850606}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1043580661}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1821193350}) 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.065 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=451780541}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.040+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3730001698}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1318414037}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2412750654}) 2024-09-17T16:41:48.066 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3005014913}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3292357916}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3551939161}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621406791}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3010640248}) 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.067 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2026126717}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2047624241}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1671652831}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2535668013}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.068 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2989999502}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2466019271}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=283281917}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1065421465}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.044+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3151062491}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.069 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597734360}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2332929893}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4268707783}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3089243625}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=80981891}) 2024-09-17T16:41:48.070 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2578413563}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4024453636}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1875552478}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.048+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3268978761}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149455793}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1295375206}) 2024-09-17T16:41:48.071 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1532707939}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3424475743}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4005527287}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694440732}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=520341805}) 2024-09-17T16:41:48.072 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265487948}) 2024-09-17T16:41:48.073 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.052+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1024998668}) 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=860956080}) 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.103 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4130417908}) 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3368043818}) 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3768874728}) 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.104 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3633594685}) 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1394026492}) 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.105 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2476937652}) 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1691035628}) 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.056+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3903934928}) 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.106 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245755921}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=669611249}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4274899451}) 2024-09-17T16:41:48.107 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1351883273}) 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1333253579}) 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.108 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3398258760}) 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2390253935}) 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2406744612}) 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.060+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=642170625}) 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.109 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3018202053}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3742224955}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3296815071}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.110 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2041877073}) 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4129892538}) 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3360848394}) 2024-09-17T16:41:48.111 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3953014788}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584587998}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=687776444}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=245655464}) 2024-09-17T16:41:48.112 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1780025594}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=879066738}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3979601432}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.113 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=825224935}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743117228}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1831115773}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291239522}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.064+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1738134735}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389632023}) 2024-09-17T16:41:48.114 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2813086405}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1705825407}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=215213970}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=760034002}) 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.115 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3581074669}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2268541818}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301114751}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.068+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1952449914}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531243228}) 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.116 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2095409524}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1975653163}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3110813553}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=718684468}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2577910498}) 2024-09-17T16:41:48.117 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=891570085}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1607882084}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.072+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2508095893}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3691296961}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2048788175}) 2024-09-17T16:41:48.118 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=586842286}) 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1043477079}) 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.119 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1155609482}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3248920056}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3717286223}) 2024-09-17T16:41:48.120 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1528314839}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.076+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2266657471}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3222553334}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053425901}) 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.121 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1177646484}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2065157596}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1157082282}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3087602710}) 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.122 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2145253769}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1982924304}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=786854544}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4004493831}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1818598238}) 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.123 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1321002838}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=583728772}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.080+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2892652785}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1130382143}) 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.124 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3406052960}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2597054758}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=507001002}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935843642}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2497999863}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.125 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1255623765}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1363884223}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4198624631}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2706378872}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3810176804}) 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.126 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1188447258}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.084+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=786877993}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2215622996}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394360100}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=756494405}) 2024-09-17T16:41:48.127 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2759619275}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4177770245}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4027345302}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3944424377}) 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.128 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3799108155}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.088+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3825371141}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517765228}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1647008473}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=285446584}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.129 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2252237980}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1857283018}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4005110022}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=455958501}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346296361}) 2024-09-17T16:41:48.130 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.092+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4113125202}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327616305}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409706419}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=166095773}) 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.166 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2847485139}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2739388803}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1534684790}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3383038609}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179684867}) 2024-09-17T16:41:48.167 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.100+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1892526300}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3617038261}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2014227104}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4085156933}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=343883483}) 2024-09-17T16:41:48.168 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2203830517}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3779949054}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1713429408}) 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.169 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496418391}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.104+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678814337}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1459042737}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3574875384}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=238576554}) 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.170 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2760611241}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3231173482}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1400748145}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4101013072}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943796765}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=701231126}) 2024-09-17T16:41:48.171 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2006278525}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3826740738}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3314275220}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2307827759}) 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.172 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3253727007}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2972547683}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2159381771}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967433060}) 2024-09-17T16:41:48.173 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.108+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4264220334}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4117913450}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1796729778}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2715793481}) 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.174 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4055937643}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3088054489}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184530488}) 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.175 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1959589677}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4223217596}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1807463992}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2188734720}) 2024-09-17T16:41:48.176 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2329034211}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3598257113}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2953913784}) 2024-09-17T16:41:48.177 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1180322264}) 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.112+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=289565943}) 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.178 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3558805337}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007245076}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195952779}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523743150}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3796993507}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3431843566}) 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.179 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=176971473}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1111653309}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1663168841}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1740823212}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818517002}) 2024-09-17T16:41:48.180 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.116+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2240576971}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3530749178}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3006335983}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3123464197}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3886772826}) 2024-09-17T16:41:48.181 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2650165707}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3981552475}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3211973080}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=366100662}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1538406421}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=682035598}) 2024-09-17T16:41:48.182 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2866492193}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3596947652}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1310297368}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2342553704}) 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.183 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519246359}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3923329938}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1428233982}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.120+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=64802173}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1656845237}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=335334838}) 2024-09-17T16:41:48.184 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302027077}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=503096604}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1081158046}) 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.185 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=967549736}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1041631557}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3892043631}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.124+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4160808365}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=77929303}) 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.186 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2141857554}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2264883436}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4258122612}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=384273387}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3665197061}) 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.187 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1492367227}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=34340072}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2370814552}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3995234660}) 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.188 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=662720774}) 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3559342578}) 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.189 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1845244124}) 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4113611613}) 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.190 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2680248026}) 2024-09-17T16:41:48.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2392141811}) 2024-09-17T16:41:48.191 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4077545396}) 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.128+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4103621441}) 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3942718935}) 2024-09-17T16:41:48.192 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3572018823}) 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3892010022}) 2024-09-17T16:41:48.193 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2500571732}) 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2006773673}) 2024-09-17T16:41:48.194 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=65240640}) 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1365483971}) 2024-09-17T16:41:48.195 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3604057581}) 2024-09-17T16:41:48.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.132+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3453775418}) 2024-09-17T16:41:48.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.196 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=306142346}) 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623231380}) 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.197 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4034830069}) 2024-09-17T16:41:48.198 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.198 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.198 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4279142142}) 2024-09-17T16:41:48.198 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.198 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=706770979}) 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2611149683}) 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.199 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3300399735}) 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3231477520}) 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.140+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=798552324}) 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.200 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3315677447}) 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3411469293}) 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.201 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.202 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3136319046}) 2024-09-17T16:41:48.202 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.202 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.202 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2401362865}) 2024-09-17T16:41:48.202 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1647323117}) 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2872434959}) 2024-09-17T16:41:48.203 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.204 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.204 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=597556313}) 2024-09-17T16:41:48.204 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.204 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277825932}) 2024-09-17T16:41:48.204 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.148+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=38237063}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2627940044}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1307533271}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3808740637}) 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.209 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1041282349}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1005154957}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2602238538}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3612388984}) 2024-09-17T16:41:48.210 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319731223}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.192+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1004426989}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4256244866}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3722564674}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.211 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2807549764}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2491239380}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=75026514}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1589646365}) 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.212 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.213 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=725845626}) 2024-09-17T16:41:48.213 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.213 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=142925176}) 2024-09-17T16:41:48.213 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.196+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3205596944}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1059759793}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=115293291}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4216081860}) 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.215 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=76058622}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1229536998}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2378764110}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.200+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=678398138}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481693214}) 2024-09-17T16:41:48.216 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.204+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4142515588}) 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=409175332}) 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.220 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2397383033}) 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=605601535}) 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.221 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2133185093}) 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.208+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=9793255}) 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.222 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=59838522}) 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1607767935}) 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2541218828}) 2024-09-17T16:41:48.223 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.212+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2871356775}) 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2463371373}) 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.228 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3581839718}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=772183077}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1837945139}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1376864228}) 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.229 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=21336762}) 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=543126254}) 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145226533}) 2024-09-17T16:41:48.230 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.216+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176370576}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247397832}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=331450700}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=930151035}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.231 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2269187327}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=834109404}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1058314053}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=235701957}) 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.232 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3335683565}) 2024-09-17T16:41:48.233 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.220+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=849574119}) 2024-09-17T16:41:48.233 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.233 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.233 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4263637341}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2726457840}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919819217}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3850334768}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3224305095}) 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.234 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=384270745}) 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2809450850}) 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=982636035}) 2024-09-17T16:41:48.235 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1789034325}) 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3868258909}) 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1751123556}) 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.237 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4198339908}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1713954549}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4258995501}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=835892050}) 2024-09-17T16:41:48.238 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.239 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.224+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.239 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=447181865}) 2024-09-17T16:41:48.239 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.239 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489394741}) 2024-09-17T16:41:48.239 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=822681663}) 2024-09-17T16:41:48.430 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.430 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.430 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2870587906}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1852947778}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1641540601}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2920859719}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.431 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3668876156}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2285832276}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=522339685}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1149909884}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.228+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2057517785}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3914377699}) 2024-09-17T16:41:48.432 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=85695665}) 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=272598251}) 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.433 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=899905663}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2906503763}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1826161891}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3409193944}) 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.434 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=642996319}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2257047395}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3682674783}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4273296624}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1451820795}) 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.435 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=877781649}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1250158845}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=665848862}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2338702132}) 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.436 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950847776}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.232+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1255118567}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687807428}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165871270}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1230526175}) 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.437 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2153718300}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2027610975}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2470099088}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4099264952}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2144819371}) 2024-09-17T16:41:48.438 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=276157949}) 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2252257876}) 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.439 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3992166170}) 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710971841}) 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4057755126}) 2024-09-17T16:41:48.440 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2603889080}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3767686773}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2303067170}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2038423817}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.236+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2613927873}) 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.441 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2076778028}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=329296248}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1346866213}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=676945413}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2450573548}) 2024-09-17T16:41:48.442 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=880990604}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2551131027}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942697727}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=557690518}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=679590886}) 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.443 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1135661684}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1046143031}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1122471373}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2749918815}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.444 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1774016289}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=779822198}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474838247}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.240+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=449716288}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494986525}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1392068973}) 2024-09-17T16:41:48.445 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450864226}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1345820011}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=923298826}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1890985096}) 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.446 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3544049986}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1938313359}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2268677583}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936251462}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3587883020}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=232241798}) 2024-09-17T16:41:48.447 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3254443844}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3898440617}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771785316}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3747601474}) 2024-09-17T16:41:48.448 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3697878659}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.244+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1536973321}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722197862}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=485152679}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3354438716}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.449 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=16323605}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3939629487}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4060667075}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=332927923}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652824402}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=189526165}) 2024-09-17T16:41:48.450 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=389918646}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1255589587}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3958331751}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3962099837}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2690781308}) 2024-09-17T16:41:48.451 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3335045103}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3575546135}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88214126}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1567549708}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1149437161}) 2024-09-17T16:41:48.452 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2201535544}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1586797486}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1107148911}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.248+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=513703299}) 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.453 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1725135539}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3952349958}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785472841}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1974050023}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2676099516}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2699517317}) 2024-09-17T16:41:48.454 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3458831552}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=669474652}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2429560234}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3165582436}) 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.455 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3544893385}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043243524}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4230405896}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2419544585}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3968102697}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.456 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1711611464}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4111656025}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4088640516}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4286720673}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.457 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2480827023}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2255383953}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.252+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2151681636}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2241169147}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=124296531}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3481023423}) 2024-09-17T16:41:48.458 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3255278575}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=913226808}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=348338018}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=311618679}) 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.459 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3988231367}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1998391202}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1203220428}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1128616768}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889432191}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.460 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3453655364}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3579910746}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=778846978}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=839411398}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393133764}) 2024-09-17T16:41:48.461 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2133758218}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615561738}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1694244501}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2095491422}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3489175047}) 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.462 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4159888902}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3484860867}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2481627521}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902501088}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.256+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=321628430}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.463 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20324116}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1389715310}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2798430345}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3378074594}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1228395416}) 2024-09-17T16:41:48.464 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=241603755}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2417414219}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802166079}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=310846380}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072425151}) 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.465 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=608417503}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410800361}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1406945853}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4198266517}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.466 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=115633573}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3726258072}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796181085}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1479253475}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.260+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1664072221}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3407828460}) 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.467 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2423032575}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3730648775}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=827973036}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2618894396}) 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.468 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2242579132}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413950088}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2577043173}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3464142990}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2791325187}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3112266435}) 2024-09-17T16:41:48.469 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2772232834}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1119173998}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3220763132}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2215838611}) 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.470 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2485014814}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3139546801}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2618383978}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3293001600}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4080981209}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4247910406}) 2024-09-17T16:41:48.471 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3598102432}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=886170626}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2935203327}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3213060864}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.264+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1887830117}) 2024-09-17T16:41:48.472 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227833167}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1767035443}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3898039088}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3654575077}) 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.473 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2843696116}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3894826372}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=891669653}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695256958}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1132734782}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.474 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535939324}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3450350923}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=467562486}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2938625036}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.475 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=647137897}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1449585851}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2714605784}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976827622}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.268+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1061709939}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=394878184}) 2024-09-17T16:41:48.476 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=494884693}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2110537296}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2531715853}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2659373045}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.477 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2604538158}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2231274190}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185061962}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=105739271}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=774261073}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2777908046}) 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.478 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767143132}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1109465015}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2221284790}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=281037229}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.479 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=526203301}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1119354138}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1520294350}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322589784}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1668223237}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4110170756}) 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.480 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2503851961}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3343152274}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826824147}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2680511220}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017794803}) 2024-09-17T16:41:48.481 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.272+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1338236193}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238562546}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1959389339}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1388847374}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137889266}) 2024-09-17T16:41:48.482 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1703774813}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2336430508}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=686284659}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1256418857}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543212972}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.483 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3554658181}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=960070038}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3888772853}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3997570195}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=190478902}) 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.484 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2863003852}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=222761094}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1011397489}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.276+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4256722624}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=890446063}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.485 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=716305721}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3206599882}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=886120032}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=907121222}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3926952017}) 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.486 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4006819972}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1028209644}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2744717970}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3477704522}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2416118001}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.487 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=277574546}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2676370626}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=611748902}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=869206999}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1089136555}) 2024-09-17T16:41:48.488 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311599960}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2658681709}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2029667310}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649618182}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3501697237}) 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.489 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=134293463}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3326858076}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4191143942}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1888377513}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1478028391}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.280+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2468408395}) 2024-09-17T16:41:48.490 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327828582}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=393754314}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2411178237}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4150754191}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.491 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1344852106}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3849613161}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2830033912}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746708367}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1675631064}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498286964}) 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.492 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414520499}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3024219023}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=936099236}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2909366152}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1266781823}) 2024-09-17T16:41:48.493 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2039201990}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2260858556}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2056091037}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3271915026}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3722443933}) 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.494 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.284+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1643919693}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=933240469}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3206241408}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=410897845}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.495 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1699530615}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969066342}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1444542028}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2485144954}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3680319852}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1390103215}) 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.496 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4114767362}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=449243209}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1004282988}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4031477775}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2337599553}) 2024-09-17T16:41:48.497 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3723393791}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725357498}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3469970164}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1249986870}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3109050575}) 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.498 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2610677226}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3207568978}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2395540859}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243738198}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.288+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1046293206}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1290864383}) 2024-09-17T16:41:48.499 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2422361850}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1848907248}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4241207375}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4291610905}) 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.500 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3357582605}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2686360336}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829580481}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4037268477}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1126576693}) 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.501 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3085021501}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=972996141}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2615785720}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2260009006}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3435742300}) 2024-09-17T16:41:48.502 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3990065921}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.292+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948518252}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1651427596}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2580401392}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3839540336}) 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.503 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=490412964}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2155637623}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2249787434}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2981296688}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3157658031}) 2024-09-17T16:41:48.504 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864337283}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3300842941}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=895223530}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=215869003}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2365407087}) 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.505 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2603489629}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=532721535}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=733155703}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1954628957}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=856897554}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.296+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1144882044}) 2024-09-17T16:41:48.506 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744239708}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1671979886}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2383593034}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1182458}) 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.507 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3129112054}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2731614270}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4181123067}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1377262806}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1915189760}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.508 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2195265477}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1794087656}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3767479233}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3021044208}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3559886494}) 2024-09-17T16:41:48.509 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3602026676}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=651312628}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849146228}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.300+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2017867236}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3293774453}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.510 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=854558642}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4197853067}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=205981807}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3293836950}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2012186058}) 2024-09-17T16:41:48.511 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3896334848}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=504871061}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2175519279}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2094009726}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4235173746}) 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.512 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1785348381}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1238322757}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4206935688}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1334228017}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3120808799}) 2024-09-17T16:41:48.513 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3372406567}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.304+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4007970777}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1488096323}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=827045376}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=771981294}) 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.514 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1878471075}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=187425749}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1832169470}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1737542825}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514378082}) 2024-09-17T16:41:48.515 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2964336763}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3196491109}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3151405820}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3597133459}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1685509115}) 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.516 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2598761641}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3476548191}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2307047140}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1069687482}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=923703721}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3486792212}) 2024-09-17T16:41:48.517 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3710909908}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.308+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1139129549}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3341234080}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3879128426}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.518 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=799709350}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190442794}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785793441}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3057879458}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3067515902}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=730212686}) 2024-09-17T16:41:48.519 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3617312074}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=322011099}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=27249330}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1643404558}) 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.520 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1587936875}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600398343}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3765303959}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2332915451}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2007845146}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1000028637}) 2024-09-17T16:41:48.521 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=869960019}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2861831502}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4215487896}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=576729334}) 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.522 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332155226}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.312+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2342292511}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=257512997}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3251507789}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4166504522}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.523 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=845168429}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=479636525}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3542518051}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=125252683}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163474727}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1854351121}) 2024-09-17T16:41:48.524 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1728483345}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3195951609}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3200762355}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=403763998}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.525 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=521222398}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1781827983}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=709001568}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171234354}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.316+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2940968343}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1152982072}) 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.526 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2095064340}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3681215097}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2973386864}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=796602540}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1887980557}) 2024-09-17T16:41:48.527 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3016196469}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1058062331}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3785745369}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393316321}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662361340}) 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.528 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2441405564}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3591781852}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=407354661}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2147944345}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3638341832}) 2024-09-17T16:41:48.529 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3008605266}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1964729566}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.320+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689378430}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3690343457}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=616218278}) 2024-09-17T16:41:48.530 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4237473145}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1743997172}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732068962}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=767864530}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964626464}) 2024-09-17T16:41:48.531 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2418944603}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913881316}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173381721}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3553457864}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=861771714}) 2024-09-17T16:41:48.532 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1189707680}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=730836101}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1379651901}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088582142}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1227065655}) 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.533 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=483312680}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1106758849}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=710288782}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.324+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=132522875}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3547406695}) 2024-09-17T16:41:48.534 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2408483082}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3180536562}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401227412}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1052999677}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3050145460}) 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.535 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1217025051}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=446028455}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3378260818}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2990021787}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.536 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=526503953}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2374719251}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1352006222}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1473984223}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1439672143}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3678760399}) 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.537 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3032940113}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4087397020}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.328+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=476451347}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4053148730}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3576744373}) 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.538 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1425410301}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982568925}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503633831}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120651017}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2971486359}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.539 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3243180567}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=947316810}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3069398106}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1347591483}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074852738}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1756676879}) 2024-09-17T16:41:48.540 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=179803619}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=255994245}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683120531}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1783252961}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.332+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.541 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4103222596}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1505749671}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1521409519}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2249267262}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3609544215}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=763036520}) 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.542 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=971811563}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2396941927}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=461491172}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352305127}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1940746599}) 2024-09-17T16:41:48.543 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=41214745}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=227958201}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60292005}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1125091707}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.336+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3727377848}) 2024-09-17T16:41:48.544 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2512756483}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1960608774}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2746642679}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4224091816}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=87728431}) 2024-09-17T16:41:48.545 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2577719162}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1062741824}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=398413938}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4128735114}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3049571733}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.546 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3081550298}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=736213461}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=622137452}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2619491470}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=512603320}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=555285199}) 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.547 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3420723457}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=180674146}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833161070}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1714632359}) 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.548 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=831737410}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.340+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=741905402}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1767451556}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4004362167}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2855524074}) 2024-09-17T16:41:48.549 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3401727651}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3050327346}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1172923776}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3330597601}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.550 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2795147616}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3963888987}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=448640582}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3405147040}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2950577249}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1113710284}) 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.551 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3375437123}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3009197630}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1331547120}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1080648242}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.552 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=292107110}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2045522630}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831474332}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.344+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180280970}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2027723701}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1551762453}) 2024-09-17T16:41:48.553 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4167029620}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=986167256}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1559889030}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2722711384}) 2024-09-17T16:41:48.554 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2474337082}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176808394}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4041689762}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1737685268}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2948528173}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.555 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1142977745}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=261392504}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4175725588}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=562701445}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.556 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=323082802}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444898685}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168847848}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1788903086}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=852622227}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=935054020}) 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.557 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2131227337}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1565438797}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1063721485}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2315643523}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1410838458}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.348+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1884630038}) 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.558 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2322872718}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3557004876}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2301925336}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2402701592}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.559 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3205799020}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=219164089}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3830256126}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1419735849}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=597298868}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1197613700}) 2024-09-17T16:41:48.560 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=523746892}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244405546}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=684462027}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=399643653}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.561 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2878815099}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2203844370}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1444507647}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3906583432}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3967429075}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1252963575}) 2024-09-17T16:41:48.562 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.352+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3535390890}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=674570623}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3218941727}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3154855106}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.563 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3709408935}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1645069717}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064881684}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3250193941}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3242341784}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1651498292}) 2024-09-17T16:41:48.564 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=548322233}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=222819283}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1240073898}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1214304817}) 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.565 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=507470802}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3575071955}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4} crc {logm=944002867,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=3172904488,mon_config_key=1079238812}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=74,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=5} crc {logm=209919884,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgrstat=1465039363,mon_config_key=2893357816}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=1248024354,mdsmap=1248375647,mgr=3567252286,mgr_metadata=3792311847,mgrstat=1020952484,mon_config_key=3563937658}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=3358302795,mdsmap=1248375647,mgr=345108039,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2741641201,mon_config_key=4180278983}) 2024-09-17T16:41:48.566 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7} crc {logm=1440139522,mdsmap=1248375647,mgr=3925673059,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=4180278983}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=3} crc {logm=29209920,mdsmap=1248375647,mgr=345108039,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2962390786,mon_config_key=3560280948}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5} crc {logm=2617381934,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=811362990,mon_config_key=2893357816}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=76,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=1} crc {logm=1520106426,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2372458814,mon_config_key=3118387037}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.356+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=71,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8} crc {logm=1389781625,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=4130194924,mon_config_key=2704324773}) 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.567 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=34,osdmap=45} crc {mon_config_key=1289100850,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=929328177,osdmap=2629223013}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=3192417521,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=2775581097}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=45} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=2769114396,osdmap=453104122}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=31,osdmap=47} crc {mon_config_key=911725316,monmap=1360384041,nvmeofgw=2613832008,osd_metadata=3224233959,osd_pg_creating=882767698,osd_snap=694906025,osdmap=2798629702}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.568 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=34,osdmap=46} crc {mon_config_key=2237870845,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3571293177,osdmap=628485755}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32,osdmap=47} crc {mon_config_key=2905665840,monmap=1360384041,nvmeofgw=598337686,osd_metadata=4058618413,osd_pg_creating=882767698,osd_snap=1202403151,osdmap=3502467929}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=34,osdmap=44} crc {mon_config_key=911725316,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=929328177,osdmap=538706228}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=44} crc {mon_config_key=911725316,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=882767698,osd_snap=3172365789,osdmap=555158816}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.360+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=34,osdmap=45} crc {mon_config_key=2237870845,monmap=1360384042,nvmeofgw=598337686,osd_metadata=1916498675,osd_pg_creating=882767698,osd_snap=1383431252,osdmap=3337735333}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1320965772}) 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.569 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=662443044}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1399442567}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3359475203}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2985568019}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=327574247}) 2024-09-17T16:41:48.570 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2794271347}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2109752884}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4227625277}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2273300616}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3899807423}) 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.571 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=86183237}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=587507903}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2334752551}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1051388522}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.572 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2656446854}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3110042100}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.364+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3848805183}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3268536164}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=894895262}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3944598506}) 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.573 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1823109831}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2259748131}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2069089048}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1872805622}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4228741716}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.368+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=406937617}) 2024-09-17T16:41:48.574 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3404552898}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3171082966}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=416372166}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2775951143}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.575 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3605535348}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3651805453}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2787222206}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=63016968}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3512787856}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1778855949}) 2024-09-17T16:41:48.576 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=62736801}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2507567036}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4282290235}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.372+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3941652642}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.577 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4280770816}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1560594431}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2316203229}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.376+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1675428740}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3194145786}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4057403455}) 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.578 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2961808321}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1721152271}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3635344946}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3352554780}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=366594069}) 2024-09-17T16:41:48.579 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3706711237}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.380+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=920523731}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3589085104}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=229038749}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1830155777}) 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.580 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3214298098}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4105315135}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3953906049}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=185231478}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500167327}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.384+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2251948499}) 2024-09-17T16:41:48.581 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=39} crc {osdmap=3935372368}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=39} crc {osdmap=3713439567}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=42} crc {osdmap=4208892443}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=38} crc {osdmap=2918724160}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.582 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=40} crc {osdmap=1213665493}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=2061993797}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=42} crc {osdmap=3914547319}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=39} crc {osdmap=3860288401}) 2024-09-17T16:41:48.583 INFO:tasks.ceph.mon.d.smithi105.stderr:2024-09-17T16:41:48.400+0000 7f9438244640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=41} crc {osdmap=2280032137}) 2024-09-17T16:41:49.024 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:41:49.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:41:49.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: get_monmap_and_config 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: monmap: 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:49.149023+0000 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:49.149023+0000 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:49.153 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:49.154 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 Processor -- start 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 -- start start 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: init 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient: _add_conns ranks=[1,8,4,10,7,2,6,3,5,9,0] 2024-09-17T16:41:49.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f114c0a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-e con 0x7f114c0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-c con 0x7f114c0a73d0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114c06ebc0 con 0x7f114c0a68e0 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f114c0a3fc0 con 0x7f114c0a73d0 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f114c0a63c0 con 0x7f114c0a4910 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f1152b0f640 10 monclient(hunting): authenticate will time out at 1792.414764s 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:53924/0 (socket says 172.21.15.105:53924) 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:56286/0 (socket says 172.21.15.105:56286) 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 1 -- 172.21.15.105:0/3716156056 learned_addr learned my addr 172.21.15.105:0/3716156056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): get_auth_request con 0x7f114c0a73d0 auth_method 0 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): get_auth_request con 0x7f114c0a4910 auth_method 0 2024-09-17T16:41:49.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient(hunting): handle_auth_done global_id 54710 payload 274 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 -- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 msgr2=0x7f114c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:49.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 -- 172.21.15.105:0/3716156056 >> v1:172.21.15.160:6789/0 conn(0x7f114c0a68e0 legacy=0x7f114c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 monclient: found mon.noname-e 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f114bfff640 1 -- 172.21.15.105:0/3716156056 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c0a3690 con 0x7f114c0a4910 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.144+0000 7f11510ac640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114bfff640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11400097f0 tx=0x7f114001c370 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c02a302522d82f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 1 -- 172.21.15.105:0/3716156056 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f11400b0070 con 0x7f114c0a4910 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: dump: 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.168 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 1 -- 172.21.15.105:0/3716156056 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11400aa810 con 0x7f114c0a4910 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 5 monclient: authenticate success, global_id 54710 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 4 set_mon_vals no callback set 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 1 -- 172.21.15.105:0/3716156056 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f11400a9300 con 0x7f114c0a4910 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f114b7fe640 10 monclient: dump: 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.170 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.171 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: shutdown 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 msgr2=0x7f114c13eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11400097f0 tx=0x7f114001c370 comp rx=0 tx=0).stop 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- 172.21.15.105:0/3716156056 shutdown_connections 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a73d0 0x7f114c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- 172.21.15.105:0/3716156056 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f114c0a4910 0x7f114c13eaf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- 172.21.15.105:0/3716156056 >> 172.21.15.105:0/3716156056 conn(0x7f114c068ba0 msgr2=0x7f114c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- 172.21.15.105:0/3716156056 shutdown_connections 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- 172.21.15.105:0/3716156056 wait complete. 2024-09-17T16:41:49.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: monmap: 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.173 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 librados: starting msgr at 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 librados: starting objecter 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 Processor -- start 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- start start 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 librados: setting wanted keys 2024-09-17T16:41:49.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 librados: calling monclient init 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: init 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient: _add_conns ranks=[2,4,3,7,0,5,6,1,8] 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-b con 0x7f114c0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a68e0 0x7f114c1d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-c con 0x7f114c0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 0x7f114c1d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): picked mon.noname-g con 0x7f114c0a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f114c143f20 con 0x7f114c0a4910 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f114c143da0 con 0x7f114c0a73d0 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f114c1440a0 con 0x7f114c0a68e0 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f1152b0f640 10 monclient(hunting): authenticate will time out at 1792.419865s 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f11508ab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.148+0000 7f11508ab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47800/0 (socket says 172.21.15.105:47800) 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a68e0 0x7f114c1d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 -- 172.21.15.105:0/4243559466 learned_addr learned my addr 172.21.15.105:0/4243559466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): get_auth_request con 0x7f114c0a4910 auth_method 0 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): get_auth_request con 0x7f114c0a68e0 auth_method 0 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11510ac640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 0x7f114c1d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient(hunting): handle_auth_done global_id 84206 payload 824 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 -- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a68e0 msgr2=0x7f114c1d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a68e0 0x7f114c1d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 -- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 msgr2=0x7f114c1d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 0x7f114c1d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 monclient: found mon.noname-b 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 -- 172.21.15.105:0/4243559466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c12a1f0 con 0x7f114c0a4910 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11510ac640 10 monclient: get_auth_request con 0x7f114c0a73d0 auth_method 0 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11510ac640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 0x7f114c1d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11508ab640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c4b0 tx=0x7f113c00c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 client.?.objecter ms_handle_connect 0x7f114c0a4910 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f113c012630 con 0x7f114c0a4910 2024-09-17T16:41:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: dump: 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113c012810 con 0x7f114c0a4910 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 5 monclient: authenticate success, global_id 84206 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _renew_subs 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c0a2580 con 0x7f114c0a4910 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f113c003600 con 0x7f114c0a4910 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114affd640 4 set_mon_vals no callback set 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 monclient: dump: 2024-09-17T16:41:49.182 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.183 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 client.84206.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _renew_subs 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c127e90 con 0x7f114c0a4910 2024-09-17T16:41:49.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 1 librados: init done 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c0a6cc0 con 0x7f114c0a4910 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f113c0050d0 con 0x7f114c0a4910 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 0x7f1138040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(542..542 src has 1..542) ==== 6769+0+0 (secure 0 0 0) 0x7f113c04f920 con 0x7f114c0a4910 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 client.84206.objecter ms_dispatch 0x7f114c000ba0 osd_map(542..542 src has 1..542) 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 3 client.84206.objecter handle_osd_map got epochs [542,542] > 0 2024-09-17T16:41:49.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 3 client.84206.objecter handle_osd_map decoding full epoch 542 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 20 client.84206.objecter dump_active .. 0 homeless 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 0x7f1138040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 10 monclient: get_auth_request con 0x7f113803e580 auth_method 0 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f11497fa640 10 client.84206.objecter ms_handle_connect 0x7f113803e580 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.152+0000 7f114bfff640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 0x7f1138040a40 secure :-1 s=READY pgs=3601 cs=0 l=1 rev1=1 crypto rx=0x7f11400ad000 tx=0x7f11400a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.156+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f113c020060 con 0x7f114c0a4910 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.156+0000 7f11497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.156+0000 7f11497fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:49.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f1152b0f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:41:49.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f1152b0f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:49.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f1152b0f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f114c0746d0 con 0x7f114c0a4910 2024-09-17T16:41:49.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f11497fa640 1 -- 172.21.15.105:0/4243559466 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3212 (secure 0 0 0) 0x7f113c003cc0 con 0x7f114c0a4910 2024-09-17T16:41:49.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f11497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:41:49.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f11497fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:49.442 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":84,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:41:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.432+0000 7f1152b0f640 10 librados: watch_flush enter 2024-09-17T16:41:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 10 librados: watch_flush exit 2024-09-17T16:41:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 20 client.84206.objecter shutdown clearing up homeless session... 2024-09-17T16:41:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 10 client.84206.objecter successfully canceled tick 2024-09-17T16:41:49.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 msgr2=0x7f1138040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 0x7f1138040a40 secure :-1 s=READY pgs=3601 cs=0 l=1 rev1=1 crypto rx=0x7f11400ad000 tx=0x7f11400a6000 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 10 monclient: shutdown 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 msgr2=0x7f114c1d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c4b0 tx=0x7f113c00c970 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 shutdown_connections 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f113803e580 0x7f1138040a40 unknown :-1 s=CLOSED pgs=3601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f114c0a73d0 0x7f114c1d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f114c0a68e0 0x7f114c1d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 --2- 172.21.15.105:0/4243559466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f114c0a4910 0x7f114c1d0fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 >> 172.21.15.105:0/4243559466 conn(0x7f114c068ba0 msgr2=0x7f114c09fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 shutdown_connections 2024-09-17T16:41:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 -- 172.21.15.105:0/4243559466 wait complete. 2024-09-17T16:41:49.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.436+0000 7f1152b0f640 1 librados: shutdown 2024-09-17T16:41:49.458 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:41:49.458 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:41:49.459 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:41:49.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:41:49.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: get_monmap_and_config 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: monmap: 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:49.576031+0000 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:49.576031+0000 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:49.580 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:49.581 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 Processor -- start 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 -- start start 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: init 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient: _add_conns ranks=[1,8,9,10,7,5,4,0,3,2,6] 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f61900a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): picked mon.noname-e con 0x7f61900a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 0x7f6190141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): picked mon.noname-a con 0x7f61900a73d0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619006ebc0 con 0x7f61900a68e0 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f61900a3fc0 con 0x7f61900a73d0 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f61900a63c0 con 0x7f61900a4910 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f6195380640 10 monclient(hunting): authenticate will time out at 1792.841713s 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:56294/0 (socket says 172.21.15.105:56294) 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 -- 172.21.15.105:0/1032374020 learned_addr learned my addr 172.21.15.105:0/1032374020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): get_auth_request con 0x7f61900a4910 auth_method 0 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 1 --2- 172.21.15.105:0/1032374020 >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 0x7f6190141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): get_auth_request con 0x7f61900a73d0 auth_method 0 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient(hunting): handle_auth_done global_id 54719 payload 274 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 -- 172.21.15.105:0/1032374020 >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 msgr2=0x7f6190141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 --2- 172.21.15.105:0/1032374020 >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 0x7f6190141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 -- 172.21.15.105:0/1032374020 >> v1:172.21.15.160:6789/0 conn(0x7f61900a68e0 legacy=0x7f61900a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 monclient: found mon.noname-e 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618e7fc640 1 -- 172.21.15.105:0/1032374020 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61900a3690 con 0x7f61900a4910 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.572+0000 7f618f7fe640 1 --2- 172.21.15.105:0/1032374020 >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 0x7f6190141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- 172.21.15.105:0/1032374020 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6184009870 tx=0x7f618401c3f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cdf891b9bc6f6fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 1 -- 172.21.15.105:0/1032374020 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61840b2070 con 0x7f61900a4910 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: dump: 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.594 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.595 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 1 -- 172.21.15.105:0/1032374020 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61840ab7f0 con 0x7f61900a4910 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 5 monclient: authenticate success, global_id 54719 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 1 -- 172.21.15.105:0/1032374020 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61840abe50 con 0x7f61900a4910 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 4 set_mon_vals no callback set 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618dffb640 10 monclient: dump: 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.597 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: shutdown 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- 172.21.15.105:0/1032374020 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 msgr2=0x7f619013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- 172.21.15.105:0/1032374020 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6184009870 tx=0x7f618401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- 172.21.15.105:0/1032374020 shutdown_connections 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- 172.21.15.105:0/1032374020 >> v2:172.21.15.105:3300/0 conn(0x7f61900a73d0 0x7f6190141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- 172.21.15.105:0/1032374020 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f61900a4910 0x7f619013eaf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- 172.21.15.105:0/1032374020 >> 172.21.15.105:0/1032374020 conn(0x7f6190068ba0 msgr2=0x7f6190068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- 172.21.15.105:0/1032374020 shutdown_connections 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- 172.21.15.105:0/1032374020 wait complete. 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: monmap: 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.599 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 librados: starting msgr at 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 librados: starting objecter 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 Processor -- start 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- start start 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 librados: setting wanted keys 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 librados: calling monclient init 2024-09-17T16:41:49.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: init 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient: _add_conns ranks=[7,2,5,8,0,3,1,6,4] 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61900a4910 0x7f6190128a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): picked mon.noname-i con 0x7f61900a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 0x7f61901d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): picked mon.noname-b con 0x7f61900a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61900a73d0 0x7f6190126510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): picked mon.noname-h con 0x7f61900a73d0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6190143f20 con 0x7f61900a68e0 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6190143da0 con 0x7f61900a73d0 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f61901440a0 con 0x7f61900a4910 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f6195380640 10 monclient(hunting): authenticate will time out at 1792.847104s 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 0x7f61901d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 0x7f61901d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47806/0 (socket says 172.21.15.105:47806) 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 -- 172.21.15.105:0/2544794955 learned_addr learned my addr 172.21.15.105:0/2544794955 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): get_auth_request con 0x7f61900a68e0 auth_method 0 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618effd640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61900a4910 0x7f6190128a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient(hunting): handle_auth_done global_id 84215 payload 824 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618f7fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61900a73d0 0x7f6190126510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61900a4910 msgr2=0x7f6190128a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f61900a4910 0x7f6190128a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61900a73d0 msgr2=0x7f6190126510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61900a73d0 0x7f6190126510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 monclient: found mon.noname-b 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190129f40 con 0x7f61900a68e0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f618e7fc640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 0x7f61901d1620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61840b0040 tx=0x7f6184019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f61777fe640 10 client.?.objecter ms_handle_connect 0x7f61900a68e0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f61777fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.576+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61840bd070 con 0x7f61900a68e0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:49.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: dump: 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.605 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61840abe10 con 0x7f61900a68e0 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 5 monclient: authenticate success, global_id 84215 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 monclient: _renew_subs 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619012a220 con 0x7f61900a68e0 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61840b2070 con 0x7f61900a68e0 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: dump: 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.607 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618d7fa640 4 set_mon_vals no callback set 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 client.84215.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 monclient: _renew_subs 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f619006e090 con 0x7f61900a68e0 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 1 librados: init done 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 0x7f6190126a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6190051e80 con 0x7f61901d1b60 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 0x7f6190126a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient: get_auth_request con 0x7f61901d1b60 auth_method 0 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6184004250 con 0x7f61900a68e0 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f617c03e550 0x7f617c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 10 monclient(hunting): handle_auth_done global_id 36288 payload 824 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61900a2580 con 0x7f61901d1b60 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(543..543 src has 1..543) ==== 6713+0+0 (secure 0 0 0) 0x7f61840e9970 con 0x7f61900a68e0 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 client.84215.objecter ms_dispatch 0x7f6190000ba0 osd_map(543..543 src has 1..543) 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 3 client.84215.objecter handle_osd_map got epochs [543,543] > 0 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 3 client.84215.objecter handle_osd_map decoding full epoch 543 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618effd640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 0x7f6190126a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f618000d740 tx=0x7f618000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 20 client.84215.objecter dump_active .. 0 homeless 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 client.84215.objecter ms_handle_connect 0x7f61901d1b60 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 client.84215.objecter resend_mon_ops 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f618000de50 con 0x7f61901d1b60 2024-09-17T16:41:49.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: dump: 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.611 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618f7fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f617c03e550 0x7f617c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618f7fe640 10 monclient: get_auth_request con 0x7f617c03e550 auth_method 0 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6180017070 con 0x7f61901d1b60 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 msgr2=0x7f6190126a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 0x7f6190126a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f618000d740 tx=0x7f618000dc00 comp rx=0 tx=0).stop 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 msgr2=0x7f6190126a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61901d1b60 0x7f6190126a50 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f618000d740 tx=0x7f618000dc00 comp rx=0 tx=0).stop 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f61777fe640 10 client.84215.objecter ms_handle_connect 0x7f617c03e550 2024-09-17T16:41:49.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.580+0000 7f618f7fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f617c03e550 0x7f617c040a10 secure :-1 s=READY pgs=3602 cs=0 l=1 rev1=1 crypto rx=0x7f6188009970 tx=0x7f6188008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 0x7f6190122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61901273a0 con 0x7f6190126f90 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 0x7f6190122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient: get_auth_request con 0x7f6190126f90 auth_method 0 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 10 monclient(hunting): handle_auth_done global_id 36297 payload 824 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 1 -- 172.21.15.105:0/2544794955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f61901293b0 con 0x7f6190126f90 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 client.84215.objecter ms_handle_connect 0x7f6190126f90 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 client.84215.objecter resend_mon_ops 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618effd640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 0x7f6190122530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6180001340 tx=0x7f6180001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61800057c0 con 0x7f6190126f90 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 monclient: dump: 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.617 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.618 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f6180005350 con 0x7f6190126f90 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 msgr2=0x7f6190122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 0x7f6190122530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6180001340 tx=0x7f6180001260 comp rx=0 tx=0).stop 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 msgr2=0x7f6190122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f61777fe640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6190126f90 0x7f6190122530 secure :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6180001340 tx=0x7f6180001260 comp rx=0 tx=0).stop 2024-09-17T16:41:49.619 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 700391, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:49.620 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.621 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:49.623 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.624 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.625 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.627 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.630 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:49.632 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:49.637 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.639 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.640 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.641 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:49.642 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.645 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:49.645 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 librados: watch_flush enter 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 librados: watch_flush exit 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 20 client.84215.objecter shutdown clearing up homeless session... 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 client.84215.objecter successfully canceled tick 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f617c03e550 msgr2=0x7f617c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f617c03e550 0x7f617c040a10 secure :-1 s=READY pgs=3602 cs=0 l=1 rev1=1 crypto rx=0x7f6188009970 tx=0x7f6188008040 comp rx=0 tx=0).stop 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 10 monclient: shutdown 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 msgr2=0x7f61901d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 --2- 172.21.15.105:0/2544794955 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f61900a68e0 0x7f61901d1620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61840b0040 tx=0x7f6184019190 comp rx=0 tx=0).stop 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618f7fe640 1 -- 172.21.15.105:0/2544794955 reap_dead start 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f618f7fe640 1 -- 172.21.15.105:0/2544794955 reap_dead start 2024-09-17T16:41:49.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 shutdown_connections 2024-09-17T16:41:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 >> 172.21.15.105:0/2544794955 conn(0x7f6190068ba0 msgr2=0x7f61900a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 shutdown_connections 2024-09-17T16:41:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 -- 172.21.15.105:0/2544794955 wait complete. 2024-09-17T16:41:49.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.588+0000 7f6195380640 1 librados: shutdown 2024-09-17T16:41:49.648 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:49.648 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:41:49.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:41:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: get_monmap_and_config 2024-09-17T16:41:49.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: monmap: 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:49.759955+0000 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:49.759955+0000 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.764 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:49.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 Processor -- start 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 -- start start 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: init 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient: _add_conns ranks=[4,1,0,6,10,5,3,8,9,7,2] 2024-09-17T16:41:49.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-c con 0x7f2aac0a7060 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2aac0a5da0 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2aac0a61a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac0a7440 con 0x7f2aac0a61a0 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac13a820 con 0x7f2aac0a5da0 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2aac06d2b0 con 0x7f2aac0a7060 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab2e87640 10 monclient(hunting): authenticate will time out at 1793.025814s 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab1424640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2aac0a61a0 0x7f2aac13a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55174/0 (socket says 172.21.15.105:55174) 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:53936/0 (socket says 172.21.15.105:53936) 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab1424640 1 -- 172.21.15.105:0/2825318838 learned_addr learned my addr 172.21.15.105:0/2825318838 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:49.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): get_auth_request con 0x7f2aac0a7060 auth_method 0 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_done global_id 74497 payload 274 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 -- 172.21.15.105:0/2825318838 >> v1:172.21.15.160:6789/0 conn(0x7f2aac0a5da0 legacy=0x7f2aac139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 -- 172.21.15.105:0/2825318838 >> v1:172.21.15.105:6789/0 conn(0x7f2aac0a61a0 legacy=0x7f2aac13a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 monclient: found mon.noname-c 2024-09-17T16:41:49.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2aab7fe640 1 -- 172.21.15.105:0/2825318838 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334891252 0 0) 0x7f2aac0a7440 con 0x7f2aac0a61a0 2024-09-17T16:41:49.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:49.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2ab0c23640 1 -- 172.21.15.105:0/2825318838 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac0a0e70 con 0x7f2aac0a7060 2024-09-17T16:41:49.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.756+0000 7f2aab7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:41:49.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- 172.21.15.105:0/2825318838 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00097f0 tx=0x7f2aa001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8d3c262b1aed65d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 1 -- 172.21.15.105:0/2825318838 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2aa00b0070 con 0x7f2aac0a7060 2024-09-17T16:41:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:49.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: dump: 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.777 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 1 -- 172.21.15.105:0/2825318838 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa00aa810 con 0x7f2aac0a7060 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 5 monclient: authenticate success, global_id 74497 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 1 -- 172.21.15.105:0/2825318838 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2aa00aae30 con 0x7f2aac0a7060 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aab7fe640 10 monclient: dump: 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.779 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 4 set_mon_vals no callback set 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: shutdown 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/2825318838 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 msgr2=0x7f2aac0a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/2825318838 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00097f0 tx=0x7f2aa001c3a0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/2825318838 shutdown_connections 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/2825318838 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a7060 0x7f2aac0a5860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/2825318838 >> 172.21.15.105:0/2825318838 conn(0x7f2aac0aa450 msgr2=0x7f2aac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/2825318838 shutdown_connections 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/2825318838 wait complete. 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: monmap: 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.781 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 librados: starting msgr at 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 librados: starting objecter 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 Processor -- start 2024-09-17T16:41:49.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- start start 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 librados: setting wanted keys 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 librados: calling monclient init 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: init 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _add_conns ranks=[4,1,3,6,8,5,0,2,7] 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-c con 0x7f2aac0a5da0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aac0a61a0 0x7f2aac13d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-f con 0x7f2aac0a61a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aac0a7060 0x7f2aac13dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): picked mon.noname-g con 0x7f2aac0a7060 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:49.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac06db60 con 0x7f2aac0a61a0 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac0a65f0 con 0x7f2aac0a7060 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2aac145490 con 0x7f2aac0a5da0 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient(hunting): authenticate will time out at 1793.030658s 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:53948/0 (socket says 172.21.15.105:53948) 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 -- 172.21.15.105:0/3439086899 learned_addr learned my addr 172.21.15.105:0/3439086899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): get_auth_request con 0x7f2aac0a5da0 auth_method 0 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aac0a61a0 0x7f2aac13d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient(hunting): handle_auth_done global_id 74506 payload 824 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aac0a7060 msgr2=0x7f2aac13dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aac0a7060 0x7f2aac13dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aac0a61a0 msgr2=0x7f2aac13d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aac0a61a0 0x7f2aac13d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 monclient: found mon.noname-c 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac1401d0 con 0x7f2aac0a5da0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aabfff640 10 monclient: get_auth_request con 0x7f2aac0a61a0 auth_method 0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aac0a61a0 0x7f2aac13d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab0c23640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00ae040 tx=0x7f2aa0019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 client.?.objecter ms_handle_connect 0x7f2aac0a5da0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2aa00b0040 con 0x7f2aac0a5da0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: dump: 2024-09-17T16:41:49.786 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.787 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa00b9070 con 0x7f2aac0a5da0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 5 monclient: authenticate success, global_id 74506 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _renew_subs 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac1407a0 con 0x7f2aac0a5da0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2aa00044b0 con 0x7f2aac0a5da0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aaaffd640 4 set_mon_vals no callback set 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2aa97fa640 10 monclient: dump: 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.789 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 client.74506.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _renew_subs 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac13ece0 con 0x7f2aac0a5da0 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.760+0000 7f2ab2e87640 1 librados: init done 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab2e87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 0x7f2aac09e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac051e80 con 0x7f2aac09e1a0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 0x7f2aac09e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient: get_auth_request con 0x7f2aac09e1a0 auth_method 0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2aa0004040 con 0x7f2aac0a5da0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 10 monclient(hunting): handle_auth_done global_id 84224 payload 824 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2aac0a5860 con 0x7f2aac09e1a0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a9803e540 0x7f2a98040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(543..543 src has 1..543) ==== 6713+0+0 (secure 0 0 0) 0x7f2aa00e9f00 con 0x7f2aac0a5da0 2024-09-17T16:41:49.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 client.74506.objecter ms_dispatch 0x7f2aac000ba0 osd_map(543..543 src has 1..543) 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 3 client.74506.objecter handle_osd_map got epochs [543,543] > 0 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 3 client.74506.objecter handle_osd_map decoding full epoch 543 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 0x7f2aac09e580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2aac13e950 tx=0x7f2a9c00d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 20 client.74506.objecter dump_active .. 0 homeless 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 client.74506.objecter ms_handle_connect 0x7f2aac09e1a0 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 client.74506.objecter resend_mon_ops 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a9c0052e0 con 0x7f2aac09e1a0 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 monclient: dump: 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.792 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.793 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab1424640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a9803e540 0x7f2a98040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab1424640 10 monclient: get_auth_request con 0x7f2a9803e540 auth_method 0 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2a9c0052e0 con 0x7f2aac09e1a0 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 msgr2=0x7f2aac09e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 0x7f2aac09e580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2aac13e950 tx=0x7f2a9c00d950 comp rx=0 tx=0).stop 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 msgr2=0x7f2aac09e580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09e1a0 0x7f2aac09e580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2aa97fa640 10 client.74506.objecter ms_handle_connect 0x7f2a9803e540 2024-09-17T16:41:49.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.764+0000 7f2ab1424640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a9803e540 0x7f2a98040a00 secure :-1 s=READY pgs=3603 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4000f90 tx=0x7f2aa4007c60 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2ab2e87640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:49.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 0x7f2aac1db3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2ab2e87640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac0a4320 con 0x7f2aac09eac0 2024-09-17T16:41:49.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 0x7f2aac1db3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient: get_auth_request con 0x7f2aac09eac0 auth_method 0 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 10 monclient(hunting): handle_auth_done global_id 84233 payload 824 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 1 -- 172.21.15.105:0/3439086899 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2aac0a9b10 con 0x7f2aac09eac0 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 10 client.74506.objecter ms_handle_connect 0x7f2aac09eac0 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 10 client.74506.objecter resend_mon_ops 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aabfff640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 0x7f2aac1db3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c014000 tx=0x7f2a9c001090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a9c013470 con 0x7f2aac09eac0 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.768+0000 7f2aa97fa640 10 monclient: dump: 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.798 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f2a9c003b60 con 0x7f2aac09eac0 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 msgr2=0x7f2aac1db3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 0x7f2aac1db3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c014000 tx=0x7f2a9c001090 comp rx=0 tx=0).stop 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 msgr2=0x7f2aac1db3e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2aa97fa640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aac09eac0 0x7f2aac1db3e0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c014000 tx=0x7f2a9c001090 comp rx=0 tx=0).stop 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 17003, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:49.800 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.801 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.802 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:49.803 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:49.804 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.806 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:49.807 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:49.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:49.812 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:49.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:49.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:49.821 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:49.822 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:49.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 10 librados: watch_flush enter 2024-09-17T16:41:49.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 10 librados: watch_flush exit 2024-09-17T16:41:49.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 20 client.74506.objecter shutdown clearing up homeless session... 2024-09-17T16:41:49.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 10 client.74506.objecter successfully canceled tick 2024-09-17T16:41:49.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a9803e540 msgr2=0x7f2a98040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2a9803e540 0x7f2a98040a00 secure :-1 s=READY pgs=3603 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4000f90 tx=0x7f2aa4007c60 comp rx=0 tx=0).stop 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 10 monclient: shutdown 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 msgr2=0x7f2aac13d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00ae040 tx=0x7f2aa0019140 comp rx=0 tx=0).stop 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab1424640 1 -- 172.21.15.105:0/3439086899 reap_dead start 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 shutdown_connections 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 --2- 172.21.15.105:0/3439086899 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2aac0a5da0 0x7f2aac13d1f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 >> 172.21.15.105:0/3439086899 conn(0x7f2aac0aa450 msgr2=0x7f2aac0a0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 shutdown_connections 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 -- 172.21.15.105:0/3439086899 wait complete. 2024-09-17T16:41:49.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.772+0000 7f2ab2e87640 1 librados: shutdown 2024-09-17T16:41:49.826 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:49.826 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:41:49.826 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.928+0000 7f615331d640 10 monclient: get_monmap_and_config 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.928+0000 7f615331d640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.928+0000 7f615331d640 10 monclient: monmap: 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:49.935410+0000 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:49.935410+0000 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.939 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:49.940 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 Processor -- start 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 -- start start 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient: init 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient: _add_conns ranks=[6,1,9,2,5,7,4,3,0,8,10] 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): picked mon.noname-d con 0x7f614c0a2620 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:49.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f614c0a2f40 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): picked mon.noname-a con 0x7f614c137b10 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c06f2e0 con 0x7f614c0a2f40 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f614c06e580 con 0x7f614c137b10 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f614c0a15e0 con 0x7f614c0a2620 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient(hunting): authenticate will time out at 1793.200899s 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46176/0 (socket says 172.21.15.105:46176) 2024-09-17T16:41:49.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:38550/0 (socket says 172.21.15.105:38550) 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 1 -- 172.21.15.105:0/4195554988 learned_addr learned my addr 172.21.15.105:0/4195554988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): get_auth_request con 0x7f614c137b10 auth_method 0 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): get_auth_request con 0x7f614c0a2620 auth_method 0 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient(hunting): handle_auth_done global_id 24855 payload 274 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 -- 172.21.15.105:0/4195554988 >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 msgr2=0x7f614c13f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 --2- 172.21.15.105:0/4195554988 >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 -- 172.21.15.105:0/4195554988 >> v1:172.21.15.160:6789/0 conn(0x7f614c0a2f40 legacy=0x7f614c137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 monclient: found mon.noname-d 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 -- 172.21.15.105:0/4195554988 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0a3c30 con 0x7f614c0a2620 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61518ba640 1 --2- 172.21.15.105:0/4195554988 >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f61510b9640 1 --2- 172.21.15.105:0/4195554988 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f61480097c0 tx=0x7f614801c360 comp rx=0 tx=0).ready entity=mon.6 client_cookie=369dc9d687e5d6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 1 -- 172.21.15.105:0/4195554988 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61480b0070 con 0x7f614c0a2620 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: dump: 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.948 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 5 monclient: authenticate success, global_id 24855 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 1 -- 172.21.15.105:0/4195554988 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61480aa810 con 0x7f614c0a2620 2024-09-17T16:41:49.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f615331d640 4 set_mon_vals no callback set 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 1 -- 172.21.15.105:0/4195554988 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61480a9300 con 0x7f614c0a2620 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:49.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.932+0000 7f6143fff640 10 monclient: dump: 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.951 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: shutdown 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:49.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/4195554988 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 msgr2=0x7f614c0a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- 172.21.15.105:0/4195554988 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f61480097c0 tx=0x7f614801c360 comp rx=0 tx=0).stop 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/4195554988 shutdown_connections 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- 172.21.15.105:0/4195554988 >> v2:172.21.15.105:3300/0 conn(0x7f614c137b10 0x7f614c13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- 172.21.15.105:0/4195554988 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c0a2620 0x7f614c0a2a00 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/4195554988 >> 172.21.15.105:0/4195554988 conn(0x7f614c068ba0 msgr2=0x7f614c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:49.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/4195554988 shutdown_connections 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/4195554988 wait complete. 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: build_initial_monmap 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: monmap: 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:49.955 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:49.956 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:49.957 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 librados: starting msgr at 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 librados: starting objecter 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 Processor -- start 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- start start 2024-09-17T16:41:49.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 librados: setting wanted keys 2024-09-17T16:41:49.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 librados: calling monclient init 2024-09-17T16:41:49.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: init 2024-09-17T16:41:49.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:49.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _add_conns ranks=[1,5,6,7,2,3,0,4,8] 2024-09-17T16:41:49.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f614c0a2620 0x7f614c132d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): picked mon.noname-f con 0x7f614c0a2620 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:49.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f614c0a2f40 0x7f614c133290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): picked mon.noname-h con 0x7f614c0a2f40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:49.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 0x7f614c09f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): picked mon.noname-d con 0x7f614c137b10 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0a1f50 con 0x7f614c0a2620 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f614c0a1dd0 con 0x7f614c0a2f40 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f614c142640 con 0x7f614c137b10 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): authenticate will time out at 1793.205688s 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 0x7f614c09f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 0x7f614c09f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:38552/0 (socket says 172.21.15.105:38552) 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 learned_addr learned my addr 172.21.15.105:0/441268189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): get_auth_request con 0x7f614c137b10 auth_method 0 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient(hunting): handle_auth_done global_id 24864 payload 824 2024-09-17T16:41:49.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f614c0a2620 0x7f614c132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f614c0a2f40 msgr2=0x7f614c133290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f614c0a2f40 0x7f614c133290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f614c0a2620 msgr2=0x7f614c132d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f614c0a2620 0x7f614c132d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient: _finish_hunting 0 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 monclient: found mon.noname-d 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0a09d0 con 0x7f614c137b10 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61518ba640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 0x7f614c09f190 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f614400efc0 tx=0x7f614400c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 client.?.objecter ms_handle_connect 0x7f614c137b10 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f614400cbe0 con 0x7f614c137b10 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: dump: 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.978 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: _finish_auth 0 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f614400cd80 con 0x7f614c137b10 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6144014d60 con 0x7f614c137b10 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: dump: 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.980 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61437fe640 4 set_mon_vals no callback set 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 5 monclient: authenticate success, global_id 24864 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _renew_subs 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c11fc00 con 0x7f614c137b10 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 client.24864.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _renew_subs 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c11f920 con 0x7f614c137b10 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 librados: init done 2024-09-17T16:41:49.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 0x7f614c09fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f614c051e80 con 0x7f614c09f6d0 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 0x7f614c09fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient: get_auth_request con 0x7f614c09f6d0 auth_method 0 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 10 monclient(hunting): handle_auth_done global_id 74515 payload 824 2024-09-17T16:41:49.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f614c11f0b0 con 0x7f614c09f6d0 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f61510b9640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 0x7f614c09fab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f614c0698c0 tx=0x7f6148019100 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 client.24864.objecter ms_handle_connect 0x7f614c09f6d0 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 client.24864.objecter resend_mon_ops 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61480b9070 con 0x7f614c09f6d0 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 10 monclient: dump: 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.983 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.936+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f614401b070 con 0x7f614c137b10 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f613003e820 0x7f6130040ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(543..543 src has 1..543) ==== 6713+0+0 (secure 0 0 0) 0x7f6144019020 con 0x7f614c137b10 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 10 client.24864.objecter ms_dispatch 0x7f614c000ba0 osd_map(543..543 src has 1..543) 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 3 client.24864.objecter handle_osd_map got epochs [543,543] > 0 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 3 client.24864.objecter handle_osd_map decoding full epoch 543 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 20 client.24864.objecter dump_active .. 0 homeless 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6148003620 con 0x7f614c09f6d0 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 msgr2=0x7f614c09fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 0x7f614c09fab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f614c0698c0 tx=0x7f6148019100 comp rx=0 tx=0).stop 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 msgr2=0x7f614c09fab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09f6d0 0x7f614c09fab0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f614c0698c0 tx=0x7f6148019100 comp rx=0 tx=0).stop 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f61508b8640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f613003e820 0x7f6130040ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f61508b8640 10 monclient: get_auth_request con 0x7f613003e820 auth_method 0 2024-09-17T16:41:49.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f6141ffb640 10 client.24864.objecter ms_handle_connect 0x7f613003e820 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.940+0000 7f61508b8640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f613003e820 0x7f6130040ce0 secure :-1 s=READY pgs=3604 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f70 tx=0x7f613c0079c0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 0x7f614c070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f614c09e350 con 0x7f614c09fff0 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 0x7f614c070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient: get_auth_request con 0x7f614c09fff0 auth_method 0 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:49.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 10 monclient(hunting): handle_auth_done global_id 74524 payload 824 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 1 -- 172.21.15.105:0/441268189 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f614c0a4280 con 0x7f614c09fff0 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61510b9640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 0x7f614c070ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61480097c0 tx=0x7f61480a8000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 client.24864.objecter ms_handle_connect 0x7f614c09fff0 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 client.24864.objecter resend_mon_ops 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61480b0040 con 0x7f614c09fff0 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 monclient: dump: 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.987 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:49.988 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f61480b5480 con 0x7f614c09fff0 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 msgr2=0x7f614c070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 0x7f614c070ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61480097c0 tx=0x7f61480a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 msgr2=0x7f614c070ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f6141ffb640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f614c09fff0 0x7f614c070ad0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61480097c0 tx=0x7f61480a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:49.989 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 149406, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:49.990 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.991 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:49.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.997 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.001 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.004 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.004 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.004 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.006 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.008 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:50.010 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.011 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.013 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.014 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.015 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.016 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:50.017 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 librados: watch_flush enter 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 librados: watch_flush exit 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 20 client.24864.objecter shutdown clearing up homeless session... 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 client.24864.objecter successfully canceled tick 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f613003e820 msgr2=0x7f6130040ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f613003e820 0x7f6130040ce0 secure :-1 s=READY pgs=3604 cs=0 l=1 rev1=1 crypto rx=0x7f613c000f70 tx=0x7f613c0079c0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 10 monclient: shutdown 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 msgr2=0x7f614c09f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 --2- 172.21.15.105:0/441268189 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f614c137b10 0x7f614c09f190 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f614400efc0 tx=0x7f614400c490 comp rx=0 tx=0).stop 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 reap_dead start 2024-09-17T16:41:50.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f61518ba640 1 -- 172.21.15.105:0/441268189 reap_dead start 2024-09-17T16:41:50.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 shutdown_connections 2024-09-17T16:41:50.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 >> 172.21.15.105:0/441268189 conn(0x7f614c068ba0 msgr2=0x7f614c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 shutdown_connections 2024-09-17T16:41:50.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 -- 172.21.15.105:0/441268189 wait complete. 2024-09-17T16:41:50.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:49.944+0000 7f615331d640 1 librados: shutdown 2024-09-17T16:41:50.020 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:50.020 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:41:50.020 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:41:50.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.132+0000 7f76ceb21640 10 monclient: get_monmap_and_config 2024-09-17T16:41:50.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.132+0000 7f76ceb21640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.132+0000 7f76ceb21640 10 monclient: monmap: 2024-09-17T16:41:50.143 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:50.139623+0000 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:50.139623+0000 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.144 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.145 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.145 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:50.145 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:50.145 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 Processor -- start 2024-09-17T16:41:50.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 -- start start 2024-09-17T16:41:50.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient: init 2024-09-17T16:41:50.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient: _add_conns ranks=[5,3,8,2,7,9,1,6,0,10,4] 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c80a68e0 0x7f76c80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-h con 0x7f76c80a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76c80a4910 0x7f76c80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-g con 0x7f76c80a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-e con 0x7f76c81396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80a3fc0 con 0x7f76c80a4910 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f76c80a63c0 con 0x7f76c80a68e0 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f76c80a7740 con 0x7f76c81396a0 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76ceb21640 10 monclient(hunting): authenticate will time out at 1793.405294s 2024-09-17T16:41:50.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:56304/0 (socket says 172.21.15.105:56304) 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 -- 172.21.15.105:0/3047852617 learned_addr learned my addr 172.21.15.105:0/3047852617 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): get_auth_request con 0x7f76c81396a0 auth_method 0 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76c7fff640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76c80a4910 0x7f76c80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cc8bd640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c80a68e0 0x7f76c80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76c7fff640 10 monclient(hunting): get_auth_request con 0x7f76c80a4910 auth_method 0 2024-09-17T16:41:50.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cc8bd640 10 monclient(hunting): get_auth_request con 0x7f76c80a68e0 auth_method 0 2024-09-17T16:41:50.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cc8bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cc8bd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cc8bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient(hunting): handle_auth_done global_id 54728 payload 274 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 -- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c80a68e0 msgr2=0x7f76c80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c80a68e0 0x7f76c80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 -- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76c80a4910 msgr2=0x7f76c80a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76c80a4910 0x7f76c80a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 monclient: found mon.noname-e 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 -- 172.21.15.105:0/3047852617 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c806ec10 con 0x7f76c81396a0 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.136+0000 7f76cd0be640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76c00098d0 tx=0x7f76c001cea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c8c6371d77836576 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 1 -- 172.21.15.105:0/3047852617 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76c00b0070 con 0x7f76c81396a0 2024-09-17T16:41:50.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: dump: 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.165 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.166 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 1 -- 172.21.15.105:0/3047852617 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c00aa810 con 0x7f76c81396a0 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 5 monclient: authenticate success, global_id 54728 2024-09-17T16:41:50.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 1 -- 172.21.15.105:0/3047852617 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76c00a9300 con 0x7f76c81396a0 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 4 set_mon_vals no callback set 2024-09-17T16:41:50.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c77fe640 10 monclient: dump: 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.169 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.170 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: shutdown 2024-09-17T16:41:50.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/3047852617 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 msgr2=0x7f76c8139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f76c00098d0 tx=0x7f76c001cea0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/3047852617 shutdown_connections 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f76c81396a0 0x7f76c8139ae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76c80a4910 0x7f76c80a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- 172.21.15.105:0/3047852617 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c80a68e0 0x7f76c80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/3047852617 >> 172.21.15.105:0/3047852617 conn(0x7f76c8068ba0 msgr2=0x7f76c8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/3047852617 shutdown_connections 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/3047852617 wait complete. 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: monmap: 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.172 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.173 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 librados: starting msgr at 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 librados: starting objecter 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 Processor -- start 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- start start 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 librados: setting wanted keys 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 librados: calling monclient init 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: init 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _add_conns ranks=[6,0,5,2,3,7,1,8,4] 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 0x7f76c81d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-d con 0x7f76c80a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 0x7f76c81d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-a con 0x7f76c80a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c81396a0 0x7f76c813d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): picked mon.noname-h con 0x7f76c81396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76c8143320 con 0x7f76c80a68e0 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f76c806e630 con 0x7f76c81396a0 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f76c81434a0 con 0x7f76c80a4910 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient(hunting): authenticate will time out at 1793.410169s 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 0x7f76c81d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 0x7f76c81d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 0x7f76c81d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:38560/0 (socket says 172.21.15.105:38560) 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 0x7f76c81d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46184/0 (socket says 172.21.15.105:46184) 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 -- 172.21.15.105:0/2394888767 learned_addr learned my addr 172.21.15.105:0/2394888767 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): get_auth_request con 0x7f76c80a4910 auth_method 0 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): get_auth_request con 0x7f76c80a68e0 auth_method 0 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cd0be640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c81396a0 0x7f76c813d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient(hunting): handle_auth_done global_id 24873 payload 824 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c81396a0 msgr2=0x7f76c813d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f76c81396a0 0x7f76c813d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 msgr2=0x7f76c81d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 0x7f76c81d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 monclient: found mon.noname-d 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c8123450 con 0x7f76c80a4910 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c7fff640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76c80a68e0 0x7f76c81d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76cc8bd640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 0x7f76c81d16b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d800 tx=0x7f76b000dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 client.?.objecter ms_handle_connect 0x7f76c80a4910 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76b000bb30 con 0x7f76c80a4910 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: dump: 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.178 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.179 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b0019070 con 0x7f76c80a4910 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76b000bcd0 con 0x7f76c80a4910 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c57fa640 10 monclient: dump: 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.181 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76c6ffd640 4 set_mon_vals no callback set 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 5 monclient: authenticate success, global_id 24873 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _renew_subs 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c813fae0 con 0x7f76c80a4910 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 client.24873.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _renew_subs 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c813ef80 con 0x7f76c80a4910 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.140+0000 7f76ceb21640 1 librados: init done 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76ceb21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76ceb21640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 0x7f76c813dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f76c8051e80 con 0x7f76c8124110 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 0x7f76c813dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient: get_auth_request con 0x7f76c8124110 auth_method 0 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f76b000b6c0 con 0x7f76c80a4910 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 10 monclient(hunting): handle_auth_done global_id 24882 payload 824 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76c8139ae0 con 0x7f76c8124110 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76a803e5b0 0x7f76a8040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(543..543 src has 1..543) ==== 6713+0+0 (secure 0 0 0) 0x7f76b000f040 con 0x7f76c80a4910 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 client.24873.objecter ms_dispatch 0x7f76c8000ba0 osd_map(543..543 src has 1..543) 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 3 client.24873.objecter handle_osd_map got epochs [543,543] > 0 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 3 client.24873.objecter handle_osd_map decoding full epoch 543 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c7fff640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 0x7f76c813dc50 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f76c813ed80 tx=0x7f76b80077c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 20 client.24873.objecter dump_active .. 0 homeless 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 client.24873.objecter ms_handle_connect 0x7f76c8124110 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 client.24873.objecter resend_mon_ops 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76b8014ac0 con 0x7f76c8124110 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76cd0be640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76a803e5b0 0x7f76a8040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 monclient: dump: 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.185 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76cd0be640 10 monclient: get_auth_request con 0x7f76a803e5b0 auth_method 0 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f76b801a070 con 0x7f76c8124110 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:50.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 msgr2=0x7f76c813dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 0x7f76c813dc50 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f76c813ed80 tx=0x7f76b80077c0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 msgr2=0x7f76c813dc50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c8124110 0x7f76c813dc50 secure :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f76c813ed80 tx=0x7f76b80077c0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76c57fa640 10 client.24873.objecter ms_handle_connect 0x7f76a803e5b0 2024-09-17T16:41:50.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.144+0000 7f76cd0be640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76a803e5b0 0x7f76a8040a70 secure :-1 s=READY pgs=3605 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ad000 tx=0x7f76c0004740 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76ceb21640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76ceb21640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 0x7f76c813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76ceb21640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f76c81238c0 con 0x7f76c813e190 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 0x7f76c813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient: get_auth_request con 0x7f76c813e190 auth_method 0 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.148+0000 7f76c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c7fff640 10 monclient(hunting): handle_auth_done global_id 24891 payload 824 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c7fff640 1 -- 172.21.15.105:0/2394888767 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f76c813eae0 con 0x7f76c813e190 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 client.24873.objecter ms_handle_connect 0x7f76c813e190 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 client.24873.objecter resend_mon_ops 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c7fff640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 0x7f76c813e5a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f76b8019000 tx=0x7f76b8004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76b80147f0 con 0x7f76c813e190 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 monclient: dump: 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.190 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9456 (secure 0 0 0) 0x7f76b8020300 con 0x7f76c813e190 2024-09-17T16:41:50.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 msgr2=0x7f76c813e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 0x7f76c813e5a0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f76b8019000 tx=0x7f76b8004a50 comp rx=0 tx=0).stop 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 msgr2=0x7f76c813e5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76c57fa640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c813e190 0x7f76c813e5a0 secure :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f76b8019000 tx=0x7f76b8004a50 comp rx=0 tx=0).stop 2024-09-17T16:41:50.192 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 857791, 2024-09-17T16:41:50.193 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:50.194 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.195 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.196 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.197 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.198 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.199 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.200 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.201 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.202 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.203 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.204 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.204 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.204 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.204 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.205 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.206 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.207 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.210 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.211 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.212 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.213 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.213 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.213 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.215 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.216 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.217 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.218 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.219 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.224 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.225 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:50.226 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:50.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 10 librados: watch_flush enter 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 10 librados: watch_flush exit 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 20 client.24873.objecter shutdown clearing up homeless session... 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 10 client.24873.objecter successfully canceled tick 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76a803e5b0 msgr2=0x7f76a8040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f76a803e5b0 0x7f76a8040a70 secure :-1 s=READY pgs=3605 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ad000 tx=0x7f76c0004740 comp rx=0 tx=0).stop 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 10 monclient: shutdown 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 msgr2=0x7f76c81d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 --2- 172.21.15.105:0/2394888767 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f76c80a4910 0x7f76c81d16b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d800 tx=0x7f76b000dcc0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76cd0be640 1 -- 172.21.15.105:0/2394888767 reap_dead start 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76cd0be640 1 -- 172.21.15.105:0/2394888767 reap_dead start 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 shutdown_connections 2024-09-17T16:41:50.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 >> 172.21.15.105:0/2394888767 conn(0x7f76c8068ba0 msgr2=0x7f76c80a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 shutdown_connections 2024-09-17T16:41:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 -- 172.21.15.105:0/2394888767 wait complete. 2024-09-17T16:41:50.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.152+0000 7f76ceb21640 1 librados: shutdown 2024-09-17T16:41:50.230 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:50.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:41:50.230 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:41:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: get_monmap_and_config 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: monmap: 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:50.354057+0000 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:50.354057+0000 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:50.359 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 Processor -- start 2024-09-17T16:41:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 -- start start 2024-09-17T16:41:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: init 2024-09-17T16:41:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient: _add_conns ranks=[4,9,6,1,10,2,7,3,5,8,0] 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 0x7f30b80a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-c con 0x7f30b80a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-a con 0x7f30b80a4ab0 addr v2:172.21.15.105:3300/0 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 0x7f30b8139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-d con 0x7f30b81397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f30b80a7c40 con 0x7f30b80a4ab0 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f30b80a7890 con 0x7f30b80a6a30 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f30b80a4080 con 0x7f30b81397f0 2024-09-17T16:41:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bef46640 10 monclient(hunting): authenticate will time out at 1793.619635s 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bd4e3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 0x7f30b8139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bcce2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 0x7f30b80a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46200/0 (socket says 172.21.15.105:46200) 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 1 -- 172.21.15.105:0/483010038 learned_addr learned my addr 172.21.15.105:0/483010038 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): get_auth_request con 0x7f30b80a4ab0 auth_method 0 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bcce2640 10 monclient(hunting): get_auth_request con 0x7f30b80a6a30 auth_method 0 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bcce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bcce2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bcce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bd4e3640 10 monclient(hunting): get_auth_request con 0x7f30b81397f0 auth_method 0 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bd4e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bd4e3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30bd4e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.348+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bd4e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bd4e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bd4e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient(hunting): handle_auth_done global_id 74533 payload 274 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 -- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 msgr2=0x7f30b8139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 0x7f30b8139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 -- 172.21.15.105:0/483010038 >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 msgr2=0x7f30b80a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 --2- 172.21.15.105:0/483010038 >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 monclient: found mon.noname-c 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 -- 172.21.15.105:0/483010038 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b80a37e0 con 0x7f30b80a6a30 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b7fff640 1 --2- 172.21.15.105:0/483010038 >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bd4e3640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 0x7f30b8139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bcce2640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 0x7f30b80a6e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30a40097f0 tx=0x7f30a401c330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=498db18a44d041ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 1 -- 172.21.15.105:0/483010038 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30a40b0070 con 0x7f30b80a6a30 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: dump: 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 1 -- 172.21.15.105:0/483010038 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a40aa810 con 0x7f30b80a6a30 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 5 monclient: authenticate success, global_id 74533 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 1 -- 172.21.15.105:0/483010038 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30a40a9300 con 0x7f30b80a6a30 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 4 set_mon_vals no callback set 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30b77fe640 10 monclient: dump: 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 10 monclient: shutdown 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 -- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 msgr2=0x7f30b80a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 0x7f30b80a6e10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30a40097f0 tx=0x7f30a401c330 comp rx=0 tx=0).stop 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 -- 172.21.15.105:0/483010038 shutdown_connections 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f30b81397f0 0x7f30b8139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 --2- 172.21.15.105:0/483010038 >> v2:172.21.15.105:3300/0 conn(0x7f30b80a4ab0 0x7f30b80a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 --2- 172.21.15.105:0/483010038 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f30b80a6a30 0x7f30b80a6e10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 -- 172.21.15.105:0/483010038 >> 172.21.15.105:0/483010038 conn(0x7f30b80a8d00 msgr2=0x7f30b80a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 -- 172.21.15.105:0/483010038 shutdown_connections 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 1 -- 172.21.15.105:0/483010038 wait complete. 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.352+0000 7f30bef46640 10 monclient: monmap: 2024-09-17T16:41:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 librados: starting msgr at 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 librados: starting objecter 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 Processor -- start 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- start start 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 librados: setting wanted keys 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 librados: calling monclient init 2024-09-17T16:41:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: init 2024-09-17T16:41:50.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _add_conns ranks=[5,2,3,7,0,6,8,1,4] 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f30b80a4ab0 0x7f30b8137cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-h con 0x7f30b80a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 0x7f30b81381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-b con 0x7f30b80a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30b81397f0 0x7f30b8138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): picked mon.noname-g con 0x7f30b81397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f30b806e770 con 0x7f30b80a6a30 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f30b80a2fc0 con 0x7f30b81397f0 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f30b8143260 con 0x7f30b80a4ab0 2024-09-17T16:41:50.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): authenticate will time out at 1793.624918s 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 0x7f30b81381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 0x7f30b81381f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47834/0 (socket says 172.21.15.105:47834) 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 -- 172.21.15.105:0/3357711060 learned_addr learned my addr 172.21.15.105:0/3357711060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): get_auth_request con 0x7f30b80a6a30 auth_method 0 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30b81397f0 0x7f30b8138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f30b80a4ab0 0x7f30b8137cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): get_auth_request con 0x7f30b80a4ab0 auth_method 0 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 10 monclient(hunting): get_auth_request con 0x7f30b81397f0 auth_method 0 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient(hunting): handle_auth_done global_id 84242 payload 824 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f30b80a4ab0 msgr2=0x7f30b8137cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f30b80a4ab0 0x7f30b8137cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30b81397f0 msgr2=0x7f30b8138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30b81397f0 0x7f30b8138730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 monclient: found mon.noname-b 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b813fb80 con 0x7f30b80a6a30 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b7fff640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 0x7f30b81381f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00e9b0 tx=0x7f30ac00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 client.?.objecter ms_handle_connect 0x7f30b80a6a30 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac017070 con 0x7f30b80a6a30 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: dump: 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.385 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac013470 con 0x7f30b80a6a30 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 5 monclient: authenticate success, global_id 84242 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _renew_subs 2024-09-17T16:41:50.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b80aa100 con 0x7f30b80a6a30 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac013a90 con 0x7f30b80a6a30 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b6ffd640 4 set_mon_vals no callback set 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 client.84242.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: dump: 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.387 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.388 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _renew_subs 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b8123a30 con 0x7f30b80a6a30 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 librados: init done 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 0x7f30b8063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f30b8051e80 con 0x7f30b813e890 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 0x7f30b8063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient: get_auth_request con 0x7f30b813e890 auth_method 0 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f30ac01a070 con 0x7f30b80a6a30 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 10 monclient(hunting): handle_auth_done global_id 54737 payload 824 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30b8123d40 con 0x7f30b813e890 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f309c03e580 0x7f309c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(544..544 src has 1..544) ==== 6705+0+0 (secure 0 0 0) 0x7f30ac00f040 con 0x7f30b80a6a30 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 client.84242.objecter ms_dispatch 0x7f30b8000ba0 osd_map(544..544 src has 1..544) 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 3 client.84242.objecter handle_osd_map got epochs [544,544] > 0 2024-09-17T16:41:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 3 client.84242.objecter handle_osd_map decoding full epoch 544 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bcce2640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 0x7f30b8063e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30b806e6e0 tx=0x7f30a40190d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 20 client.84242.objecter dump_active .. 0 homeless 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 client.84242.objecter ms_handle_connect 0x7f30b813e890 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 client.84242.objecter resend_mon_ops 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30a40b9070 con 0x7f30b813e890 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: dump: 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f309c03e580 0x7f309c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30bd4e3640 10 monclient: get_auth_request con 0x7f309c03e580 auth_method 0 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f30a4003620 con 0x7f30b813e890 2024-09-17T16:41:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 msgr2=0x7f30b8063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 0x7f30b8063e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30b806e6e0 tx=0x7f30a40190d0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 msgr2=0x7f30b8063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.356+0000 7f30b57fa640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b813e890 0x7f30b8063e00 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30b806e6e0 tx=0x7f30a40190d0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.360+0000 7f30bd4e3640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f309c03e580 0x7f309c040a40 secure :-1 s=READY pgs=3606 cs=0 l=1 rev1=1 crypto rx=0x7f30b813fa60 tx=0x7f30b000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.360+0000 7f30b57fa640 10 client.84242.objecter ms_handle_connect 0x7f309c03e580 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 0x7f30b802d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f30b806a200 con 0x7f30b8064340 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 0x7f30b802d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient: get_auth_request con 0x7f30b8064340 auth_method 0 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 10 monclient(hunting): handle_auth_done global_id 54746 payload 824 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 1 -- 172.21.15.105:0/3357711060 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f30b80a1120 con 0x7f30b8064340 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 client.84242.objecter ms_handle_connect 0x7f30b8064340 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 client.84242.objecter resend_mon_ops 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bcce2640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 0x7f30b802d450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f30a401c160 tx=0x7f30a40a8000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30a40b9040 con 0x7f30b8064340 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 monclient: dump: 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.399 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f30a40b58c0 con 0x7f30b8064340 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 msgr2=0x7f30b802d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 0x7f30b802d450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f30a401c160 tx=0x7f30a40a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 msgr2=0x7f30b802d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30b57fa640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f30b8064340 0x7f30b802d450 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f30a401c160 tx=0x7f30a40a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:50.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 417361, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:50.401 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.402 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.403 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:50.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.405 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.406 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.407 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:50.408 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.411 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:50.416 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.417 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.422 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.423 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.424 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.425 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:50.426 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:50.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 librados: watch_flush enter 2024-09-17T16:41:50.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 librados: watch_flush exit 2024-09-17T16:41:50.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 20 client.84242.objecter shutdown clearing up homeless session... 2024-09-17T16:41:50.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 client.84242.objecter successfully canceled tick 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f309c03e580 msgr2=0x7f309c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f309c03e580 0x7f309c040a40 secure :-1 s=READY pgs=3606 cs=0 l=1 rev1=1 crypto rx=0x7f30b813fa60 tx=0x7f30b000b040 comp rx=0 tx=0).stop 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 10 monclient: shutdown 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 msgr2=0x7f30b81381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 --2- 172.21.15.105:0/3357711060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30b80a6a30 0x7f30b81381f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00e9b0 tx=0x7f30ac00ee70 comp rx=0 tx=0).stop 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bd4e3640 1 -- 172.21.15.105:0/3357711060 reap_dead start 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bd4e3640 1 -- 172.21.15.105:0/3357711060 reap_dead start 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 shutdown_connections 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 >> 172.21.15.105:0/3357711060 conn(0x7f30b80a8d00 msgr2=0x7f30b813fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 shutdown_connections 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 -- 172.21.15.105:0/3357711060 wait complete. 2024-09-17T16:41:50.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.364+0000 7f30bef46640 1 librados: shutdown 2024-09-17T16:41:50.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:50.430 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:41:50.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: get_monmap_and_config 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: monmap: 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.545 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:50.541006+0000 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:50.541006+0000 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.546 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.547 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:50.548 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:50.548 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 Processor -- start 2024-09-17T16:41:50.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 -- start start 2024-09-17T16:41:50.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: init 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient: _add_conns ranks=[5,2,0,1,8,7,9,10,4,3,6] 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44640a68e0 0x7f44640a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-h con 0x7f44640a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 0x7f44640a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-b con 0x7f44640a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44641396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446406ebc0 con 0x7f44641396a0 2024-09-17T16:41:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f44640a3fc0 con 0x7f44640a4910 2024-09-17T16:41:50.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f44640a63c0 con 0x7f44640a68e0 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4469d00640 10 monclient(hunting): authenticate will time out at 1793.806746s 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4463fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f44641396a0 0x7f4464139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55190/0 (socket says 172.21.15.105:55190) 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4463fff640 1 -- 172.21.15.105:0/2103597426 learned_addr learned my addr 172.21.15.105:0/2103597426 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 0x7f44640a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): get_auth_request con 0x7f44640a4910 auth_method 0 2024-09-17T16:41:50.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44640a68e0 0x7f44640a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): get_auth_request con 0x7f44640a68e0 auth_method 0 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 1 -- 172.21.15.105:0/2103597426 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132078976 0 0) 0x7f446406ebc0 con 0x7f44641396a0 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 10 monclient(hunting): _negotiate 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 10 monclient(hunting): my global_id is 36342 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44627fc640 1 -- 172.21.15.105:0/2103597426 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f444c003530 con 0x7f44641396a0 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient(hunting): handle_auth_done global_id 84251 payload 274 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 -- 172.21.15.105:0/2103597426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44640a68e0 msgr2=0x7f44640a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44640a68e0 0x7f44640a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 -- 172.21.15.105:0/2103597426 >> v1:172.21.15.105:6789/0 conn(0x7f44641396a0 legacy=0x7f4464139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 monclient: found mon.noname-b 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:50.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.536+0000 7f4462ffd640 1 -- 172.21.15.105:0/2103597426 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44640a3690 con 0x7f44640a4910 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4462ffd640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 0x7f44640a7200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f44580097f0 tx=0x7f445801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc193988c7bf9154 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 1 -- 172.21.15.105:0/2103597426 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44580b0070 con 0x7f44640a4910 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: dump: 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.575 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 1 -- 172.21.15.105:0/2103597426 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44580aa810 con 0x7f44640a4910 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 1 -- 172.21.15.105:0/2103597426 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44580a9300 con 0x7f44640a4910 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 5 monclient: authenticate success, global_id 84251 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 4 set_mon_vals no callback set 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:50.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f44627fc640 10 monclient: dump: 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.578 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: shutdown 2024-09-17T16:41:50.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- 172.21.15.105:0/2103597426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 msgr2=0x7f44640a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 0x7f44640a7200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f44580097f0 tx=0x7f445801c3a0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- 172.21.15.105:0/2103597426 shutdown_connections 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44640a4910 0x7f44640a7200 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- 172.21.15.105:0/2103597426 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f44640a68e0 0x7f44640a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- 172.21.15.105:0/2103597426 >> 172.21.15.105:0/2103597426 conn(0x7f4464068ba0 msgr2=0x7f4464068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- 172.21.15.105:0/2103597426 shutdown_connections 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- 172.21.15.105:0/2103597426 wait complete. 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: monmap: 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.581 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.582 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 librados: starting msgr at 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 librados: starting objecter 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 Processor -- start 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- start start 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 librados: setting wanted keys 2024-09-17T16:41:50.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 librados: calling monclient init 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: init 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient: _add_conns ranks=[4,0,7,8,1,3,2,5,6] 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44640a4910 0x7f44641d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-c con 0x7f44640a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 0x7f44641d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-a con 0x7f44640a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44641396a0 0x7f446413d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): picked mon.noname-i con 0x7f44641396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44641444b0 con 0x7f44640a68e0 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f4464144330 con 0x7f44640a4910 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f4464144630 con 0x7f44641396a0 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.540+0000 7f4469d00640 10 monclient(hunting): authenticate will time out at 1793.811918s 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44640a4910 0x7f44641d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 0x7f44641d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 0x7f44641d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46206/0 (socket says 172.21.15.105:46206) 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 -- 172.21.15.105:0/1089402863 learned_addr learned my addr 172.21.15.105:0/1089402863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): get_auth_request con 0x7f44640a68e0 auth_method 0 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): get_auth_request con 0x7f44640a4910 auth_method 0 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient(hunting): handle_auth_done global_id 36351 payload 824 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44641396a0 msgr2=0x7f446413d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f44641396a0 0x7f446413d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44640a4910 msgr2=0x7f44641d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44640a4910 0x7f44641d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 monclient: found mon.noname-a 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4464122af0 con 0x7f44640a68e0 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f44640a4910 0x7f44641d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4462ffd640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 0x7f44641d1d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4458009710 tx=0x7f4458019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.?.objecter ms_handle_connect 0x7f44640a68e0 2024-09-17T16:41:50.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44580b0040 con 0x7f44640a68e0 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: dump: 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.590 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.591 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44580044b0 con 0x7f44640a68e0 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 5 monclient: authenticate success, global_id 36351 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 monclient: _renew_subs 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4464140b30 con 0x7f44640a68e0 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44580aac00 con 0x7f44640a68e0 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: dump: 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.593 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 client.36351.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4461ffb640 4 set_mon_vals no callback set 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 monclient: _renew_subs 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44640a9de0 con 0x7f44640a68e0 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 1 librados: init done 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 0x7f446413df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4464051e80 con 0x7f4464122530 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4458004040 con 0x7f44640a68e0 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 0x7f446413df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient: get_auth_request con 0x7f4464122530 auth_method 0 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f445003e540 0x7f4450040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(544..544 src has 1..544) ==== 6705+0+0 (secure 0 0 0) 0x7f44580e9b20 con 0x7f44640a68e0 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.36351.objecter ms_dispatch 0x7f4464000ba0 osd_map(544..544 src has 1..544) 2024-09-17T16:41:50.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 3 client.36351.objecter handle_osd_map got epochs [544,544] > 0 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 3 client.36351.objecter handle_osd_map decoding full epoch 544 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 20 client.36351.objecter dump_active .. 0 homeless 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4463fff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f445003e540 0x7f4450040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4463fff640 10 monclient: get_auth_request con 0x7f445003e540 auth_method 0 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 10 monclient(hunting): handle_auth_done global_id 84196 payload 824 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4464122910 con 0x7f4464122530 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.36351.objecter ms_handle_connect 0x7f4464122530 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.36351.objecter resend_mon_ops 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f44637fe640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 0x7f446413df50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4454011cf0 tx=0x7f445400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44540148b0 con 0x7f4464122530 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 monclient: dump: 2024-09-17T16:41:50.596 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.597 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f444bfff640 10 client.36351.objecter ms_handle_connect 0x7f445003e540 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.544+0000 7f4463fff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f445003e540 0x7f4450040a00 secure :-1 s=READY pgs=3607 cs=0 l=1 rev1=1 crypto rx=0x7f445c0015b0 tx=0x7f445c001600 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4454014440 con 0x7f4464122530 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 msgr2=0x7f446413df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 0x7f446413df50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4454011cf0 tx=0x7f445400f610 comp rx=0 tx=0).stop 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 msgr2=0x7f446413df50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.548+0000 7f444bfff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4464122530 0x7f446413df50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f4469d00640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f4469d00640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 0x7f446402d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f4469d00640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4464139ae0 con 0x7f446402d450 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 0x7f446402d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient: get_auth_request con 0x7f446402d450 auth_method 0 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 10 monclient(hunting): handle_auth_done global_id 84205 payload 824 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 1 -- 172.21.15.105:0/1089402863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f44640a1120 con 0x7f446402d450 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 10 client.36351.objecter ms_handle_connect 0x7f446402d450 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 10 client.36351.objecter resend_mon_ops 2024-09-17T16:41:50.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f44637fe640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 0x7f446402d830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f445401b000 tx=0x7f44540046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f445401dca0 con 0x7f446402d450 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.552+0000 7f444bfff640 10 monclient: dump: 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.600 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9286 (secure 0 0 0) 0x7f44540273f0 con 0x7f446402d450 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 msgr2=0x7f446402d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 0x7f446402d830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f445401b000 tx=0x7f44540046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 msgr2=0x7f446402d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f444bfff640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f446402d450 0x7f446402d830 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f445401b000 tx=0x7f44540046d0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 17787, 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:50.602 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.603 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:50.604 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:50.606 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:50.607 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.608 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:50.610 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.611 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.612 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.613 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.615 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.617 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.618 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.619 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.621 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.621 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:50.621 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.621 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.624 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.626 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.627 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.629 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:50.630 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 10 librados: watch_flush enter 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 10 librados: watch_flush exit 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 20 client.36351.objecter shutdown clearing up homeless session... 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 10 client.36351.objecter successfully canceled tick 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f445003e540 msgr2=0x7f4450040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f445003e540 0x7f4450040a00 secure :-1 s=READY pgs=3607 cs=0 l=1 rev1=1 crypto rx=0x7f445c0015b0 tx=0x7f445c001600 comp rx=0 tx=0).stop 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 10 monclient: shutdown 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 msgr2=0x7f44641d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 --2- 172.21.15.105:0/1089402863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44640a68e0 0x7f44641d1d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4458009710 tx=0x7f4458019140 comp rx=0 tx=0).stop 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4463fff640 1 -- 172.21.15.105:0/1089402863 reap_dead start 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4463fff640 1 -- 172.21.15.105:0/1089402863 reap_dead start 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 shutdown_connections 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 >> 172.21.15.105:0/1089402863 conn(0x7f4464068ba0 msgr2=0x7f44640a2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 shutdown_connections 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 -- 172.21.15.105:0/1089402863 wait complete. 2024-09-17T16:41:50.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.556+0000 7f4469d00640 1 librados: shutdown 2024-09-17T16:41:50.633 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:50.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:41:50.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: get_monmap_and_config 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: monmap: 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:50.750067+0000 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:50.750067+0000 2024-09-17T16:41:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:50.755 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 Processor -- start 2024-09-17T16:41:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 -- start start 2024-09-17T16:41:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: init 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient: _add_conns ranks=[8,5,4,2,1,6,3,7,9,0,10] 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-e con 0x7fa3cc0a2620 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-h con 0x7fa3cc0a2f40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 0x7fa3cc137d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-c con 0x7fa3cc137940 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa3cc06e580 con 0x7fa3cc137940 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa3cc06f2e0 con 0x7fa3cc0a2f40 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa3cc0a15e0 con 0x7fa3cc0a2620 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d3968640 10 monclient(hunting): authenticate will time out at 1794.015798s 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d1704640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.744+0000 7fa3d1704640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:56340/0 (socket says 172.21.15.105:56340) 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 -- 172.21.15.105:0/3386876498 learned_addr learned my addr 172.21.15.105:0/3386876498 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): get_auth_request con 0x7fa3cc0a2620 auth_method 0 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 0x7fa3cc137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): get_auth_request con 0x7fa3cc137940 auth_method 0 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient(hunting): handle_auth_done global_id 54755 payload 274 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d0f03640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 -- 172.21.15.105:0/3386876498 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 msgr2=0x7fa3cc137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 -- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 msgr2=0x7fa3cc137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 0x7fa3cc137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 monclient: found mon.noname-e 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 -- 172.21.15.105:0/3386876498 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc0a3c30 con 0x7fa3cc0a2620 2024-09-17T16:41:50.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1f05640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 0x7fa3cc137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d0f03640 10 monclient: get_auth_request con 0x7fa3cc0a2f40 auth_method 0 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d0f03640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d1704640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098a0 tx=0x7fa3bc01c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=42517e925f5496d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 1 -- 172.21.15.105:0/3386876498 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3bc0b0070 con 0x7fa3cc0a2620 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: dump: 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 1 -- 172.21.15.105:0/3386876498 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc0aa810 con 0x7fa3cc0a2620 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 1 -- 172.21.15.105:0/3386876498 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3bc0a9300 con 0x7fa3cc0a2620 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 5 monclient: authenticate success, global_id 54755 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 4 set_mon_vals no callback set 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3c3fff640 10 monclient: dump: 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.766 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 10 monclient: shutdown 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 -- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 msgr2=0x7fa3cc0a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098a0 tx=0x7fa3bc01c440 comp rx=0 tx=0).stop 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 -- 172.21.15.105:0/3386876498 shutdown_connections 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa3cc137940 0x7fa3cc137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 --2- 172.21.15.105:0/3386876498 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa3cc0a2620 0x7fa3cc0a2a00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 -- 172.21.15.105:0/3386876498 >> 172.21.15.105:0/3386876498 conn(0x7fa3cc068ba0 msgr2=0x7fa3cc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 -- 172.21.15.105:0/3386876498 shutdown_connections 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.748+0000 7fa3d3968640 1 -- 172.21.15.105:0/3386876498 wait complete. 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: monmap: 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.770 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:50.771 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.772 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 librados: starting msgr at 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 librados: starting objecter 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 Processor -- start 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- start start 2024-09-17T16:41:50.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 librados: setting wanted keys 2024-09-17T16:41:50.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 librados: calling monclient init 2024-09-17T16:41:50.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: init 2024-09-17T16:41:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _add_conns ranks=[1,5,6,3,4,8,2,0,7] 2024-09-17T16:41:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0a2620 0x7fa3cc11e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-f con 0x7fa3cc0a2620 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc11e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-h con 0x7fa3cc0a2f40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 0x7fa3cc11ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): picked mon.noname-d con 0x7fa3cc137940 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0a1f50 con 0x7fa3cc0a2620 2024-09-17T16:41:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa3cc0a1dd0 con 0x7fa3cc0a2f40 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa3cc141500 con 0x7fa3cc137940 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): authenticate will time out at 1794.021113s 2024-09-17T16:41:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 0x7fa3cc11ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 0x7fa3cc11ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:38608/0 (socket says 172.21.15.105:38608) 2024-09-17T16:41:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 learned_addr learned my addr 172.21.15.105:0/894192128 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): get_auth_request con 0x7fa3cc137940 auth_method 0 2024-09-17T16:41:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0a2620 0x7fa3cc11e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): get_auth_request con 0x7fa3cc0a2620 auth_method 0 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient(hunting): handle_auth_done global_id 24909 payload 824 2024-09-17T16:41:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 msgr2=0x7fa3cc11e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa3cc0a2f40 0x7fa3cc11e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0a2620 msgr2=0x7fa3cc11e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3cc0a2620 0x7fa3cc11e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 monclient: found mon.noname-d 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc120940 con 0x7fa3cc137940 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1f05640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 0x7fa3cc11ecf0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400e900 tx=0x7fa3c400edc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 client.?.objecter ms_handle_connect 0x7fa3cc137940 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3c4017070 con 0x7fa3cc137940 2024-09-17T16:41:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: dump: 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c400cdb0 con 0x7fa3cc137940 2024-09-17T16:41:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3c40138e0 con 0x7fa3cc137940 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c37fe640 4 set_mon_vals no callback set 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 monclient: dump: 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 5 monclient: authenticate success, global_id 24909 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _renew_subs 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc13d8d0 con 0x7fa3cc137940 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 client.24909.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _renew_subs 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc120f10 con 0x7fa3cc137940 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 librados: init done 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 0x7fa3cc02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc136560 con 0x7fa3cc1cfbe0 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa3c401a070 con 0x7fa3cc137940 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 0x7fa3cc02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient: get_auth_request con 0x7fa3cc1cfbe0 auth_method 0 2024-09-17T16:41:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3b003e550 0x7fa3b0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(544..544 src has 1..544) ==== 6705+0+0 (secure 0 0 0) 0x7fa3c400f040 con 0x7fa3cc137940 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 10 client.24909.objecter ms_dispatch 0x7fa3cc000ba0 osd_map(544..544 src has 1..544) 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 3 client.24909.objecter handle_osd_map got epochs [544,544] > 0 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 3 client.24909.objecter handle_osd_map decoding full epoch 544 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3c1ffb640 20 client.24909.objecter dump_active .. 0 homeless 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d0f03640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3b003e550 0x7fa3b0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d0f03640 10 monclient: get_auth_request con 0x7fa3b003e550 auth_method 0 2024-09-17T16:41:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 10 monclient(hunting): handle_auth_done global_id 54804 payload 824 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.752+0000 7fa3d1704640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3cc0a2a90 con 0x7fa3cc1cfbe0 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 client.24909.objecter ms_handle_connect 0x7fa3cc1cfbe0 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3d1704640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 0x7fa3cc02d450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc0191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 client.24909.objecter resend_mon_ops 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3bc0b9030 con 0x7fa3cc1cfbe0 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 monclient: dump: 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 client.24909.objecter ms_handle_connect 0x7fa3b003e550 2024-09-17T16:41:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3d0f03640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3b003e550 0x7fa3b0040a10 secure :-1 s=READY pgs=3608 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc13d730 tx=0x7fa3c80083d0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa3bc0b1040 con 0x7fa3cc1cfbe0 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 msgr2=0x7fa3cc02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 0x7fa3cc02d450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc0191e0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 msgr2=0x7fa3cc02d450 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.756+0000 7fa3c1ffb640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cfbe0 0x7fa3cc02d450 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d3968640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d3968640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 0x7fa3cc02d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d3968640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc051e80 con 0x7fa3cc1cffc0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 0x7fa3cc02d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient: get_auth_request con 0x7fa3cc1cffc0 auth_method 0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.760+0000 7fa3d1704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d1704640 10 monclient(hunting): handle_auth_done global_id 54813 payload 824 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d1704640 1 -- 172.21.15.105:0/894192128 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa3cc137400 con 0x7fa3cc1cffc0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d1704640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 0x7fa3cc02d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01c270 tx=0x7fa3bc0a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 client.24909.objecter ms_handle_connect 0x7fa3cc1cffc0 2024-09-17T16:41:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 client.24909.objecter resend_mon_ops 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3bc0b9040 con 0x7fa3cc1cffc0 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 monclient: dump: 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fa3bc0c0d60 con 0x7fa3cc1cffc0 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 msgr2=0x7fa3cc02d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 0x7fa3cc02d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01c270 tx=0x7fa3bc0a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 msgr2=0x7fa3cc02d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3c1ffb640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3cc1cffc0 0x7fa3cc02d990 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01c270 tx=0x7fa3bc0a8000 comp rx=0 tx=0).stop 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 417775, 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.808 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:50.810 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.811 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.812 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.814 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.815 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.815 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.818 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:50.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.824 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.827 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.828 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:50.829 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:50.832 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.832 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.832 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:50.832 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 10 librados: watch_flush enter 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 10 librados: watch_flush exit 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 20 client.24909.objecter shutdown clearing up homeless session... 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 10 client.24909.objecter successfully canceled tick 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3b003e550 msgr2=0x7fa3b0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa3b003e550 0x7fa3b0040a10 secure :-1 s=READY pgs=3608 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc13d730 tx=0x7fa3c80083d0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 10 monclient: shutdown 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 msgr2=0x7fa3cc11ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 --2- 172.21.15.105:0/894192128 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa3cc137940 0x7fa3cc11ecf0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400e900 tx=0x7fa3c400edc0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 reap_dead start 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d1f05640 1 -- 172.21.15.105:0/894192128 reap_dead start 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 shutdown_connections 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 >> 172.21.15.105:0/894192128 conn(0x7fa3cc068ba0 msgr2=0x7fa3cc0715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 shutdown_connections 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 -- 172.21.15.105:0/894192128 wait complete. 2024-09-17T16:41:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.764+0000 7fa3d3968640 1 librados: shutdown 2024-09-17T16:41:50.835 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:50.835 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:41:50.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: get_monmap_and_config 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: monmap: 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:50.948492+0000 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:50.948492+0000 2024-09-17T16:41:50.952 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:50.953 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 Processor -- start 2024-09-17T16:41:50.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 -- start start 2024-09-17T16:41:50.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: init 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient: _add_conns ranks=[0,1,2,10,6,8,4,9,5,3,7] 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff6940a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff6940a4910 addr v1:172.21.15.160:6789/0 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 0x7ff694142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-b con 0x7ff6940a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69406ebc0 con 0x7ff6940a68e0 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6940a3690 con 0x7ff6940a4910 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff6940a3fc0 con 0x7ff6940a73d0 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69bf83640 10 monclient(hunting): authenticate will time out at 1794.214266s 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 0x7ff694142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff699d1f640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff6940a68e0 0x7ff6940a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55204/0 (socket says 172.21.15.105:55204) 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff699d1f640 1 -- 172.21.15.105:0/451429366 learned_addr learned my addr 172.21.15.105:0/451429366 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): get_auth_request con 0x7ff6940a73d0 auth_method 0 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 1 -- 172.21.15.105:0/451429366 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987090724 0 0) 0x7ff69406ebc0 con 0x7ff6940a68e0 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 10 monclient(hunting): _negotiate 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 10 monclient(hunting): my global_id is 36360 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff698d1d640 1 -- 172.21.15.105:0/451429366 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff680003530 con 0x7ff6940a68e0 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient(hunting): handle_auth_done global_id 84260 payload 274 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 1 -- 172.21.15.105:0/451429366 >> v1:172.21.15.160:6789/0 conn(0x7ff6940a4910 legacy=0x7ff69413eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 1 -- 172.21.15.105:0/451429366 >> v1:172.21.15.105:6789/0 conn(0x7ff6940a68e0 legacy=0x7ff6940a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 1 monclient: found mon.noname-b 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:41:50.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.944+0000 7ff69a520640 1 -- 172.21.15.105:0/451429366 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6940a3970 con 0x7ff6940a73d0 2024-09-17T16:41:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 1 --2- 172.21.15.105:0/451429366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 0x7ff694142220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009dc0 tx=0x7ff68c01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584d645a20bab2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 1 -- 172.21.15.105:0/451429366 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff68c0b2070 con 0x7ff6940a73d0 2024-09-17T16:41:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: dump: 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 1 -- 172.21.15.105:0/451429366 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff68c0a5a90 con 0x7ff6940a73d0 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 1 -- 172.21.15.105:0/451429366 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff68c0a5c40 con 0x7ff6940a73d0 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 5 monclient: authenticate success, global_id 84260 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 4 set_mon_vals no callback set 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff698d1d640 10 monclient: dump: 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: shutdown 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- 172.21.15.105:0/451429366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 msgr2=0x7ff694142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 --2- 172.21.15.105:0/451429366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 0x7ff694142220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009dc0 tx=0x7ff68c01c850 comp rx=0 tx=0).stop 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- 172.21.15.105:0/451429366 shutdown_connections 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 --2- 172.21.15.105:0/451429366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff6940a73d0 0x7ff694142220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- 172.21.15.105:0/451429366 >> 172.21.15.105:0/451429366 conn(0x7ff694068ba0 msgr2=0x7ff694068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- 172.21.15.105:0/451429366 shutdown_connections 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- 172.21.15.105:0/451429366 wait complete. 2024-09-17T16:41:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: build_initial_monmap 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: monmap: 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 librados: starting msgr at 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 librados: starting objecter 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 Processor -- start 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- start start 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 librados: setting wanted keys 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 librados: calling monclient init 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: init 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient: _add_conns ranks=[1,0,5,6,2,3,8,4,7] 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6940a4910 0x7ff694125ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-f con 0x7ff6940a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 0x7ff694126530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-a con 0x7ff6940a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff6940a73d0 0x7ff694126a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): picked mon.noname-h con 0x7ff6940a73d0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff694145310 con 0x7ff6940a68e0 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff694145190 con 0x7ff6940a4910 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff694145490 con 0x7ff6940a73d0 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69bf83640 10 monclient(hunting): authenticate will time out at 1794.219516s 2024-09-17T16:41:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 0x7ff694126530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 0x7ff694126530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46208/0 (socket says 172.21.15.105:46208) 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 1 -- 172.21.15.105:0/4222500610 learned_addr learned my addr 172.21.15.105:0/4222500610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): get_auth_request con 0x7ff6940a68e0 auth_method 0 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69951e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff6940a73d0 0x7ff694126a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff699d1f640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6940a4910 0x7ff694125ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 10 monclient(hunting): get_auth_request con 0x7ff6940a73d0 auth_method 0 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.948+0000 7ff69a520640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): get_auth_request con 0x7ff6940a4910 auth_method 0 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 10 monclient(hunting): handle_auth_done global_id 36369 payload 824 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff6940a73d0 msgr2=0x7ff694126a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff6940a73d0 0x7ff694126a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6940a4910 msgr2=0x7ff694125ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6940a4910 0x7ff694125ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 10 monclient: _finish_hunting 0 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 monclient: found mon.noname-a 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff694129e40 con 0x7ff6940a68e0 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69951e640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 0x7ff694126530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c500 tx=0x7ff68400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.?.objecter ms_handle_connect 0x7ff6940a68e0 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff684017070 con 0x7ff6940a68e0 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: dump: 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.974 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff684003600 con 0x7ff6940a68e0 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 5 monclient: authenticate success, global_id 36369 2024-09-17T16:41:50.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 monclient: _renew_subs 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff684003ca0 con 0x7ff6940a68e0 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69406df40 con 0x7ff6940a68e0 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68bfff640 4 set_mon_vals no callback set 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: dump: 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.976 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 client.36369.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:50.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 monclient: _renew_subs 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff694129b60 con 0x7ff6940a68e0 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 1 librados: init done 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 0x7ff694122910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff684003830 con 0x7ff6940a68e0 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff694051e80 con 0x7ff694122530 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff67803e550 0x7ff678040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(544..544 src has 1..544) ==== 6705+0+0 (secure 0 0 0) 0x7ff68404f250 con 0x7ff6940a68e0 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.36369.objecter ms_dispatch 0x7ff694000ba0 osd_map(544..544 src has 1..544) 2024-09-17T16:41:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 3 client.36369.objecter handle_osd_map got epochs [544,544] > 0 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 3 client.36369.objecter handle_osd_map decoding full epoch 544 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 0x7ff694122910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient: get_auth_request con 0x7ff694122530 auth_method 0 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 20 client.36369.objecter dump_active .. 0 homeless 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69a520640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff67803e550 0x7ff678040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69a520640 10 monclient: get_auth_request con 0x7ff67803e550 auth_method 0 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 10 monclient(hunting): handle_auth_done global_id 74309 payload 824 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6940a1f70 con 0x7ff694122530 2024-09-17T16:41:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.36369.objecter ms_handle_connect 0x7ff694122530 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff699d1f640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 0x7ff694122910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff69000e980 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.36369.objecter resend_mon_ops 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 client.36369.objecter ms_handle_connect 0x7ff67803e550 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff69a520640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff67803e550 0x7ff678040a10 secure :-1 s=READY pgs=3609 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af000 tx=0x7ff68c0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6900070d0 con 0x7ff694122530 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: dump: 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff690018070 con 0x7ff694122530 2024-09-17T16:41:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 msgr2=0x7ff694122910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 0x7ff694122910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff69000e980 comp rx=0 tx=0).stop 2024-09-17T16:41:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 msgr2=0x7ff694122910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.952+0000 7ff68a7fc640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122530 0x7ff694122910 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff69000e980 comp rx=0 tx=0).stop 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff69bf83640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff69bf83640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 0x7ff6941db3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff69bf83640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff6940a6cc0 con 0x7ff694122e50 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 0x7ff6941db3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient: get_auth_request con 0x7ff694122e50 auth_method 0 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 10 monclient(hunting): handle_auth_done global_id 74318 payload 824 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 1 -- 172.21.15.105:0/4222500610 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff6940746d0 con 0x7ff694122e50 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff699d1f640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 0x7ff6941db3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff690005800 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 10 client.36369.objecter ms_handle_connect 0x7ff694122e50 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 10 client.36369.objecter resend_mon_ops 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff690007540 con 0x7ff694122e50 2024-09-17T16:41:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.960+0000 7ff68a7fc640 10 monclient: dump: 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff6900070d0 con 0x7ff694122e50 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 msgr2=0x7ff6941db3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 0x7ff6941db3e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff690005800 comp rx=0 tx=0).stop 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 msgr2=0x7ff6941db3e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff68a7fc640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff694122e50 0x7ff6941db3e0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff690005890 tx=0x7ff690005800 comp rx=0 tx=0).stop 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150434, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:50.988 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.989 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.990 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:50.991 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:50.992 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.993 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.994 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:50.996 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:50.998 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:50.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.002 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.004 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:51.005 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.006 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:51.007 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.008 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:51.009 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 10 librados: watch_flush enter 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 10 librados: watch_flush exit 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 20 client.36369.objecter shutdown clearing up homeless session... 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 10 client.36369.objecter successfully canceled tick 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff67803e550 msgr2=0x7ff678040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff67803e550 0x7ff678040a10 secure :-1 s=READY pgs=3609 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0af000 tx=0x7ff68c0a7070 comp rx=0 tx=0).stop 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 10 monclient: shutdown 2024-09-17T16:41:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 msgr2=0x7ff694126530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 --2- 172.21.15.105:0/4222500610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6940a68e0 0x7ff694126530 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c500 tx=0x7ff68400c9c0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69a520640 1 -- 172.21.15.105:0/4222500610 reap_dead start 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69a520640 1 -- 172.21.15.105:0/4222500610 reap_dead start 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 shutdown_connections 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 >> 172.21.15.105:0/4222500610 conn(0x7ff694068ba0 msgr2=0x7ff69409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 shutdown_connections 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 -- 172.21.15.105:0/4222500610 wait complete. 2024-09-17T16:41:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:50.964+0000 7ff69bf83640 1 librados: shutdown 2024-09-17T16:41:51.012 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:51.013 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:41:51.013 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 10 monclient: get_monmap_and_config 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 10 monclient: build_initial_monmap 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 10 monclient: monmap: 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:41:51.126200+0000 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:41:51.126200+0000 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:41:51.131 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 1 Processor -- start 2024-09-17T16:41:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 1 -- start start 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.120+0000 7f33fcac8640 10 monclient: init 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient: _add_conns ranks=[10,3,8,9,1,6,4,5,0,2,7] 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 0x7f33f80a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-f con 0x7f33f80a2620 addr v2:172.21.15.160:3300/0 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f80a2f40 0x7f33f8137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-g con 0x7f33f80a2f40 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-e con 0x7f33f8137940 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f33f806e580 con 0x7f33f80a2620 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f33f806f2e0 con 0x7f33f80a2f40 2024-09-17T16:41:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f33f80a15e0 con 0x7f33f8137940 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient(hunting): authenticate will time out at 1794.392414s 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:56354/0 (socket says 172.21.15.105:56354) 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 -- 172.21.15.105:0/964053770 learned_addr learned my addr 172.21.15.105:0/964053770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): get_auth_request con 0x7f33f8137940 auth_method 0 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 1 --2- 172.21.15.105:0/964053770 >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 0x7f33f80a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 10 monclient(hunting): get_auth_request con 0x7f33f80a2620 auth_method 0 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f5d9b640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f80a2f40 0x7f33f8137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f5d9b640 10 monclient(hunting): get_auth_request con 0x7f33f80a2f40 auth_method 0 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_done global_id 54764 payload 274 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 -- 172.21.15.105:0/964053770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f80a2f40 msgr2=0x7f33f8137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f80a2f40 0x7f33f8137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 -- 172.21.15.105:0/964053770 >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 msgr2=0x7f33f80a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/964053770 >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 0x7f33f80a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:41:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 monclient: found mon.noname-e 2024-09-17T16:41:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 -- 172.21.15.105:0/964053770 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f80a3c30 con 0x7f33f8137940 2024-09-17T16:41:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f659c640 1 --2- 172.21.15.105:0/964053770 >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 0x7f33f80a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009870 tx=0x7f33f001c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4b58880395326f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 1 -- 172.21.15.105:0/964053770 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33f00b0070 con 0x7f33f8137940 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: dump: 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.144 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.145 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: _finish_auth 0 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: _check_auth_tickets 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 5 monclient: authenticate success, global_id 54764 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 1 -- 172.21.15.105:0/964053770 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f00aa810 con 0x7f33f8137940 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient: get_monmap_and_config success 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 4 set_mon_vals no callback set 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 1 -- 172.21.15.105:0/964053770 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33f00a9300 con 0x7f33f8137940 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33f559a640 10 monclient: dump: 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 10 monclient: shutdown 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- 172.21.15.105:0/964053770 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 msgr2=0x7f33f8137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009870 tx=0x7f33f001c420 comp rx=0 tx=0).stop 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- 172.21.15.105:0/964053770 shutdown_connections 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f8137940 0x7f33f8137df0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- 172.21.15.105:0/964053770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f80a2f40 0x7f33f8137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 --2- 172.21.15.105:0/964053770 >> v2:172.21.15.160:3300/0 conn(0x7f33f80a2620 0x7f33f80a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.124+0000 7f33fcac8640 1 -- 172.21.15.105:0/964053770 >> 172.21.15.105:0/964053770 conn(0x7f33f8068ba0 msgr2=0x7f33f8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- 172.21.15.105:0/964053770 shutdown_connections 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- 172.21.15.105:0/964053770 wait complete. 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: build_initial_monmap 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: monmap: 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:41:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 librados: starting msgr at 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 librados: starting objecter 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 Processor -- start 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- start start 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 librados: setting wanted keys 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 librados: calling monclient init 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: init 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _reopen_session rank -1 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _add_conns ranks=[1,8,3,0,5,2,7,4,6] 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33f80a2620 0x7f33f809f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-f con 0x7f33f80a2620 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f80a2f40 0x7f33f809f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-e con 0x7f33f80a2f40 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f8137940 0x7f33f809fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): picked mon.noname-g con 0x7f33f8137940 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:41:51.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f33f80a1f50 con 0x7f33f80a2620 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f33f80a1dd0 con 0x7f33f8137940 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f33f8141500 con 0x7f33f80a2f40 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): _renew_subs 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): authenticate will time out at 1794.397672s 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33f80a2620 0x7f33f809f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33f80a2620 0x7f33f809f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:59716/0 (socket says 172.21.15.105:59716) 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f8137940 0x7f33f809fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 1 -- 172.21.15.105:0/3685425214 learned_addr learned my addr 172.21.15.105:0/3685425214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): get_auth_request con 0x7f33f80a2620 auth_method 0 2024-09-17T16:41:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f80a2f40 0x7f33f809f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): get_auth_request con 0x7f33f8137940 auth_method 0 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): get_auth_request con 0x7f33f80a2f40 auth_method 0 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient(hunting): handle_auth_done global_id 54773 payload 824 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f8137940 msgr2=0x7f33f809fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33f8137940 0x7f33f809fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33f80a2620 msgr2=0x7f33f809f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33f80a2620 0x7f33f809f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 monclient: found mon.noname-e 2024-09-17T16:41:51.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f80a70d0 con 0x7f33f80a2f40 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 client.?.objecter ms_handle_connect 0x7f33f80a2f40 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f5d9b640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f80a2f40 0x7f33f809f6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33e8007e10 tx=0x7f33e800e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33e8014890 con 0x7f33f80a2f40 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: dump: 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:41:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e8014a70 con 0x7f33f80a2f40 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33e8014c20 con 0x7f33f80a2f40 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 5 monclient: authenticate success, global_id 54773 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33f4d99640 4 set_mon_vals no callback set 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 10 monclient: dump: 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _renew_subs 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f813c760 con 0x7f33f80a2f40 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 client.54773.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _renew_subs 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f80a1190 con 0x7f33f80a2f40 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 librados: init done 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 0x7f33f8063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f33f8051e80 con 0x7f33f802e5d0 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.128+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f33e801b030 con 0x7f33f80a2f40 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 0x7f33f8063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f33d803e550 0x7f33d8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient: get_auth_request con 0x7f33f802e5d0 auth_method 0 2024-09-17T16:41:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(544..544 src has 1..544) ==== 6705+0+0 (secure 0 0 0) 0x7f33e8019020 con 0x7f33f80a2f40 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 client.54773.objecter ms_dispatch 0x7f33f8000ba0 osd_map(544..544 src has 1..544) 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 3 client.54773.objecter handle_osd_map got epochs [544,544] > 0 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 3 client.54773.objecter handle_osd_map decoding full epoch 544 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 20 client.54773.objecter dump_active .. 0 homeless 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f33d803e550 0x7f33d8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f6d9d640 10 monclient: get_auth_request con 0x7f33d803e550 auth_method 0 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 10 monclient(hunting): handle_auth_done global_id 64573 payload 824 2024-09-17T16:41:51.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33f802e9b0 con 0x7f33f802e5d0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 client.54773.objecter ms_handle_connect 0x7f33f802e5d0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 client.54773.objecter resend_mon_ops 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f659c640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 0x7f33f8063e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33ec011830 tx=0x7f33ec011cf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 client.54773.objecter ms_handle_connect 0x7f33d803e550 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33f6d9d640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f33d803e550 0x7f33d8040a10 secure :-1 s=READY pgs=3610 cs=0 l=1 rev1=1 crypto rx=0x7f33f00ad000 tx=0x7f33f00a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33ec019070 con 0x7f33f802e5d0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 monclient: dump: 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.160 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f33ec00f830 con 0x7f33f802e5d0 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:41:51.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:41:51.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 msgr2=0x7f33f8063e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 0x7f33f8063e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33ec011830 tx=0x7f33ec011cf0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 msgr2=0x7f33f8063e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:51.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.132+0000 7f33e6ffd640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f802e5d0 0x7f33f8063e00 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33ec011830 tx=0x7f33ec011cf0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.136+0000 7f33fcac8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.136+0000 7f33fcac8640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 0x7f33f81db490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f33f813dbd0 con 0x7f33f8064340 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 0x7f33f81db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient: get_auth_request con 0x7f33f8064340 auth_method 0 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:41:51.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 10 monclient(hunting): handle_auth_done global_id 64582 payload 824 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 1 -- 172.21.15.105:0/3685425214 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f33f80a2a00 con 0x7f33f8064340 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f659c640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 0x7f33f81db490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33ec008010 tx=0x7f33ec004740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 client.54773.objecter ms_handle_connect 0x7f33f8064340 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 client.54773.objecter resend_mon_ops 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33ec00f040 con 0x7f33f8064340 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 monclient: dump: 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:41:51.165 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9454 (secure 0 0 0) 0x7f33ec0143e0 con 0x7f33f8064340 2024-09-17T16:41:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 msgr2=0x7f33f81db490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 0x7f33f81db490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f33ec008010 tx=0x7f33ec004740 comp rx=0 tx=0).stop 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 msgr2=0x7f33f81db490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33e6ffd640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33f8064340 0x7f33f81db490 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 84, 2024-09-17T16:41:51.167 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 284245, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:51.168 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.169 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:51.170 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:41:51.171 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:41:51.172 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:41:51.173 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:41:51.174 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.176 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:41:51.178 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:41:51.179 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.180 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.181 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.182 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.183 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:41:51.184 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.185 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:41:51.186 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:41:51.187 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.188 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:51.189 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:41:51.190 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:41:51.191 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:41:51.191 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:41:51.191 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:41:51.191 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:41:51.191 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 10 librados: watch_flush enter 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 10 librados: watch_flush exit 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 20 client.54773.objecter shutdown clearing up homeless session... 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 10 client.54773.objecter successfully canceled tick 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f33d803e550 msgr2=0x7f33d8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f33d803e550 0x7f33d8040a10 secure :-1 s=READY pgs=3610 cs=0 l=1 rev1=1 crypto rx=0x7f33f00ad000 tx=0x7f33f00a6000 comp rx=0 tx=0).stop 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 10 monclient: shutdown 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f80a2f40 msgr2=0x7f33f809f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 --2- 172.21.15.105:0/3685425214 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33f80a2f40 0x7f33f809f6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33e8007e10 tx=0x7f33e800e4b0 comp rx=0 tx=0).stop 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f6d9d640 1 -- 172.21.15.105:0/3685425214 reap_dead start 2024-09-17T16:41:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33f6d9d640 1 -- 172.21.15.105:0/3685425214 reap_dead start 2024-09-17T16:41:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 shutdown_connections 2024-09-17T16:41:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 >> 172.21.15.105:0/3685425214 conn(0x7f33f8068ba0 msgr2=0x7f33f806f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:41:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 shutdown_connections 2024-09-17T16:41:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 -- 172.21.15.105:0/3685425214 wait complete. 2024-09-17T16:41:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:41:51.140+0000 7f33fcac8640 1 librados: shutdown 2024-09-17T16:41:51.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:41:51.194 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['d'] 2024-09-17T16:41:51.194 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['g'] 2024-09-17T16:41:51.195 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-17T16:41:51.195 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-17T16:41:51.195 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-17T16:41:51.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:41:51.216 INFO:tasks.ceph.mon.d:Stopped 2024-09-17T16:41:51.216 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-17T16:41:51.217 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-17T16:41:51.217 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:42:06.218 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-17T16:42:06.218 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-17T16:42:06.218 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-17T16:42:06.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: get_monmap_and_config 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: build_initial_monmap 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: monmap: 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:06.342222+0000 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:06.342222+0000 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:06.347 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 Processor -- start 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 -- start start 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: init 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient: _add_conns ranks=[1,6,10,3,2,4,5,9,7,0,8] 2024-09-17T16:42:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe3b80a2de0 addr v1:172.21.15.160:6789/0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-d con 0x7fe3b80a38d0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-f con 0x7fe3b813ced0 addr v2:172.21.15.160:3300/0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80a20b0 con 0x7fe3b80a2de0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b809c9a0 con 0x7fe3b813ced0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe3b80a2770 con 0x7fe3b80a38d0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.335+0000 7fe3bf5b2640 10 monclient(hunting): authenticate will time out at 1809.607997s 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bcb4d640 1 -- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 msgr2=0x7fe3b813c990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bcb4d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:54754/0 (socket says 172.21.15.105:54754) 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 learned_addr learned my addr 172.21.15.105:0/4122953499 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe3b813ced0 auth_method 0 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.339+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:06.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe3b813ced0 auth_method 0 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 msgr2=0x7fe3b813c990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:06.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:42:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 msgr2=0x7fe3b813c990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:42:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe3b813ced0 auth_method 0 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:06.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:06.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 msgr2=0x7fe3b813c990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe3b813ced0 auth_method 0 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:42:07.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:07.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:07.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:07.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:07.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): tick 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 monclient(hunting): continuing hunt 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 msgr2=0x7fe3b813c990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:09.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe3b80a38d0 0x7fe3b813c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 msgr2=0x7fe3b813f2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.160:3300/0 conn(0x7fe3b813ced0 0x7fe3b813f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> v1:172.21.15.160:6789/0 conn(0x7fe3b80a2de0 legacy=0x7fe3b80a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient: _add_conns ranks=[8,5,4,9,0,7,2,3] 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-e con 0x7fe39c003830 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-h con 0x7fe39c006320 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-c con 0x7fe39c008ea0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:09.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe3b809c9a0 con 0x7fe39c008ea0 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe3b80a2770 con 0x7fe39c006320 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe39c00ba90 con 0x7fe39c003830 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3aaffd640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request con 0x7fe39c003830 auth_method 0 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe39c008ea0 auth_method 0 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request con 0x7fe39c006320 auth_method 0 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:09.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:09.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.339+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe39c008ea0 auth_method 0 2024-09-17T16:42:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request con 0x7fe39c003830 auth_method 0 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request con 0x7fe39c006320 auth_method 0 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.543+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request con 0x7fe39c003830 auth_method 0 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe39c008ea0 auth_method 0 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request con 0x7fe39c006320 auth_method 0 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:09.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:09.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:09.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:42:09.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:09.947+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request con 0x7fe39c003830 auth_method 0 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe39c008ea0 auth_method 0 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:10.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request con 0x7fe39c006320 auth_method 0 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:42:10.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:10.751+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:42:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): tick 2024-09-17T16:42:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:42:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 monclient(hunting): continuing hunt 2024-09-17T16:42:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:42:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 msgr2=0x7fe39c005c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe39c003830 0x7fe39c005c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 msgr2=0x7fe39c0087b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe39c006320 0x7fe39c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 msgr2=0x7fe39c00b310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe39c008ea0 0x7fe39c00b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient: _add_conns ranks=[9,7,0,2,3] 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.105:3300/0 conn(0x7fe39c010500 0x7fe39c0128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-a con 0x7fe39c010500 addr v2:172.21.15.105:3300/0 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 0x7fe39c0153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-i con 0x7fe39c012f00 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:42:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe39c015a80 addr v1:172.21.15.105:6789/0 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80a24f0 con 0x7fe39c015a80 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b809c9a0 con 0x7fe39c010500 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 1 -- 172.21.15.105:0/4122953499 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe3b80a2770 con 0x7fe39c012f00 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3aaffd640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 reap_dead start 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bdb4f640 1 -- 172.21.15.105:0/4122953499 reap_dead start 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 0x7fe39c0153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request con 0x7fe39c012f00 auth_method 0 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient(hunting): handle_auth_done global_id 64591 payload 274 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> v2:172.21.15.105:3300/0 conn(0x7fe39c010500 msgr2=0x7fe39c0128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.105:3300/0 conn(0x7fe39c010500 0x7fe39c0128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 >> v1:172.21.15.105:6789/0 conn(0x7fe39c015a80 legacy=0x7fe39c017f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 monclient: found mon.noname-i 2024-09-17T16:42:12.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:42:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.339+0000 7fe3bcb4d640 1 -- 172.21.15.105:0/4122953499 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c00bfd0 con 0x7fe39c012f00 2024-09-17T16:42:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 0x7fe39c0153c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac003990 tx=0x7fe3ac01d550 comp rx=0 tx=0).ready entity=mon.7 client_cookie=466451532e9e49cb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 1 -- 172.21.15.105:0/4122953499 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3ac0b3070 con 0x7fe39c012f00 2024-09-17T16:42:12.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: dump: 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.362 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 1 -- 172.21.15.105:0/4122953499 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ac0ac780 con 0x7fe39c012f00 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 5 monclient: authenticate success, global_id 64591 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 1 -- 172.21.15.105:0/4122953499 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3ac0ace60 con 0x7fe39c012f00 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3abfff640 10 monclient: dump: 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 4 set_mon_vals no callback set 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 10 monclient: shutdown 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 msgr2=0x7fe39c0153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 0x7fe39c0153c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac003990 tx=0x7fe3ac01d550 comp rx=0 tx=0).stop 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/4122953499 shutdown_connections 2024-09-17T16:42:12.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/4122953499 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe39c012f00 0x7fe39c0153c0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac003990 tx=0x7fe3ac01d550 comp rx=0 tx=0).stop 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/4122953499 >> v2:172.21.15.105:3300/0 conn(0x7fe39c010500 0x7fe39c0128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.351+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/4122953499 >> 172.21.15.105:0/4122953499 conn(0x7fe3b80a83e0 msgr2=0x7fe3b80a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/4122953499 shutdown_connections 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/4122953499 wait complete. 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient: build_initial_monmap 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient: monmap: 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:12.367 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:12.368 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 librados: starting msgr at 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 librados: starting objecter 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 Processor -- start 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- start start 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 librados: setting wanted keys 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 librados: calling monclient init 2024-09-17T16:42:12.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient: init 2024-09-17T16:42:12.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:12.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient: _add_conns ranks=[2,6,1,4,7,0,8,3,5] 2024-09-17T16:42:12.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-b con 0x7fe39c010500 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 0x7fe3b81cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-d con 0x7fe39c015a80 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 0x7fe3b812d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): picked mon.noname-f con 0x7fe3b81cdd50 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80a41d0 con 0x7fe3b81cdd50 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80a18f0 con 0x7fe39c010500 2024-09-17T16:42:12.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe3b81319a0 con 0x7fe39c015a80 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.355+0000 7fe3bf5b2640 10 monclient(hunting): authenticate will time out at 1815.627894s 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bcb4d640 1 -- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 msgr2=0x7fe3b81cd810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bcb4d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 0x7fe3b81cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58276/0 (socket says 172.21.15.105:58276) 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 -- 172.21.15.105:0/2018142302 learned_addr learned my addr 172.21.15.105:0/2018142302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request con 0x7fe39c010500 auth_method 0 2024-09-17T16:42:12.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 0x7fe3b812d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request con 0x7fe3b81cdd50 auth_method 0 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient(hunting): handle_auth_done global_id 84269 payload 824 2024-09-17T16:42:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 -- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 msgr2=0x7fe3b81cd810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 0x7fe3b81cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 -- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 msgr2=0x7fe3b812d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 0x7fe3b812d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 monclient: found mon.noname-b 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 -- 172.21.15.105:0/2018142302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b809d610 con 0x7fe39c010500 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bdb4f640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 0x7fe3b812d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bd34e640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003f20 tx=0x7fe3b4001670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 client.?.objecter ms_handle_connect 0x7fe39c010500 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b401a020 con 0x7fe39c010500 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: dump: 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.376 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b4016ad0 con 0x7fe39c010500 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bf5b2640 5 monclient: authenticate success, global_id 84269 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bf5b2640 10 monclient: _renew_subs 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bf5b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b809dad0 con 0x7fe39c010500 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b4016c80 con 0x7fe39c010500 2024-09-17T16:42:12.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3ab7fe640 4 set_mon_vals no callback set 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.359+0000 7fe3a9ffb640 10 monclient: dump: 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.379 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 client.84269.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 monclient: _renew_subs 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b812f7c0 con 0x7fe39c010500 2024-09-17T16:42:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 1 librados: init done 2024-09-17T16:42:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b809b860 con 0x7fe39c010500 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe3b401d020 con 0x7fe39c010500 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 0x7fe39c04b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(547..547 src has 1..547) ==== 6718+0+0 (secure 0 0 0) 0x7fe3b4010040 con 0x7fe39c010500 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 10 client.84269.objecter ms_dispatch 0x7fe3b8000ba0 osd_map(547..547 src has 1..547) 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 3 client.84269.objecter handle_osd_map got epochs [547,547] > 0 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 3 client.84269.objecter handle_osd_map decoding full epoch 547 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 20 client.84269.objecter dump_active .. 0 homeless 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 0x7fe39c04b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bcb4d640 10 monclient: get_auth_request con 0x7fe39c0490c0 auth_method 0 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3a9ffb640 10 client.84269.objecter ms_handle_connect 0x7fe39c0490c0 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.363+0000 7fe3bcb4d640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 0x7fe39c04b580 secure :-1 s=READY pgs=3612 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0b0000 tx=0x7fe3ac0a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.371+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3b4024030 con 0x7fe39c010500 2024-09-17T16:42:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.371+0000 7fe3a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.371+0000 7fe3a9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3bf5b2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:42:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3bf5b2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:42:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3bf5b2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe3b80a4350 con 0x7fe39c010500 2024-09-17T16:42:12.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3a9ffb640 1 -- 172.21.15.105:0/2018142302 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3206 (secure 0 0 0) 0x7fe3b809d610 con 0x7fe39c010500 2024-09-17T16:42:12.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3a9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:42:12.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.611+0000 7fe3a9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:12.623 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":90,"quorum":[0,1,2,3,4,5,7,8],"quorum_names":["a","f","b","g","c","h","i","e"],"quorum_leader_name":"e","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 10 librados: watch_flush enter 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 10 librados: watch_flush exit 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 20 client.84269.objecter shutdown clearing up homeless session... 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 10 client.84269.objecter successfully canceled tick 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 msgr2=0x7fe39c04b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 0x7fe39c04b580 secure :-1 s=READY pgs=3612 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0b0000 tx=0x7fe3ac0a7000 comp rx=0 tx=0).stop 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 10 monclient: shutdown 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 msgr2=0x7fe3b81cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003f20 tx=0x7fe3b4001670 comp rx=0 tx=0).stop 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 shutdown_connections 2024-09-17T16:42:12.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe39c0490c0 0x7fe39c04b580 unknown :-1 s=CLOSED pgs=3612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3b81cdd50 0x7fe3b812d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe39c015a80 0x7fe3b81cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 --2- 172.21.15.105:0/2018142302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe39c010500 0x7fe3b81cd2d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 >> 172.21.15.105:0/2018142302 conn(0x7fe3b80a83e0 msgr2=0x7fe3b80a61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 shutdown_connections 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 -- 172.21.15.105:0/2018142302 wait complete. 2024-09-17T16:42:12.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.615+0000 7fe3bf5b2640 1 librados: shutdown 2024-09-17T16:42:12.642 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-17T16:42:12.643 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-17T16:42:12.643 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:42:12.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: get_monmap_and_config 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: build_initial_monmap 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: monmap: 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:12.759902+0000 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:12.759902+0000 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:12.764 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:12.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 Processor -- start 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 -- start start 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: init 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient: _add_conns ranks=[2,0,9,3,1,6,7,5,10,8,4] 2024-09-17T16:42:12.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 0x7fb4580a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): picked mon.noname-b con 0x7fb4580a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb4580a4ab0 addr v1:172.21.15.105:6789/0 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): picked mon.noname-a con 0x7fb4580a7350 addr v2:172.21.15.105:3300/0 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4580a4080 con 0x7fb4580a4ab0 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb4580a7c40 con 0x7fb4580a7350 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb4580a37e0 con 0x7fb4580a6a30 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb460716640 10 monclient(hunting): authenticate will time out at 1816.025779s 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 0x7fb4580a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45dcb1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb4580a4ab0 0x7fb4581397f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53364/0 (socket says 172.21.15.105:53364) 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45dcb1640 1 -- 172.21.15.105:0/1145699127 learned_addr learned my addr 172.21.15.105:0/1145699127 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): get_auth_request con 0x7fb4580a6a30 auth_method 0 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 10 monclient(hunting): get_auth_request con 0x7fb4580a7350 auth_method 0 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 1 -- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 msgr2=0x7fb458139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 1 -- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 msgr2=0x7fb458139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_done global_id 84278 payload 274 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 -- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 msgr2=0x7fb458139f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 -- 172.21.15.105:0/1145699127 >> v1:172.21.15.105:6789/0 conn(0x7fb4580a4ab0 legacy=0x7fb4581397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 monclient: found mon.noname-b 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45e4b2640 1 -- 172.21.15.105:0/1145699127 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4580a3ac0 con 0x7fb4580a6a30 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45d4b0640 1 -- 172.21.15.105:0/1145699127 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300103415 0 0) 0x7fb4580a4080 con 0x7fb4580a4ab0 2024-09-17T16:42:12.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.755+0000 7fb45d4b0640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/1145699127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 0x7fb4580a6e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097c0 tx=0x7fb44801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e8c7fd15a1a1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 1 -- 172.21.15.105:0/1145699127 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4480b1070 con 0x7fb4580a6a30 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: dump: 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 1 -- 172.21.15.105:0/1145699127 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4480aa7b0 con 0x7fb4580a6a30 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 1 -- 172.21.15.105:0/1145699127 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4480aae10 con 0x7fb4580a6a30 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45d4b0640 10 monclient: dump: 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.780 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 5 monclient: authenticate success, global_id 84278 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 4 set_mon_vals no callback set 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: shutdown 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- 172.21.15.105:0/1145699127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 msgr2=0x7fb4580a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- 172.21.15.105:0/1145699127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 0x7fb4580a6e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097c0 tx=0x7fb44801c330 comp rx=0 tx=0).stop 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- 172.21.15.105:0/1145699127 shutdown_connections 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- 172.21.15.105:0/1145699127 >> v2:172.21.15.105:3300/0 conn(0x7fb4580a7350 0x7fb458139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- 172.21.15.105:0/1145699127 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb4580a6a30 0x7fb4580a6e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- 172.21.15.105:0/1145699127 >> 172.21.15.105:0/1145699127 conn(0x7fb4580a8d00 msgr2=0x7fb4580a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- 172.21.15.105:0/1145699127 shutdown_connections 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- 172.21.15.105:0/1145699127 wait complete. 2024-09-17T16:42:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: build_initial_monmap 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: monmap: 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 librados: starting msgr at 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 librados: starting objecter 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 Processor -- start 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- start start 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 librados: setting wanted keys 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 librados: calling monclient init 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: init 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient: _add_conns ranks=[5,4,3,7,2,0,1,8,6] 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb4580a4ab0 0x7fb45813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): picked mon.noname-h con 0x7fb4580a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 0x7fb45813e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): picked mon.noname-c con 0x7fb4580a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4580a7350 0x7fb45813e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): picked mon.noname-g con 0x7fb4580a7350 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fb45813a440 con 0x7fb4580a7350 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb45806e770 con 0x7fb4580a6a30 2024-09-17T16:42:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb45813a5c0 con 0x7fb4580a4ab0 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb460716640 10 monclient(hunting): authenticate will time out at 1816.031221s 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 0x7fb45813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 0x7fb45813e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:49582/0 (socket says 172.21.15.105:49582) 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 -- 172.21.15.105:0/2662445873 learned_addr learned my addr 172.21.15.105:0/2662445873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): get_auth_request con 0x7fb4580a6a30 auth_method 0 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb4580a4ab0 0x7fb45813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4580a7350 0x7fb45813e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient(hunting): handle_auth_done global_id 74560 payload 824 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb4580a4ab0 msgr2=0x7fb45813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb4580a4ab0 0x7fb45813ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4580a7350 msgr2=0x7fb45813e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4580a7350 0x7fb45813e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 monclient: found mon.noname-c 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581237a0 con 0x7fb4580a6a30 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45e4b2640 10 monclient: get_auth_request con 0x7fb4580a4ab0 auth_method 0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb4580a4ab0 0x7fb45813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45ecb3640 10 monclient: get_auth_request con 0x7fb4580a7350 auth_method 0 2024-09-17T16:42:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4580a7350 0x7fb45813e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb45dcb1640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 0x7fb45813e410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c420 tx=0x7fb45400c8e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb44effd640 10 client.?.objecter ms_handle_connect 0x7fb4580a6a30 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb44effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.759+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4540047f0 con 0x7fb4580a6a30 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: dump: 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb454004990 con 0x7fb4580a6a30 2024-09-17T16:42:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 5 monclient: authenticate success, global_id 74560 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 monclient: _renew_subs 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4580a23c0 con 0x7fb4580a6a30 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4540133c0 con 0x7fb4580a6a30 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45ccaf640 4 set_mon_vals no callback set 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 client.74560.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: dump: 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.791 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 monclient: _renew_subs 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45813ee90 con 0x7fb4580a6a30 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 1 librados: init done 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 0x7fb4580a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4581401c0 con 0x7fb45813f0c0 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb454011070 con 0x7fb4580a6a30 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 0x7fb4580a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient: get_auth_request con 0x7fb45813f0c0 auth_method 0 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb43c03e550 0x7fb43c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(547..547 src has 1..547) ==== 6718+0+0 (secure 0 0 0) 0x7fb45404fc70 con 0x7fb4580a6a30 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 client.74560.objecter ms_dispatch 0x7fb458000ba0 osd_map(547..547 src has 1..547) 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_done global_id 36396 payload 824 2024-09-17T16:42:12.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 3 client.74560.objecter handle_osd_map got epochs [547,547] > 0 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 3 client.74560.objecter handle_osd_map decoding full epoch 547 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb45806de60 con 0x7fb45813f0c0 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 20 client.74560.objecter dump_active .. 0 homeless 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 client.74560.objecter ms_handle_connect 0x7fb45813f0c0 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 client.74560.objecter resend_mon_ops 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 0x7fb4580a0a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb45806e6e0 tx=0x7fb4480190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb43c03e550 0x7fb43c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4480ba030 con 0x7fb45813f0c0 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: dump: 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.795 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45ecb3640 10 monclient: get_auth_request con 0x7fb43c03e550 auth_method 0 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb448003620 con 0x7fb45813f0c0 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 msgr2=0x7fb4580a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 0x7fb4580a0a20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb45806e6e0 tx=0x7fb4480190d0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 msgr2=0x7fb4580a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45813f0c0 0x7fb4580a0a20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb44effd640 10 client.74560.objecter ms_handle_connect 0x7fb43c03e550 2024-09-17T16:42:12.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.763+0000 7fb45ecb3640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb43c03e550 0x7fb43c040a10 secure :-1 s=READY pgs=3613 cs=0 l=1 rev1=1 crypto rx=0x7fb450009610 tx=0x7fb45000b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 0x7fb4580a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580a1e70 con 0x7fb4580a0f60 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 0x7fb4580a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient: get_auth_request con 0x7fb4580a0f60 auth_method 0 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 10 monclient(hunting): handle_auth_done global_id 36405 payload 824 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 1 -- 172.21.15.105:0/2662445873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb4580a6e10 con 0x7fb4580a0f60 2024-09-17T16:42:12.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 client.74560.objecter ms_handle_connect 0x7fb4580a0f60 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45e4b2640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 0x7fb4580a1370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb448019f60 tx=0x7fb4480a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 client.74560.objecter resend_mon_ops 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4480ba040 con 0x7fb4580a0f60 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 monclient: dump: 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.800 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7fb4480b6480 con 0x7fb4580a0f60 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 msgr2=0x7fb4580a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 0x7fb4580a1370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb448019f60 tx=0x7fb4480a6000 comp rx=0 tx=0).stop 2024-09-17T16:42:12.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 msgr2=0x7fb4580a1370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb44effd640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4580a0f60 0x7fb4580a1370 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb448019f60 tx=0x7fb4480a6000 comp rx=0 tx=0).stop 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 723575, 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:12.802 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.803 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.804 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:12.805 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:12.806 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:12.806 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:12.808 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:12.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.810 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.812 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:12.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:12.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.817 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.818 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.820 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:12.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:12.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:12.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.824 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.825 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.826 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:12.826 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 librados: watch_flush enter 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 librados: watch_flush exit 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 20 client.74560.objecter shutdown clearing up homeless session... 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 client.74560.objecter successfully canceled tick 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb43c03e550 msgr2=0x7fb43c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb43c03e550 0x7fb43c040a10 secure :-1 s=READY pgs=3613 cs=0 l=1 rev1=1 crypto rx=0x7fb450009610 tx=0x7fb45000b040 comp rx=0 tx=0).stop 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 10 monclient: shutdown 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 msgr2=0x7fb45813e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 --2- 172.21.15.105:0/2662445873 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb4580a6a30 0x7fb45813e410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c420 tx=0x7fb45400c8e0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45ecb3640 1 -- 172.21.15.105:0/2662445873 reap_dead start 2024-09-17T16:42:12.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb45ecb3640 1 -- 172.21.15.105:0/2662445873 reap_dead start 2024-09-17T16:42:12.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 shutdown_connections 2024-09-17T16:42:12.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 >> 172.21.15.105:0/2662445873 conn(0x7fb4580a8d00 msgr2=0x7fb4580aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:12.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 shutdown_connections 2024-09-17T16:42:12.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 -- 172.21.15.105:0/2662445873 wait complete. 2024-09-17T16:42:12.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.771+0000 7fb460716640 1 librados: shutdown 2024-09-17T16:42:12.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:12.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-17T16:42:12.829 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: get_monmap_and_config 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: build_initial_monmap 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: monmap: 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.944 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:12.940556+0000 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:12.940556+0000 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:12.945 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:12.946 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 Processor -- start 2024-09-17T16:42:12.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 -- start start 2024-09-17T16:42:12.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: init 2024-09-17T16:42:12.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient: _add_conns ranks=[5,4,0,8,10,3,1,9,7,2,6] 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a68e0 0x7f02980a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-h con 0x7f02980a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 0x7f02980a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-c con 0x7f02980a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02981396a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029806ebc0 con 0x7f02981396a0 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f02980a3fc0 con 0x7f02980a4910 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f02980a63c0 con 0x7f02980a68e0 2024-09-17T16:42:12.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029ed22640 10 monclient(hunting): authenticate will time out at 1816.206340s 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029d2bf640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f02981396a0 0x7f0298139ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53374/0 (socket says 172.21.15.105:53374) 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029d2bf640 1 -- 172.21.15.105:0/994356714 learned_addr learned my addr 172.21.15.105:0/994356714 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 0x7f02980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): get_auth_request con 0x7f02980a4910 auth_method 0 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029cabe640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a68e0 0x7f02980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029cabe640 10 monclient(hunting): get_auth_request con 0x7f02980a68e0 auth_method 0 2024-09-17T16:42:12.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029cabe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029cabe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f029cabe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 1 -- 172.21.15.105:0/994356714 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472693468 0 0) 0x7f029806ebc0 con 0x7f02981396a0 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 10 monclient(hunting): _negotiate 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 10 monclient(hunting): my global_id is 36414 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f02977fe640 1 -- 172.21.15.105:0/994356714 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0280003530 con 0x7f02981396a0 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient(hunting): handle_auth_done global_id 74569 payload 274 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 -- 172.21.15.105:0/994356714 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a68e0 msgr2=0x7f02980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a68e0 0x7f02980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 -- 172.21.15.105:0/994356714 >> v1:172.21.15.105:6789/0 conn(0x7f02981396a0 legacy=0x7f0298139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 monclient: found mon.noname-c 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.935+0000 7f0297fff640 1 -- 172.21.15.105:0/994356714 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02980a3690 con 0x7f02980a4910 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 0x7f02980a7200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f028c0097f0 tx=0x7f028c01c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f5896743c99373a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 1 -- 172.21.15.105:0/994356714 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f028c0b0070 con 0x7f02980a4910 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: dump: 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.958 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 1 -- 172.21.15.105:0/994356714 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0aa810 con 0x7f02980a4910 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 5 monclient: authenticate success, global_id 74569 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 1 -- 172.21.15.105:0/994356714 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f028c0a9300 con 0x7f02980a4910 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 4 set_mon_vals no callback set 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f02977fe640 10 monclient: dump: 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.960 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: shutdown 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:12.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- 172.21.15.105:0/994356714 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 msgr2=0x7f02980a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 0x7f02980a7200 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f028c0097f0 tx=0x7f028c01c3a0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- 172.21.15.105:0/994356714 shutdown_connections 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a4910 0x7f02980a7200 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- 172.21.15.105:0/994356714 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a68e0 0x7f02980a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- 172.21.15.105:0/994356714 >> 172.21.15.105:0/994356714 conn(0x7f0298068ba0 msgr2=0x7f0298068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- 172.21.15.105:0/994356714 shutdown_connections 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- 172.21.15.105:0/994356714 wait complete. 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: build_initial_monmap 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: monmap: 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:12.962 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 librados: starting msgr at 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 librados: starting objecter 2024-09-17T16:42:12.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 Processor -- start 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- start start 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 librados: setting wanted keys 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 librados: calling monclient init 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: init 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient: _add_conns ranks=[5,4,7,8,2,6,0,1,3] 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a4910 0x7f02981d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-h con 0x7f02980a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 0x7f02981d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-c con 0x7f02980a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f02981396a0 0x7f029813d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): picked mon.noname-i con 0x7f02981396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:42:12.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f02981444b0 con 0x7f02980a68e0 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0298144330 con 0x7f02980a4910 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0298144630 con 0x7f02981396a0 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029ed22640 10 monclient(hunting): authenticate will time out at 1816.211421s 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 0x7f02981d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 0x7f02981d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:49604/0 (socket says 172.21.15.105:49604) 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 -- 172.21.15.105:0/3083561717 learned_addr learned my addr 172.21.15.105:0/3083561717 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): get_auth_request con 0x7f02980a68e0 auth_method 0 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient(hunting): handle_auth_done global_id 74578 payload 824 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f02981396a0 msgr2=0x7f029813d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f029cabe640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a4910 0x7f02981d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f02981396a0 0x7f029813d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a4910 msgr2=0x7f02981d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f02980a4910 0x7f02981d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient: _finish_hunting 0 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 monclient: found mon.noname-c 2024-09-17T16:42:12.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.939+0000 7f0297fff640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0298122af0 con 0x7f02980a68e0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f0297fff640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 0x7f02981d1d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f028c009710 tx=0x7f028c019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.?.objecter ms_handle_connect 0x7f02980a68e0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f028c0b0040 con 0x7f02980a68e0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: dump: 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: _finish_auth 0 2024-09-17T16:42:12.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: _check_auth_tickets 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0044b0 con 0x7f02980a68e0 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 5 monclient: authenticate success, global_id 74578 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 monclient: _renew_subs 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0298140b30 con 0x7f02980a68e0 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f028c0aaca0 con 0x7f02980a68e0 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f0296ffd640 4 set_mon_vals no callback set 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 client.74578.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: dump: 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.969 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.970 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 monclient: _renew_subs 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f02980a9de0 con 0x7f02980a68e0 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 1 librados: init done 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 0x7f029813df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0298051e80 con 0x7f0298122530 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f028c004040 con 0x7f02980a68e0 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 0x7f029813df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient: get_auth_request con 0x7f0298122530 auth_method 0 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f028403e540 0x7f0284040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(547..547 src has 1..547) ==== 6718+0+0 (secure 0 0 0) 0x7f028c0e9ca0 con 0x7f02980a68e0 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.74578.objecter ms_dispatch 0x7f0298000ba0 osd_map(547..547 src has 1..547) 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 3 client.74578.objecter handle_osd_map got epochs [547,547] > 0 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 3 client.74578.objecter handle_osd_map decoding full epoch 547 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 10 monclient(hunting): handle_auth_done global_id 84287 payload 824 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 20 client.74578.objecter dump_active .. 0 homeless 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0298122910 con 0x7f0298122530 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029d2bf640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f028403e540 0x7f0284040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029d2bf640 10 monclient: get_auth_request con 0x7f028403e540 auth_method 0 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.74578.objecter ms_handle_connect 0x7f0298122530 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.74578.objecter resend_mon_ops 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029cabe640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 0x7f029813df50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f028800e9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0288017070 con 0x7f0298122530 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: dump: 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.973 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f028801c450 con 0x7f0298122530 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 msgr2=0x7f029813df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 0x7f029813df50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f028800e9f0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 msgr2=0x7f029813df50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0298122530 0x7f029813df50 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f028800e9f0 comp rx=0 tx=0).stop 2024-09-17T16:42:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f02957fa640 10 client.74578.objecter ms_handle_connect 0x7f028403e540 2024-09-17T16:42:12.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.943+0000 7f029d2bf640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f028403e540 0x7f0284040a00 secure :-1 s=READY pgs=3614 cs=0 l=1 rev1=1 crypto rx=0x7f02900015b0 tx=0x7f0290001600 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 0x7f029802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0298139ae0 con 0x7f029802d450 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 0x7f029802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient: get_auth_request con 0x7f029802d450 auth_method 0 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 10 monclient(hunting): handle_auth_done global_id 84296 payload 824 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 1 -- 172.21.15.105:0/3083561717 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f02980a1120 con 0x7f029802d450 2024-09-17T16:42:12.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029cabe640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 0x7f029802d830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f0288004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 client.74578.objecter ms_handle_connect 0x7f029802d450 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 client.74578.objecter resend_mon_ops 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f028800f040 con 0x7f029802d450 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 monclient: dump: 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:12.978 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7f0288013d30 con 0x7f029802d450 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 msgr2=0x7f029802d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 0x7f029802d830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f0288004740 comp rx=0 tx=0).stop 2024-09-17T16:42:12.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 msgr2=0x7f029802d830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f02957fa640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f029802d450 0x7f029802d830 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f028800e530 tx=0x7f0288004740 comp rx=0 tx=0).stop 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 40187, 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.981 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.982 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:12.983 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:12.984 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.985 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:12.986 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.987 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:12.988 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.989 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.990 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.993 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.998 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:12.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.000 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.001 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 librados: watch_flush enter 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 librados: watch_flush exit 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 20 client.74578.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 client.74578.objecter successfully canceled tick 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f028403e540 msgr2=0x7f0284040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f028403e540 0x7f0284040a00 secure :-1 s=READY pgs=3614 cs=0 l=1 rev1=1 crypto rx=0x7f02900015b0 tx=0x7f0290001600 comp rx=0 tx=0).stop 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 10 monclient: shutdown 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 msgr2=0x7f02981d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 --2- 172.21.15.105:0/3083561717 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f02980a68e0 0x7f02981d1d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f028c009710 tx=0x7f028c019140 comp rx=0 tx=0).stop 2024-09-17T16:42:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029d2bf640 1 -- 172.21.15.105:0/3083561717 reap_dead start 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029d2bf640 1 -- 172.21.15.105:0/3083561717 reap_dead start 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 shutdown_connections 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 >> 172.21.15.105:0/3083561717 conn(0x7f0298068ba0 msgr2=0x7f02980a2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 shutdown_connections 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 -- 172.21.15.105:0/3083561717 wait complete. 2024-09-17T16:42:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:12.951+0000 7f029ed22640 1 librados: shutdown 2024-09-17T16:42:13.004 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.004 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-17T16:42:13.005 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:42:13.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: get_monmap_and_config 2024-09-17T16:42:13.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: monmap: 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.119932+0000 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.119932+0000 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.124 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:13.125 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 Processor -- start 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 -- start start 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: init 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient: _add_conns ranks=[4,0,5,7,2,1,9,3,10,8,6] 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-c con 0x7fb910139d20 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb9100a4d90 addr v1:172.21.15.105:6789/0 2024-09-17T16:42:13.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb9100a58a0 0x7fb9100a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-h con 0x7fb9100a58a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9100745d0 con 0x7fb9100a4d90 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb9100a4110 con 0x7fb910139d20 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb9100a42e0 con 0x7fb9100a58a0 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb91868d640 10 monclient(hunting): authenticate will time out at 1816.385695s 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:49608/0 (socket says 172.21.15.105:49608) 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 -- 172.21.15.105:0/1087824854 learned_addr learned my addr 172.21.15.105:0/1087824854 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): get_auth_request con 0x7fb910139d20 auth_method 0 2024-09-17T16:42:13.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient(hunting): handle_auth_done global_id 74587 payload 274 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 -- 172.21.15.105:0/1087824854 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb9100a58a0 msgr2=0x7fb9100a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 --2- 172.21.15.105:0/1087824854 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb9100a58a0 0x7fb9100a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 -- 172.21.15.105:0/1087824854 >> v1:172.21.15.105:6789/0 conn(0x7fb9100a4d90 legacy=0x7fb9100a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 monclient: found mon.noname-c 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:42:13.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.115+0000 7fb916429640 1 -- 172.21.15.105:0/1087824854 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91006ead0 con 0x7fb910139d20 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb916429640 1 --2- 172.21.15.105:0/1087824854 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c01c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=32b38f028a165587 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 1 -- 172.21.15.105:0/1087824854 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c0b0070 con 0x7fb910139d20 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: dump: 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.131 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 5 monclient: authenticate success, global_id 74587 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 1 -- 172.21.15.105:0/1087824854 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c0aa810 con 0x7fb910139d20 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 4 set_mon_vals no callback set 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 1 -- 172.21.15.105:0/1087824854 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c0a9300 con 0x7fb910139d20 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb915427640 10 monclient: dump: 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.133 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: shutdown 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- 172.21.15.105:0/1087824854 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 msgr2=0x7fb91013a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- 172.21.15.105:0/1087824854 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c01c440 comp rx=0 tx=0).stop 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- 172.21.15.105:0/1087824854 shutdown_connections 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- 172.21.15.105:0/1087824854 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb9100a58a0 0x7fb9100a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- 172.21.15.105:0/1087824854 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb910139d20 0x7fb91013a100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- 172.21.15.105:0/1087824854 >> 172.21.15.105:0/1087824854 conn(0x7fb9100aa250 msgr2=0x7fb9100aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- 172.21.15.105:0/1087824854 shutdown_connections 2024-09-17T16:42:13.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- 172.21.15.105:0/1087824854 wait complete. 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: monmap: 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.138 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 librados: starting msgr at 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 librados: starting objecter 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 Processor -- start 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 -- start start 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 librados: setting wanted keys 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 librados: calling monclient init 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: init 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient: _add_conns ranks=[2,3,0,1,8,4,5,6,7] 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 0x7fb91013dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-b con 0x7fb9100a4d90 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9100a58a0 0x7fb9101d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-g con 0x7fb9100a58a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.119+0000 7fb91868d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb910139d20 0x7fb9101d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): picked mon.noname-a con 0x7fb910139d20 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb91013a640 con 0x7fb910139d20 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb91013a7c0 con 0x7fb9100a4d90 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fb910144480 con 0x7fb9100a58a0 2024-09-17T16:42:13.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): authenticate will time out at 1816.392082s 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 0x7fb91013dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 0x7fb91013dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58316/0 (socket says 172.21.15.105:58316) 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 -- 172.21.15.105:0/3741736347 learned_addr learned my addr 172.21.15.105:0/3741736347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): get_auth_request con 0x7fb9100a4d90 auth_method 0 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb910139d20 0x7fb9101d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): get_auth_request con 0x7fb910139d20 auth_method 0 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916c2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient(hunting): handle_auth_done global_id 84305 payload 824 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9100a58a0 msgr2=0x7fb9101d1390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9100a58a0 0x7fb9101d1390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb910139d20 msgr2=0x7fb9101d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb910139d20 0x7fb9101d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 monclient: found mon.noname-b 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91013e470 con 0x7fb9100a4d90 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb916429640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 0x7fb91013dae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097c0 tx=0x7fb90c0191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 client.?.objecter ms_handle_connect 0x7fb9100a4d90 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c0aae60 con 0x7fb9100a4d90 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: dump: 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.143 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c0d2040 con 0x7fb9100a4d90 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 5 monclient: authenticate success, global_id 84305 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient: _renew_subs 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91013f8a0 con 0x7fb9100a4d90 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb90c004250 con 0x7fb9100a4d90 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb907fff640 4 set_mon_vals no callback set 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 monclient: dump: 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.145 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 client.84305.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient: _renew_subs 2024-09-17T16:42:13.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91013f2e0 con 0x7fb9100a4d90 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 librados: init done 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 0x7fb91013b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb9100a0e60 con 0x7fb9101d1e10 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb90c003620 con 0x7fb9100a4d90 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8f403e5b0 0x7fb8f4040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7fb90c10c560 con 0x7fb9100a4d90 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 10 client.84305.objecter ms_dispatch 0x7fb910000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 3 client.84305.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.123+0000 7fb906ffd640 3 client.84305.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb906ffd640 20 client.84305.objecter dump_active .. 0 homeless 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 0x7fb91013b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient: get_auth_request con 0x7fb9101d1e10 auth_method 0 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb916c2a640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8f403e5b0 0x7fb8f4040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb916c2a640 10 monclient: get_auth_request con 0x7fb8f403e5b0 auth_method 0 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 10 monclient(hunting): handle_auth_done global_id 74596 payload 824 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9100aa8c0 con 0x7fb9101d1e10 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb906ffd640 10 client.84305.objecter ms_handle_connect 0x7fb9101d1e10 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb906ffd640 10 client.84305.objecter resend_mon_ops 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb915c28640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 0x7fb91013b5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb91013cd30 tx=0x7fb90000c7a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb906ffd640 10 client.84305.objecter ms_handle_connect 0x7fb8f403e5b0 2024-09-17T16:42:13.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.127+0000 7fb916c2a640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8f403e5b0 0x7fb8f4040a70 secure :-1 s=READY pgs=3615 cs=0 l=1 rev1=1 crypto rx=0x7fb908009f60 tx=0x7fb90800d040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb900017070 con 0x7fb9101d1e10 2024-09-17T16:42:13.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 10 monclient: dump: 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.151 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb9000050d0 con 0x7fb9101d1e10 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 msgr2=0x7fb91013b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 0x7fb91013b5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb91013cd30 tx=0x7fb90000c7a0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 msgr2=0x7fb91013b5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.131+0000 7fb906ffd640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb9101d1e10 0x7fb91013b5d0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb91013cd30 tx=0x7fb90000c7a0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb91868d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:13.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb91868d640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 0x7fb91013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb91868d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb910051e80 con 0x7fb91013bb10 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 0x7fb91013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient: get_auth_request con 0x7fb91013bb10 auth_method 0 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 10 monclient(hunting): handle_auth_done global_id 74605 payload 824 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 1 -- 172.21.15.105:0/3741736347 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb9101d21f0 con 0x7fb91013bb10 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb915c28640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 0x7fb91013bf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb900007fd0 tx=0x7fb900000ee0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb906ffd640 10 client.84305.objecter ms_handle_connect 0x7fb91013bb10 2024-09-17T16:42:13.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.135+0000 7fb906ffd640 10 client.84305.objecter resend_mon_ops 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb900017030 con 0x7fb91013bb10 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 10 monclient: dump: 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.154 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9610 (secure 0 0 0) 0x7fb90001ab40 con 0x7fb91013bb10 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 msgr2=0x7fb91013bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 0x7fb91013bf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb900007fd0 tx=0x7fb900000ee0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 msgr2=0x7fb91013bf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb906ffd640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb91013bb10 0x7fb91013bf20 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb900007fd0 tx=0x7fb900000ee0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 172602, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:13.156 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.157 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.158 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.159 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.160 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.162 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:13.164 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.168 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.169 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.170 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.172 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.175 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:42:13.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.177 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.178 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 10 librados: watch_flush enter 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 10 librados: watch_flush exit 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 20 client.84305.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 10 client.84305.objecter successfully canceled tick 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8f403e5b0 msgr2=0x7fb8f4040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb8f403e5b0 0x7fb8f4040a70 secure :-1 s=READY pgs=3615 cs=0 l=1 rev1=1 crypto rx=0x7fb908009f60 tx=0x7fb90800d040 comp rx=0 tx=0).stop 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 10 monclient: shutdown 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 msgr2=0x7fb91013dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 --2- 172.21.15.105:0/3741736347 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9100a4d90 0x7fb91013dae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097c0 tx=0x7fb90c0191e0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb916c2a640 1 -- 172.21.15.105:0/3741736347 reap_dead start 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb916c2a640 1 -- 172.21.15.105:0/3741736347 reap_dead start 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 shutdown_connections 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 >> 172.21.15.105:0/3741736347 conn(0x7fb9100aa250 msgr2=0x7fb91006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 shutdown_connections 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 -- 172.21.15.105:0/3741736347 wait complete. 2024-09-17T16:42:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.139+0000 7fb91868d640 1 librados: shutdown 2024-09-17T16:42:13.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-17T16:42:13.182 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:42:13.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: get_monmap_and_config 2024-09-17T16:42:13.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: monmap: 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.295880+0000 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.295880+0000 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.301 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.301 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.301 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:13.301 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:13.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 Processor -- start 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 -- start start 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: init 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient: _add_conns ranks=[1,6,8,9,5,2,7,10,3,0,4] 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa1d40a57d0 addr v1:172.21.15.160:6789/0 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 0x7fa1d413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-d con 0x7fa1d40a1d90 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-e con 0x7fa1d40a62c0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d406e6e0 con 0x7fa1d40a57d0 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa1d406db40 con 0x7fa1d40a1d90 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa1d406d8c0 con 0x7fa1d40a62c0 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1db23c640 10 monclient(hunting): authenticate will time out at 1816.561665s 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d3fff640 1 -- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 msgr2=0x7fa1d413e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 0x7fa1d413e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49032/0 (socket says 172.21.15.105:49032) 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 -- 172.21.15.105:0/3643948805 learned_addr learned my addr 172.21.15.105:0/3643948805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): get_auth_request con 0x7fa1d40a62c0 auth_method 0 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_done global_id 54782 payload 274 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 -- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 msgr2=0x7fa1d413e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 --2- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 0x7fa1d413e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 -- 172.21.15.105:0/3643948805 >> v1:172.21.15.160:6789/0 conn(0x7fa1d40a57d0 legacy=0x7fa1d40a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 monclient: found mon.noname-e 2024-09-17T16:42:13.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:42:13.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.291+0000 7fa1d97d9640 1 -- 172.21.15.105:0/3643948805 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d406ebe0 con 0x7fa1d40a62c0 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 1 --2- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40098a0 tx=0x7fa1c401c460 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9083a777a4c2eed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 1 -- 172.21.15.105:0/3643948805 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c40ab8b0 con 0x7fa1d40a62c0 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: dump: 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.313 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 1 -- 172.21.15.105:0/3643948805 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c40b0070 con 0x7fa1d40a62c0 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 1 -- 172.21.15.105:0/3643948805 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c40aa3a0 con 0x7fa1d40a62c0 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 5 monclient: authenticate success, global_id 54782 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 4 set_mon_vals no callback set 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d37fe640 10 monclient: dump: 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.315 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: shutdown 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 msgr2=0x7fa1d4140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40098a0 tx=0x7fa1c401c460 comp rx=0 tx=0).stop 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- 172.21.15.105:0/3643948805 shutdown_connections 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d40a62c0 0x7fa1d4140ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- 172.21.15.105:0/3643948805 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa1d40a1d90 0x7fa1d413e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- 172.21.15.105:0/3643948805 >> 172.21.15.105:0/3643948805 conn(0x7fa1d40aa3c0 msgr2=0x7fa1d40a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- 172.21.15.105:0/3643948805 shutdown_connections 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- 172.21.15.105:0/3643948805 wait complete. 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: monmap: 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:13.317 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 librados: starting msgr at 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 librados: starting objecter 2024-09-17T16:42:13.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 Processor -- start 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- start start 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 librados: setting wanted keys 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 librados: calling monclient init 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: init 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient: _add_conns ranks=[1,7,3,5,6,0,8,4,2] 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1d40a1d90 0x7fa1d413f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-f con 0x7fa1d40a1d90 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa1d40a57d0 0x7fa1d4139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-i con 0x7fa1d40a57d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 0x7fa1d413e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): picked mon.noname-g con 0x7fa1d40a62c0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4143dd0 con 0x7fa1d40a1d90 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d4143c50 con 0x7fa1d40a62c0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa1d4143f50 con 0x7fa1d40a57d0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1db23c640 10 monclient(hunting): authenticate will time out at 1816.567042s 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 0x7fa1d413e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa1d40a57d0 0x7fa1d4139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1d40a1d90 0x7fa1d413f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 0x7fa1d413e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:53152/0 (socket says 172.21.15.105:53152) 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 learned_addr learned my addr 172.21.15.105:0/1234790320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): get_auth_request con 0x7fa1d40a62c0 auth_method 0 2024-09-17T16:42:13.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): get_auth_request con 0x7fa1d40a57d0 auth_method 0 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): get_auth_request con 0x7fa1d40a1d90 auth_method 0 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.295+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 10 monclient(hunting): handle_auth_done global_id 54840 payload 824 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa1d40a57d0 msgr2=0x7fa1d4139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa1d40a57d0 0x7fa1d4139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1d40a1d90 msgr2=0x7fa1d413f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1d40a1d90 0x7fa1d413f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 monclient: found mon.noname-g 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:42:13.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d4140b80 con 0x7fa1d40a62c0 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d97d9640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 0x7fa1d413e0f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1c401ab10 tx=0x7fa1c4019200 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.?.objecter ms_handle_connect 0x7fa1d40a62c0 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c4003dc0 con 0x7fa1d40a62c0 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: dump: 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.323 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 5 monclient: authenticate success, global_id 54840 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 monclient: _renew_subs 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4003950 con 0x7fa1d40a62c0 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d40a3e10 con 0x7fa1d40a62c0 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1c4003b00 con 0x7fa1d40a62c0 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: dump: 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.325 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d2ffd640 4 set_mon_vals no callback set 2024-09-17T16:42:13.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 client.54840.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 monclient: _renew_subs 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d40a0530 con 0x7fa1d40a62c0 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 1 librados: init done 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 0x7fa1d413e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa1d4051e80 con 0x7fa1d41c8de0 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 0x7fa1d413e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient: get_auth_request con 0x7fa1d41c8de0 auth_method 0 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_done global_id 54791 payload 824 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1d41d1d00 con 0x7fa1d41c8de0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.54840.objecter ms_handle_connect 0x7fa1d41c8de0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d8fd8640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 0x7fa1d413e630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007fd0 tx=0x7fa1cc00ff70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.54840.objecter resend_mon_ops 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa1c40bd020 con 0x7fa1d40a62c0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa1b403e550 0x7fa1b4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7fa1c40ea700 con 0x7fa1d40a62c0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.54840.objecter ms_dispatch 0x7fa1d4000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 3 client.54840.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 3 client.54840.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:13.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 20 client.54840.objecter dump_active .. 0 homeless 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1cc00d7c0 con 0x7fa1d41c8de0 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: dump: 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.329 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa1cc018070 con 0x7fa1d41c8de0 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 msgr2=0x7fa1d413e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 0x7fa1d413e630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007fd0 tx=0x7fa1cc00ff70 comp rx=0 tx=0).stop 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 msgr2=0x7fa1d413e630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d41c8de0 0x7fa1d413e630 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007fd0 tx=0x7fa1cc00ff70 comp rx=0 tx=0).stop 2024-09-17T16:42:13.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d3fff640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa1b403e550 0x7fa1b4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d3fff640 10 monclient: get_auth_request con 0x7fa1b403e550 auth_method 0 2024-09-17T16:42:13.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d3fff640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa1b403e550 0x7fa1b4040a10 secure :-1 s=READY pgs=3616 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc003990 tx=0x7fa1bc009110 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.299+0000 7fa1d17fa640 10 client.54840.objecter ms_handle_connect 0x7fa1b403e550 2024-09-17T16:42:13.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:13.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 0x7fa1d4071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fa1d41407b0 con 0x7fa1d413eb70 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 0x7fa1d4071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient: get_auth_request con 0x7fa1d413eb70 auth_method 0 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 10 monclient(hunting): handle_auth_done global_id 54800 payload 824 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 1 -- 172.21.15.105:0/1234790320 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa1d4069f60 con 0x7fa1d413eb70 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d8fd8640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 0x7fa1d4071710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc017000 tx=0x7fa1cc0014b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 client.54840.objecter ms_handle_connect 0x7fa1d413eb70 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 client.54840.objecter resend_mon_ops 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa1cc00d9d0 con 0x7fa1d413eb70 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 monclient: dump: 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.334 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9445 (secure 0 0 0) 0x7fa1cc006b60 con 0x7fa1d413eb70 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 msgr2=0x7fa1d4071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 0x7fa1d4071710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc017000 tx=0x7fa1cc0014b0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 msgr2=0x7fa1d4071710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d17fa640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fa1d413eb70 0x7fa1d4071710 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc017000 tx=0x7fa1cc0014b0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 440305, 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:13.336 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.337 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.338 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.340 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:13.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.342 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.344 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.346 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.347 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.348 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.349 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:13.351 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:13.353 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.355 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.357 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.358 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 librados: watch_flush enter 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 librados: watch_flush exit 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 20 client.54840.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 client.54840.objecter successfully canceled tick 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa1b403e550 msgr2=0x7fa1b4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa1b403e550 0x7fa1b4040a10 secure :-1 s=READY pgs=3616 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc003990 tx=0x7fa1bc009110 comp rx=0 tx=0).stop 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 10 monclient: shutdown 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 msgr2=0x7fa1d413e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 --2- 172.21.15.105:0/1234790320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1d40a62c0 0x7fa1d413e0f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1c401ab10 tx=0x7fa1c4019200 comp rx=0 tx=0).stop 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 reap_dead start 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1d97d9640 1 -- 172.21.15.105:0/1234790320 reap_dead start 2024-09-17T16:42:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 shutdown_connections 2024-09-17T16:42:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 >> 172.21.15.105:0/1234790320 conn(0x7fa1d40aa3c0 msgr2=0x7fa1d413cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 shutdown_connections 2024-09-17T16:42:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 -- 172.21.15.105:0/1234790320 wait complete. 2024-09-17T16:42:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.307+0000 7fa1db23c640 1 librados: shutdown 2024-09-17T16:42:13.361 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.361 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-17T16:42:13.361 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: get_monmap_and_config 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: monmap: 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.472599+0000 2024-09-17T16:42:13.476 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.472599+0000 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:13.477 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 Processor -- start 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 -- start start 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: init 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient: _add_conns ranks=[7,9,8,2,0,6,4,10,3,1,5] 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-i con 0x7fdcac0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:42:13.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcac0a4ab0 0x7fdcac0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-a con 0x7fdcac0a4ab0 addr v2:172.21.15.105:3300/0 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac1397f0 0x7fdcac139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-e con 0x7fdcac1397f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdcac0a7c40 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdcac0a7890 con 0x7fdcac0a6a30 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fdcac0a4080 con 0x7fdcac1397f0 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb4495640 10 monclient(hunting): authenticate will time out at 1816.738261s 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcac0a4ab0 0x7fdcac0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcac0a4ab0 0x7fdcac0a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33548/0 (socket says 172.21.15.105:33548) 2024-09-17T16:42:13.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 -- 172.21.15.105:0/3630414950 learned_addr learned my addr 172.21.15.105:0/3630414950 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): get_auth_request con 0x7fdcac0a4ab0 auth_method 0 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac1397f0 0x7fdcac139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): get_auth_request con 0x7fdcac1397f0 auth_method 0 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient(hunting): handle_auth_done global_id 36432 payload 274 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 -- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac1397f0 msgr2=0x7fdcac139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac1397f0 0x7fdcac139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 -- 172.21.15.105:0/3630414950 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 msgr2=0x7fdcac0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 monclient: found mon.noname-a 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:42:13.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.467+0000 7fdcb1a30640 1 -- 172.21.15.105:0/3630414950 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac0a37e0 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac0a4ab0 0x7fdcac0a7350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdca00098a0 tx=0x7fdca001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b73a08cbaacd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 1 -- 172.21.15.105:0/3630414950 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdca00b0070 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: dump: 2024-09-17T16:42:13.488 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 1 -- 172.21.15.105:0/3630414950 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca00aa810 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 5 monclient: authenticate success, global_id 36432 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 1 -- 172.21.15.105:0/3630414950 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdca00a9300 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 4 set_mon_vals no callback set 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:42:13.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb122f640 10 monclient: dump: 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.491 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: shutdown 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac0a4ab0 msgr2=0x7fdcac0a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac0a4ab0 0x7fdcac0a7350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdca00098a0 tx=0x7fdca001c3d0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- 172.21.15.105:0/3630414950 shutdown_connections 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac1397f0 0x7fdcac139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac0a4ab0 0x7fdcac0a7350 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3630414950 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- 172.21.15.105:0/3630414950 >> 172.21.15.105:0/3630414950 conn(0x7fdcac0a8d00 msgr2=0x7fdcac0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- 172.21.15.105:0/3630414950 shutdown_connections 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- 172.21.15.105:0/3630414950 wait complete. 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: monmap: 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 librados: starting msgr at 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 librados: starting objecter 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 Processor -- start 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- start start 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 librados: setting wanted keys 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 librados: calling monclient init 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: init 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient: _add_conns ranks=[8,7,0,3,6,5,4,1,2] 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 0x7fdcac1226c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-e con 0x7fdcac0a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac1d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-i con 0x7fdcac0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:42:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac1397f0 0x7fdcac13d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): picked mon.noname-a con 0x7fdcac1397f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06e770 con 0x7fdcac1397f0 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdcac0a2fc0 con 0x7fdcac0a6a30 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fdcac143260 con 0x7fdcac0a4ab0 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb4495640 10 monclient(hunting): authenticate will time out at 1816.743305s 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 0x7fdcac1226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac1397f0 0x7fdcac13d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 0x7fdcac1226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49076/0 (socket says 172.21.15.105:49076) 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 1 -- 172.21.15.105:0/3617987903 learned_addr learned my addr 172.21.15.105:0/3617987903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): get_auth_request con 0x7fdcac1397f0 auth_method 0 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): get_auth_request con 0x7fdcac0a4ab0 auth_method 0 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient(hunting): handle_auth_done global_id 36441 payload 824 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 msgr2=0x7fdcac1226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 0x7fdcac1226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac1d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 msgr2=0x7fdcac1d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdcac0a6a30 0x7fdcac1d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 monclient: found mon.noname-a 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2a32640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac123050 con 0x7fdcac1397f0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.471+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdcac0a4ab0 0x7fdcac1226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2a32640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac1397f0 0x7fdcac13d540 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00eb80 tx=0x7fdc9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.?.objecter ms_handle_connect 0x7fdcac1397f0 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc9c018020 con 0x7fdcac1397f0 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: dump: 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9c014480 con 0x7fdcac1397f0 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 5 monclient: authenticate success, global_id 36441 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 monclient: _renew_subs 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac123ec0 con 0x7fdcac1397f0 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc9c014aa0 con 0x7fdcac1397f0 2024-09-17T16:42:13.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: dump: 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 client.36441.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 monclient: _renew_subs 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac0aa090 con 0x7fdcac1397f0 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 1 librados: init done 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcabfff640 4 set_mon_vals no callback set 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 0x7fdcac123860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac051e80 con 0x7fdcac1d1d30 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdc9c01b020 con 0x7fdcac1397f0 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 0x7fdcac123860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient: get_auth_request con 0x7fdcac1d1d30 auth_method 0 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc9803e580 0x7fdc98040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7fdc9c0509c0 con 0x7fdcac1397f0 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.36441.objecter ms_dispatch 0x7fdcac000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 3 client.36441.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 3 client.36441.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 20 client.36441.objecter dump_active .. 0 homeless 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc9803e580 0x7fdc98040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb1a30640 10 monclient: get_auth_request con 0x7fdc9803e580 auth_method 0 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 10 monclient(hunting): handle_auth_done global_id 84277 payload 824 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdcac13faf0 con 0x7fdcac1d1d30 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 0x7fdcac123860 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdca400fca0 tx=0x7fdca400d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.36441.objecter ms_handle_connect 0x7fdcac1d1d30 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.36441.objecter resend_mon_ops 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdca40138b0 con 0x7fdcac1d1d30 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: dump: 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 client.36441.objecter ms_handle_connect 0x7fdc9803e580 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcb1a30640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc9803e580 0x7fdc98040a40 secure :-1 s=READY pgs=3617 cs=0 l=1 rev1=1 crypto rx=0x7fdca00ad000 tx=0x7fdca00a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdca4013440 con 0x7fdcac1d1d30 2024-09-17T16:42:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 msgr2=0x7fdcac123860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 0x7fdcac123860 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdca400fca0 tx=0x7fdca400d590 comp rx=0 tx=0).stop 2024-09-17T16:42:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 msgr2=0x7fdcac123860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.475+0000 7fdcaaffd640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac1d1d30 0x7fdcac123860 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 0x7fdcac13de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06a200 con 0x7fdcac13da80 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 0x7fdcac13de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient: get_auth_request con 0x7fdcac13da80 auth_method 0 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 10 monclient(hunting): handle_auth_done global_id 84286 payload 824 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 1 -- 172.21.15.105:0/3617987903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdcac13e3d0 con 0x7fdcac13da80 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb2231640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 0x7fdcac13de90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdca400fca0 tx=0x7fdca4004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 client.36441.objecter ms_handle_connect 0x7fdcac13da80 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 client.36441.objecter resend_mon_ops 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdca40044c0 con 0x7fdcac13da80 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 monclient: dump: 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.509 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9275 (secure 0 0 0) 0x7fdca4013cc0 con 0x7fdcac13da80 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 msgr2=0x7fdcac13de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 0x7fdcac13de90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdca400fca0 tx=0x7fdca4004740 comp rx=0 tx=0).stop 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 msgr2=0x7fdcac13de90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcaaffd640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcac13da80 0x7fdcac13de90 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdca400fca0 tx=0x7fdca4004740 comp rx=0 tx=0).stop 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 40719, 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:13.511 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:13.513 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:13.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.515 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:13.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.518 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.519 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.521 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.522 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.523 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.524 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:13.526 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.527 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:13.528 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.531 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.532 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 librados: watch_flush enter 2024-09-17T16:42:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 librados: watch_flush exit 2024-09-17T16:42:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 20 client.36441.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 client.36441.objecter successfully canceled tick 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc9803e580 msgr2=0x7fdc98040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdc9803e580 0x7fdc98040a40 secure :-1 s=READY pgs=3617 cs=0 l=1 rev1=1 crypto rx=0x7fdca00ad000 tx=0x7fdca00a6000 comp rx=0 tx=0).stop 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 10 monclient: shutdown 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.483+0000 7fdcb4495640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac1397f0 msgr2=0x7fdcac13d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 --2- 172.21.15.105:0/3617987903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcac1397f0 0x7fdcac13d540 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00eb80 tx=0x7fdc9c00c490 comp rx=0 tx=0).stop 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb2a32640 1 -- 172.21.15.105:0/3617987903 reap_dead start 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb2a32640 1 -- 172.21.15.105:0/3617987903 reap_dead start 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 shutdown_connections 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 >> 172.21.15.105:0/3617987903 conn(0x7fdcac0a8d00 msgr2=0x7fdcac0a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 shutdown_connections 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 -- 172.21.15.105:0/3617987903 wait complete. 2024-09-17T16:42:13.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.487+0000 7fdcb4495640 1 librados: shutdown 2024-09-17T16:42:13.536 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.536 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-17T16:42:13.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:42:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: get_monmap_and_config 2024-09-17T16:42:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: monmap: 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.648917+0000 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.648917+0000 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:13.654 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 Processor -- start 2024-09-17T16:42:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 -- start start 2024-09-17T16:42:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: init 2024-09-17T16:42:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient: _add_conns ranks=[8,6,10,3,9,4,7,2,0,5,1] 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-e con 0x7f998c0a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-d con 0x7f998c0a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 0x7f998c139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-f con 0x7f998c1396a0 addr v2:172.21.15.160:3300/0 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f998c0a3fc0 con 0x7f998c1396a0 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f998c0a63c0 con 0x7f998c0a4910 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f998c0a7740 con 0x7f998c0a68e0 2024-09-17T16:42:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9992eee640 10 monclient(hunting): authenticate will time out at 1816.914878s 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f998bfff640 1 -- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 msgr2=0x7f998c0a7200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f998bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c0a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49082/0 (socket says 172.21.15.105:49082) 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 -- 172.21.15.105:0/961413441 learned_addr learned my addr 172.21.15.105:0/961413441 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): get_auth_request con 0x7f998c0a68e0 auth_method 0 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f999148b640 1 --2- 172.21.15.105:0/961413441 >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 0x7f998c139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient(hunting): handle_auth_done global_id 54827 payload 274 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 -- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 msgr2=0x7f998c0a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 --2- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c0a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 -- 172.21.15.105:0/961413441 >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 msgr2=0x7f998c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 --2- 172.21.15.105:0/961413441 >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 0x7f998c139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 monclient: found mon.noname-e 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f9990c8a640 1 -- 172.21.15.105:0/961413441 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c06ebc0 con 0x7f998c0a68e0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f999148b640 10 monclient: get_auth_request con 0x7f998c1396a0 auth_method 0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.643+0000 7f999148b640 1 --2- 172.21.15.105:0/961413441 >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 0x7f998c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9990c8a640 1 --2- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f997c009870 tx=0x7f997c01c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d2fec32e5b889fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 1 -- 172.21.15.105:0/961413441 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f997c0b2070 con 0x7f998c0a68e0 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: dump: 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.662 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 5 monclient: authenticate success, global_id 54827 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 1 -- 172.21.15.105:0/961413441 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c0ab7b0 con 0x7f998c0a68e0 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 4 set_mon_vals no callback set 2024-09-17T16:42:13.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 1 -- 172.21.15.105:0/961413441 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f997c0abe50 con 0x7f998c0a68e0 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f998b7fe640 10 monclient: dump: 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.664 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: shutdown 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 msgr2=0x7f998c0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f997c009870 tx=0x7f997c01c410 comp rx=0 tx=0).stop 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- 172.21.15.105:0/961413441 shutdown_connections 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- 172.21.15.105:0/961413441 >> v2:172.21.15.160:3300/0 conn(0x7f998c1396a0 0x7f998c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- 172.21.15.105:0/961413441 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f998c0a68e0 0x7f998c0a6cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- 172.21.15.105:0/961413441 >> 172.21.15.105:0/961413441 conn(0x7f998c068ba0 msgr2=0x7f998c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- 172.21.15.105:0/961413441 shutdown_connections 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- 172.21.15.105:0/961413441 wait complete. 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: monmap: 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.666 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 librados: starting msgr at 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 librados: starting objecter 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 Processor -- start 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- start start 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 librados: setting wanted keys 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 librados: calling monclient init 2024-09-17T16:42:13.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: init 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient: _add_conns ranks=[6,5,2,3,1,4,0,7,8] 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c1380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-d con 0x7f998c0a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f998c0a68e0 0x7f998c138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-h con 0x7f998c0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): picked mon.noname-b con 0x7f998c1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f998c06e630 con 0x7f998c1396a0 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f998c0a7af0 con 0x7f998c0a68e0 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f998c143320 con 0x7f998c0a4910 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9992eee640 10 monclient(hunting): authenticate will time out at 1816.919756s 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9990c8a640 1 -- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 msgr2=0x7f998c1380c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3303/0 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f9990c8a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c1380c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:58318/0 (socket says 172.21.15.105:58318) 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 1 -- 172.21.15.105:0/630044535 learned_addr learned my addr 172.21.15.105:0/630044535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 10 monclient(hunting): get_auth_request con 0x7f998c1396a0 auth_method 0 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.647+0000 7f999148b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f998c0a68e0 0x7f998c138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient(hunting): handle_auth_done global_id 84314 payload 824 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 msgr2=0x7f998c1380c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f998c0a4910 0x7f998c1380c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f998c0a68e0 msgr2=0x7f998c138600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f998c0a68e0 0x7f998c138600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 monclient: found mon.noname-b 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c13fcb0 con 0x7f998c1396a0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 10 monclient: get_auth_request con 0x7f998c0a68e0 auth_method 0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f998c0a68e0 0x7f998c138600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f999148b640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f998400cd50 tx=0x7f998400a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.?.objecter ms_handle_connect 0x7f998c1396a0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f998400f8f0 con 0x7f998c1396a0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: dump: 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998400fad0 con 0x7f998c1396a0 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 5 monclient: authenticate success, global_id 84314 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 monclient: _renew_subs 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c123ec0 con 0x7f998c1396a0 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f998400fc80 con 0x7f998c1396a0 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: dump: 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998affd640 4 set_mon_vals no callback set 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 client.84314.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 monclient: _renew_subs 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c0a30d0 con 0x7f998c1396a0 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 1 librados: init done 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 0x7f998c13dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f998c051e80 con 0x7f998c123860 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f998400f480 con 0x7f998c1396a0 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f997803e550 0x7f9978040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7f9984050e30 con 0x7f998c1396a0 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.84314.objecter ms_dispatch 0x7f998c000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 3 client.84314.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 3 client.84314.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 0x7f998c13dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient: get_auth_request con 0x7f998c123860 auth_method 0 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 20 client.84314.objecter dump_active .. 0 homeless 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f997803e550 0x7f9978040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 10 monclient: get_auth_request con 0x7f997803e550 auth_method 0 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 10 monclient(hunting): handle_auth_done global_id 54849 payload 824 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f998c139ae0 con 0x7f998c123860 2024-09-17T16:42:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.84314.objecter ms_handle_connect 0x7f997803e550 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f998bfff640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f997803e550 0x7f9978040a10 secure :-1 s=READY pgs=3618 cs=0 l=1 rev1=1 crypto rx=0x7f998c139420 tx=0x7f9980008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.84314.objecter ms_handle_connect 0x7f998c123860 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 client.84314.objecter resend_mon_ops 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f9990c8a640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 0x7f998c13dc40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f998c074620 tx=0x7f997c01ac50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f997c0bc070 con 0x7f998c123860 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: dump: 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f997c004100 con 0x7f998c123860 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 msgr2=0x7f998c13dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 0x7f998c13dc40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f998c074620 tx=0x7f997c01ac50 comp rx=0 tx=0).stop 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 msgr2=0x7f998c13dc40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.651+0000 7f99897fa640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c123860 0x7f998c13dc40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9992eee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9992eee640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 0x7f998c13e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9992eee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f998c13ead0 con 0x7f998c13e180 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 0x7f998c13e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient: get_auth_request con 0x7f998c13e180 auth_method 0 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 10 monclient(hunting): handle_auth_done global_id 54858 payload 824 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 1 -- 172.21.15.105:0/630044535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f998c123c40 con 0x7f998c13e180 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f9990c8a640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 0x7f998c13e590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f997c01ccb0 tx=0x7f997c0191b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 10 client.84314.objecter ms_handle_connect 0x7f998c13e180 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 10 client.84314.objecter resend_mon_ops 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f997c0bc050 con 0x7f998c13e180 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.659+0000 7f99897fa640 10 monclient: dump: 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.686 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9276 (secure 0 0 0) 0x7f997c0bd040 con 0x7f998c13e180 2024-09-17T16:42:13.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 msgr2=0x7f998c13e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 0x7f998c13e590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f997c01ccb0 tx=0x7f997c0191b0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 msgr2=0x7f998c13e590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f99897fa640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f998c13e180 0x7f998c13e590 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f997c01ccb0 tx=0x7f997c0191b0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:13.688 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 440671, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:13.689 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:13.690 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:13.691 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:13.692 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.693 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:13.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.701 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:13.703 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.704 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.706 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.707 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.708 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.709 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 10 librados: watch_flush enter 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 10 librados: watch_flush exit 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 20 client.84314.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 10 client.84314.objecter successfully canceled tick 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f997803e550 msgr2=0x7f9978040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f997803e550 0x7f9978040a10 secure :-1 s=READY pgs=3618 cs=0 l=1 rev1=1 crypto rx=0x7f998c139420 tx=0x7f9980008040 comp rx=0 tx=0).stop 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 10 monclient: shutdown 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 msgr2=0x7f998c13d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f998400cd50 tx=0x7f998400a650 comp rx=0 tx=0).stop 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f999148b640 1 -- 172.21.15.105:0/630044535 reap_dead start 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 shutdown_connections 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 --2- 172.21.15.105:0/630044535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f998c1396a0 0x7f998c13d700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 >> 172.21.15.105:0/630044535 conn(0x7f998c068ba0 msgr2=0x7f998c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 shutdown_connections 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 -- 172.21.15.105:0/630044535 wait complete. 2024-09-17T16:42:13.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.663+0000 7f9992eee640 1 librados: shutdown 2024-09-17T16:42:13.713 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-17T16:42:13.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:42:13.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: get_monmap_and_config 2024-09-17T16:42:13.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: monmap: 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.819891+0000 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.819891+0000 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:13.825 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 Processor -- start 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 -- start start 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: init 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient: _add_conns ranks=[0,2,8,6,10,1,3,5,9,7,4] 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08800a6a30 addr v1:172.21.15.105:6789/0 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 0x7f088013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): picked mon.noname-b con 0x7f08800a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): picked mon.noname-e con 0x7f08800a7520 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08800a4080 con 0x7f08800a6a30 2024-09-17T16:42:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f08800a7c40 con 0x7f08800a4ab0 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f08800a37e0 con 0x7f08800a7520 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0887620640 10 monclient(hunting): authenticate will time out at 1817.085505s 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f08853bc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f08800a6a30 0x7f08800a6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53400/0 (socket says 172.21.15.105:53400) 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f08853bc640 1 -- 172.21.15.105:0/1200664497 learned_addr learned my addr 172.21.15.105:0/1200664497 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 0x7f088013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): get_auth_request con 0x7f08800a4ab0 auth_method 0 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): get_auth_request con 0x7f08800a7520 auth_method 0 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 1 -- 172.21.15.105:0/1200664497 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3466552194 0 0) 0x7f08800a4080 con 0x7f08800a6a30 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 10 monclient(hunting): _negotiate 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 10 monclient(hunting): my global_id is 36450 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0877fff640 1 -- 172.21.15.105:0/1200664497 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0868003530 con 0x7f08800a6a30 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient(hunting): handle_auth_done global_id 84323 payload 274 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 -- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 msgr2=0x7f0880140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 -- 172.21.15.105:0/1200664497 >> v1:172.21.15.105:6789/0 conn(0x7f08800a6a30 legacy=0x7f08800a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 monclient: found mon.noname-b 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0884bbb640 1 -- 172.21.15.105:0/1200664497 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800a3ac0 con 0x7f08800a4ab0 2024-09-17T16:42:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.815+0000 7f0885bbd640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0884bbb640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 0x7f088013ea30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f087c009dc0 tx=0x7f087c01c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2288fa041388f7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 1 -- 172.21.15.105:0/1200664497 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f087c0b2070 con 0x7f08800a4ab0 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: dump: 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 1 -- 172.21.15.105:0/1200664497 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c0a5b60 con 0x7f08800a4ab0 2024-09-17T16:42:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 5 monclient: authenticate success, global_id 84323 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 1 -- 172.21.15.105:0/1200664497 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f087c0a5d50 con 0x7f08800a4ab0 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 4 set_mon_vals no callback set 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0877fff640 10 monclient: dump: 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: shutdown 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 msgr2=0x7f088013ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 0x7f088013ea30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f087c009dc0 tx=0x7f087c01c810 comp rx=0 tx=0).stop 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/1200664497 shutdown_connections 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- 172.21.15.105:0/1200664497 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08800a4ab0 0x7f088013ea30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/1200664497 >> 172.21.15.105:0/1200664497 conn(0x7f08800a8d00 msgr2=0x7f08800a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/1200664497 shutdown_connections 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/1200664497 wait complete. 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: build_initial_monmap 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: monmap: 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 librados: starting msgr at 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 librados: starting objecter 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 Processor -- start 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- start start 2024-09-17T16:42:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 librados: setting wanted keys 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 librados: calling monclient init 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: init 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _add_conns ranks=[1,5,8,2,4,7,6,0,3] 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08800a4ab0 0x7f0880128850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): picked mon.noname-f con 0x7f08800a4ab0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08800a6a30 0x7f08801263c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): picked mon.noname-h con 0x7f08800a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): picked mon.noname-e con 0x7f08800a7520 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0880143e60 con 0x7f08800a4ab0 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0880143ce0 con 0x7f08800a6a30 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0880143fe0 con 0x7f08800a7520 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient(hunting): authenticate will time out at 1817.090782s 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49102/0 (socket says 172.21.15.105:49102) 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 -- 172.21.15.105:0/3420107935 learned_addr learned my addr 172.21.15.105:0/3420107935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): get_auth_request con 0x7f08800a7520 auth_method 0 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0884bbb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08800a6a30 0x7f08801263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08800a4ab0 0x7f0880128850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient(hunting): handle_auth_done global_id 54845 payload 824 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08800a6a30 msgr2=0x7f08801263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08800a6a30 0x7f08801263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08800a4ab0 msgr2=0x7f0880128850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08800a4ab0 0x7f0880128850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient: _finish_hunting 0 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 monclient: found mon.noname-e 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801291e0 con 0x7f08800a7520 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f08853bc640 10 monclient: get_auth_request con 0x7f08800a4ab0 auth_method 0 2024-09-17T16:42:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0884bbb640 10 monclient: get_auth_request con 0x7f08800a6a30 auth_method 0 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08800a4ab0 0x7f0880128850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0884bbb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08800a6a30 0x7f08801263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0885bbd640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f087000e7f0 tx=0x7f087000ecb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 client.?.objecter ms_handle_connect 0x7f08800a7520 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0870017070 con 0x7f08800a7520 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: dump: 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.847 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: _finish_auth 0 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:42:13.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087000cdb0 con 0x7f08800a7520 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08700138e0 con 0x7f08800a7520 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 5 monclient: authenticate success, global_id 54845 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _renew_subs 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0880127cf0 con 0x7f08800a7520 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f08777fe640 4 set_mon_vals no callback set 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0875ffb640 10 monclient: dump: 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.849 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 client.54845.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _renew_subs 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f088012a550 con 0x7f08800a7520 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.819+0000 7f0887620640 1 librados: init done 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0887620640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0887620640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 0x7f08801d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0880051e80 con 0x7f0880126e40 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0870013470 con 0x7f08800a7520 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 0x7f08801d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient: get_auth_request con 0x7f0880126e40 auth_method 0 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f086c03e550 0x7f086c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7f087000f040 con 0x7f08800a7520 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 client.54845.objecter ms_dispatch 0x7f0880000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 3 client.54845.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 3 client.54845.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 20 client.54845.objecter dump_active .. 0 homeless 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0884bbb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f086c03e550 0x7f086c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0884bbb640 10 monclient: get_auth_request con 0x7f086c03e550 auth_method 0 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 10 monclient(hunting): handle_auth_done global_id 74336 payload 824 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0880127220 con 0x7f0880126e40 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 client.54845.objecter ms_handle_connect 0x7f0880126e40 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 client.54845.objecter resend_mon_ops 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 0x7f08801d0ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f087800db80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 client.54845.objecter ms_handle_connect 0x7f086c03e550 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0884bbb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f086c03e550 0x7f086c040a10 secure :-1 s=READY pgs=3619 cs=0 l=1 rev1=1 crypto rx=0x7f087c0af000 tx=0x7f087c0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0878005d30 con 0x7f0880126e40 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 monclient: dump: 2024-09-17T16:42:13.853 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.854 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0878017070 con 0x7f0880126e40 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 msgr2=0x7f08801d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 0x7f08801d0ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f087800db80 comp rx=0 tx=0).stop 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 msgr2=0x7f08801d0ef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.823+0000 7f0875ffb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0880126e40 0x7f08801d0ef0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f087800db80 comp rx=0 tx=0).stop 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0887620640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0887620640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 0x7f08801d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0887620640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f08801d1d80 con 0x7f08801d1430 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 0x7f08801d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient: get_auth_request con 0x7f08801d1430 auth_method 0 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:13.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 10 monclient(hunting): handle_auth_done global_id 74345 payload 824 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 1 -- 172.21.15.105:0/3420107935 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f08800a6e10 con 0x7f08801d1430 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f08853bc640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 0x7f08801d1840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f0878005800 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 client.54845.objecter ms_handle_connect 0x7f08801d1430 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 client.54845.objecter resend_mon_ops 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f087801ce10 con 0x7f08801d1430 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 monclient: dump: 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:13.858 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:13.859 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9276 (secure 0 0 0) 0x7f0878007280 con 0x7f08801d1430 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 msgr2=0x7f08801d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 0x7f08801d1840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f0878005800 comp rx=0 tx=0).stop 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 msgr2=0x7f08801d1840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.831+0000 7f0875ffb640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f08801d1430 0x7f08801d1840 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0878005890 tx=0x7f0878005800 comp rx=0 tx=0).stop 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.860 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 173306, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:42:13.861 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.862 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.863 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.864 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:13.865 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.866 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.867 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.869 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.872 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.874 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.875 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.877 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:13.878 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:13.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:13.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:13.881 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:13.882 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 10 librados: watch_flush enter 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 10 librados: watch_flush exit 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 20 client.54845.objecter shutdown clearing up homeless session... 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 10 client.54845.objecter successfully canceled tick 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f086c03e550 msgr2=0x7f086c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f086c03e550 0x7f086c040a10 secure :-1 s=READY pgs=3619 cs=0 l=1 rev1=1 crypto rx=0x7f087c0af000 tx=0x7f087c0a7070 comp rx=0 tx=0).stop 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 10 monclient: shutdown 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:13.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 msgr2=0x7f0880126900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f087000e7f0 tx=0x7f087000ecb0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0885bbd640 1 -- 172.21.15.105:0/3420107935 reap_dead start 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 shutdown_connections 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 --2- 172.21.15.105:0/3420107935 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f08800a7520 0x7f0880126900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 >> 172.21.15.105:0/3420107935 conn(0x7f08800a8d00 msgr2=0x7f088009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 shutdown_connections 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 -- 172.21.15.105:0/3420107935 wait complete. 2024-09-17T16:42:13.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.835+0000 7f0887620640 1 librados: shutdown 2024-09-17T16:42:13.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:13.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-17T16:42:13.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: get_monmap_and_config 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: build_initial_monmap 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: monmap: 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:42:13.996310+0000 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:42:13.996310+0000 2024-09-17T16:42:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:42:14.001 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 Processor -- start 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 -- start start 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: init 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient: _add_conns ranks=[5,9,8,0,7,2,6,4,10,3,1] 2024-09-17T16:42:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6fcc0a68e0 0x7f6fcc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-h con 0x7f6fcc0a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 0x7f6fcc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-a con 0x7f6fcc0a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-e con 0x7f6fcc1396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6fcc0a3fc0 con 0x7f6fcc0a4910 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a63c0 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a7740 con 0x7f6fcc1396a0 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fd1c94640 10 monclient(hunting): authenticate will time out at 1817.262031s 2024-09-17T16:42:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 0x7f6fcc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49118/0 (socket says 172.21.15.105:49118) 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1780090801 learned_addr learned my addr 172.21.15.105:0/1780090801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): get_auth_request con 0x7f6fcc1396a0 auth_method 0 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a4910 auth_method 0 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6fcc0a68e0 0x7f6fcc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcb7fe640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a68e0 auth_method 0 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_done global_id 54854 payload 274 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1780090801 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6fcc0a68e0 msgr2=0x7f6fcc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6fcc0a68e0 0x7f6fcc0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1780090801 >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 msgr2=0x7f6fcc0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1780090801 >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 0x7f6fcc0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:42:14.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 monclient: found mon.noname-e 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1780090801 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc06ebc0 con 0x7f6fcc1396a0 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.991+0000 7f6fcaffd640 1 --2- 172.21.15.105:0/1780090801 >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 0x7f6fcc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e50 tx=0x7f6fc401c8c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9b70d1daaa02254b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 1 -- 172.21.15.105:0/1780090801 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc40090d0 con 0x7f6fcc1396a0 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: dump: 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: _finish_auth 0 2024-09-17T16:42:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 1 -- 172.21.15.105:0/1780090801 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc40a5b90 con 0x7f6fcc1396a0 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 5 monclient: authenticate success, global_id 54854 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: get_monmap_and_config success 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 1 -- 172.21.15.105:0/1780090801 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc40a5d80 con 0x7f6fcc1396a0 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 4 set_mon_vals no callback set 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fca7fc640 10 monclient: dump: 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.010 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.011 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: shutdown 2024-09-17T16:42:14.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:14.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1780090801 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 msgr2=0x7f6fcc139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e50 tx=0x7f6fc401c8c0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1780090801 shutdown_connections 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc1396a0 0x7f6fcc139b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1780090801 >> v2:172.21.15.105:3300/0 conn(0x7f6fcc0a4910 0x7f6fcc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1780090801 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6fcc0a68e0 0x7f6fcc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1780090801 >> 172.21.15.105:0/1780090801 conn(0x7f6fcc068ba0 msgr2=0x7f6fcc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1780090801 shutdown_connections 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1780090801 wait complete. 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: build_initial_monmap 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: monmap: 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:42:14.019 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 librados: starting msgr at 2024-09-17T16:42:14.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 librados: starting objecter 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 Processor -- start 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- start start 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 librados: setting wanted keys 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 librados: calling monclient init 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: init 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _reopen_session rank -1 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _add_conns ranks=[3,8,0,7,6,4,2,5,1] 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fcc0a4910 0x7f6fcc138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-g con 0x7f6fcc0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 0x7f6fcc1386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-e con 0x7f6fcc0a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 0x7f6fcc13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): picked mon.noname-a con 0x7f6fcc1396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc06e630 con 0x7f6fcc1396a0 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a7af0 con 0x7f6fcc0a4910 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6fcc143320 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): _renew_subs 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient(hunting): authenticate will time out at 1817.267051s 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 0x7f6fcc1386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 0x7f6fcc13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 0x7f6fcc13d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33572/0 (socket says 172.21.15.105:33572) 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 0x7f6fcc1386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:49120/0 (socket says 172.21.15.105:49120) 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1438742233 learned_addr learned my addr 172.21.15.105:0/1438742233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): get_auth_request con 0x7f6fcc1396a0 auth_method 0 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a68e0 auth_method 0 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fcc0a4910 0x7f6fcc138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient(hunting): handle_auth_done global_id 54863 payload 824 2024-09-17T16:42:14.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fcc0a4910 msgr2=0x7f6fcc138160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fcc0a4910 0x7f6fcc138160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 msgr2=0x7f6fcc13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 0x7f6fcc13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient: _finish_hunting 0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 monclient: found mon.noname-e 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0a1ba0 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcb7fe640 10 monclient: get_auth_request con 0x7f6fcc0a4910 auth_method 0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fcc1396a0 0x7f6fcc13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fcc0a4910 0x7f6fcc138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fcaffd640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 0x7f6fcc1386a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00093f0 tx=0x7f6fc000ed60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 client.?.objecter ms_handle_connect 0x7f6fcc0a68e0 2024-09-17T16:42:14.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc0017020 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: dump: 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.025 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: _finish_auth 0 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc0004240 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 5 monclient: authenticate success, global_id 54863 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _renew_subs 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc123c70 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc00043f0 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fc9ffb640 4 set_mon_vals no callback set 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fb3fff640 10 monclient: dump: 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.027 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 client.54863.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _renew_subs 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.995+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc13f190 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fd1c94640 1 librados: init done 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fd1c94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 0x7f6fcc13dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6fcc051e80 con 0x7f6fcc123690 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6fc001a020 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6fb803e580 0x7f6fb8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(548..548 src has 1..548) ==== 6718+0+0 (secure 0 0 0) 0x7f6fc000f040 con 0x7f6fcc0a68e0 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 client.54863.objecter ms_dispatch 0x7f6fcc000ba0 osd_map(548..548 src has 1..548) 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 3 client.54863.objecter handle_osd_map got epochs [548,548] > 0 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 3 client.54863.objecter handle_osd_map decoding full epoch 548 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 20 client.54863.objecter dump_active .. 0 homeless 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 0x7f6fcc13dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient: get_auth_request con 0x7f6fcc123690 auth_method 0 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6fb803e580 0x7f6fb8040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcbfff640 10 monclient: get_auth_request con 0x7f6fb803e580 auth_method 0 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_done global_id 64609 payload 824 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fcc139b50 con 0x7f6fcc123690 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 client.54863.objecter ms_handle_connect 0x7f6fb803e580 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcbfff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6fb803e580 0x7f6fb8040a40 secure :-1 s=READY pgs=3620 cs=0 l=1 rev1=1 crypto rx=0x7f6fc401d160 tx=0x7f6fc40a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 client.54863.objecter ms_handle_connect 0x7f6fcc123690 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 client.54863.objecter resend_mon_ops 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 0x7f6fcc13dc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e740 tx=0x7f6fbc00ec00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fcc123690 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:42:14.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 monclient: dump: 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.031 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6fbc00ed30 con 0x7f6fcc123690 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 msgr2=0x7f6fcc13dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 0x7f6fcc13dc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e740 tx=0x7f6fbc00ec00 comp rx=0 tx=0).stop 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 msgr2=0x7f6fcc13dc10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:13.999+0000 7f6fb3fff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc123690 0x7f6fcc13dc10 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e740 tx=0x7f6fbc00ec00 comp rx=0 tx=0).stop 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fd1c94640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 0x7f6fcc13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fd1c94640 10 monclient(hunting): start opening mon connection 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f6fcc13eaa0 con 0x7f6fcc13e150 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 0x7f6fcc13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient: get_auth_request con 0x7f6fcc13e150 auth_method 0 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:42:14.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 10 monclient(hunting): handle_auth_done global_id 64618 payload 824 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 1 -- 172.21.15.105:0/1438742233 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6fcc0746d0 con 0x7f6fcc13e150 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fcb7fe640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 0x7f6fcc13e560 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c10 tx=0x7f6fbc0046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 client.54863.objecter ms_handle_connect 0x7f6fcc13e150 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 client.54863.objecter resend_mon_ops 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc00f040 con 0x7f6fcc13e150 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 monclient: dump: 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:42:14.034 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:42:14.035 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9443 (secure 0 0 0) 0x7f6fbc00ce30 con 0x7f6fcc13e150 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 msgr2=0x7f6fcc13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 0x7f6fcc13e560 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c10 tx=0x7f6fbc0046d0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 msgr2=0x7f6fcc13e560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:42:14.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fb3fff640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f6fcc13e150 0x7f6fcc13e560 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c10 tx=0x7f6fbc0046d0 comp rx=0 tx=0).stop 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 90, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 307112, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:42:14.037 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:14.038 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:14.039 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:42:14.040 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:42:14.041 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:42:14.042 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:42:14.044 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.046 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.048 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.050 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:42:14.051 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.054 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:42:14.055 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:42:14.056 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:14.057 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:42:14.058 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.007+0000 7f6fd1c94640 10 librados: watch_flush enter 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 10 librados: watch_flush exit 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 20 client.54863.objecter shutdown clearing up homeless session... 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 10 client.54863.objecter successfully canceled tick 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6fb803e580 msgr2=0x7f6fb8040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f6fb803e580 0x7f6fb8040a40 secure :-1 s=READY pgs=3620 cs=0 l=1 rev1=1 crypto rx=0x7f6fc401d160 tx=0x7f6fc40a7070 comp rx=0 tx=0).stop 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 10 monclient: shutdown 2024-09-17T16:42:14.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 msgr2=0x7f6fcc1386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 --2- 172.21.15.105:0/1438742233 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f6fcc0a68e0 0x7f6fcc1386a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00093f0 tx=0x7f6fc000ed60 comp rx=0 tx=0).stop 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1438742233 reap_dead start 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fcbfff640 1 -- 172.21.15.105:0/1438742233 reap_dead start 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 shutdown_connections 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 >> 172.21.15.105:0/1438742233 conn(0x7f6fcc068ba0 msgr2=0x7f6fcc0a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 shutdown_connections 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 -- 172.21.15.105:0/1438742233 wait complete. 2024-09-17T16:42:14.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:42:14.011+0000 7f6fd1c94640 1 librados: shutdown 2024-09-17T16:42:14.061 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-17T16:42:14.062 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:42:54.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (8972 ms) 2024-09-17T16:42:54.199 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (5967 ms) 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5078 ms) 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (34660 ms) 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (25479 ms) 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (11881 ms) 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-17T16:42:54.200 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 00000012 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12035 ms) 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: require_osd_release = squid 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (5989 ms) 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (676523 ms total) 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (863 ms) 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10991 ms) 2024-09-17T16:42:54.201 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18176 ms) 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (6 ms) 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30036 ms total) 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (6932 ms) 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (7996 ms) 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: waiting for scrub... 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: done waiting 2024-09-17T16:42:54.202 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (38484 ms) 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (14720 ms) 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7976 ms) 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8171 ms) 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10030 ms) 2024-09-17T16:42:54.203 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-17T16:43:44.063 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-17T16:43:44.063 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-17T16:43:44.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-17T16:43:44.067 INFO:tasks.ceph.mon.d:Started 2024-09-17T16:43:44.067 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-17T16:43:44.068 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-17T16:43:44.068 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:43:59.069 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-17T16:43:59.069 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-17T16:43:59.070 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:43:59.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: get_monmap_and_config 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: monmap: 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:43:59.205679+0000 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:43:59.205679+0000 2024-09-17T16:43:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:43:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:43:59.212 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 Processor -- start 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 -- start start 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: init 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient: _add_conns ranks=[9,5,6,4,2,8,7,1,0,3,10] 2024-09-17T16:43:59.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7180a6a30 0x7fa7180a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): picked mon.noname-a con 0x7fa7180a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7180a4ab0 0x7fa7180a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): picked mon.noname-h con 0x7fa7180a4ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7181397f0 0x7fa718139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): picked mon.noname-d con 0x7fa7181397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa7180a7c40 con 0x7fa7180a6a30 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa7180a7890 con 0x7fa7180a4ab0 2024-09-17T16:43:59.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa7180a4080 con 0x7fa7181397f0 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa720259640 10 monclient(hunting): authenticate will time out at 1922.471520s 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7180a6a30 0x7fa7180a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7180a6a30 0x7fa7180a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36292/0 (socket says 172.21.15.105:36292) 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 -- 172.21.15.105:0/2767495780 learned_addr learned my addr 172.21.15.105:0/2767495780 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): get_auth_request con 0x7fa7180a6a30 auth_method 0 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7181397f0 0x7fa718139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): get_auth_request con 0x7fa7181397f0 auth_method 0 2024-09-17T16:43:59.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71d7f4640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7180a4ab0 0x7fa7180a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient(hunting): handle_auth_done global_id 36477 payload 274 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 -- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7181397f0 msgr2=0x7fa718139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7181397f0 0x7fa718139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 -- 172.21.15.105:0/2767495780 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7180a4ab0 msgr2=0x7fa7180a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7180a4ab0 0x7fa7180a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 1 monclient: found mon.noname-a 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.203+0000 7fa71dff5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71dff5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71dff5640 1 -- 172.21.15.105:0/2767495780 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7180a37e0 con 0x7fa7180a6a30 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180a6a30 0x7fa7180a6e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098a0 tx=0x7fa70801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a97112e9d7671ed server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 1 -- 172.21.15.105:0/2767495780 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7080b0070 con 0x7fa7180a6a30 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: dump: 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.224 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.225 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 5 monclient: authenticate success, global_id 36477 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 1 -- 172.21.15.105:0/2767495780 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7080aa810 con 0x7fa7180a6a30 2024-09-17T16:43:59.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 10 monclient: get_monmap_and_config success 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 1 -- 172.21.15.105:0/2767495780 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7080a9300 con 0x7fa7180a6a30 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 4 set_mon_vals no callback set 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa71cff3640 10 monclient: dump: 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.227 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 10 monclient: shutdown 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 -- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180a6a30 msgr2=0x7fa7180a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180a6a30 0x7fa7180a6e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa7080098a0 tx=0x7fa70801c3d0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 -- 172.21.15.105:0/2767495780 shutdown_connections 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa7181397f0 0x7fa718139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa7180a4ab0 0x7fa7180a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 --2- 172.21.15.105:0/2767495780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7180a6a30 0x7fa7180a6e10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 -- 172.21.15.105:0/2767495780 >> 172.21.15.105:0/2767495780 conn(0x7fa7180a8d00 msgr2=0x7fa7180a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 -- 172.21.15.105:0/2767495780 shutdown_connections 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 -- 172.21.15.105:0/2767495780 wait complete. 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 10 monclient: monmap: 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:43:59.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 librados: starting msgr at 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.207+0000 7fa720259640 1 librados: starting objecter 2024-09-17T16:43:59.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 Processor -- start 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- start start 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 librados: setting wanted keys 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 librados: calling monclient init 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: init 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _add_conns ranks=[4,7,2,1,3,8,0,5,6] 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): picked mon.noname-c con 0x7fa7180a4ab0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa7180a6a30 0x7fa71813d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): picked mon.noname-i con 0x7fa7180a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 0x7fa71813dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): picked mon.noname-b con 0x7fa7181397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa71806e770 con 0x7fa7181397f0 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fa7180a2fc0 con 0x7fa7180a4ab0 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fa718143260 con 0x7fa7180a6a30 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient(hunting): authenticate will time out at 1922.476605s 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 0x7fa71813dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34034/0 (socket says 172.21.15.105:34034) 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 -- 172.21.15.105:0/2691825284 learned_addr learned my addr 172.21.15.105:0/2691825284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): get_auth_request con 0x7fa7180a4ab0 auth_method 0 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): get_auth_request con 0x7fa7181397f0 auth_method 0 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient(hunting): handle_auth_done global_id 74614 payload 824 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 -- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa7180a6a30 msgr2=0x7fa71813d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71d7f4640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa7180a6a30 0x7fa71813d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa7180a6a30 0x7fa71813d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 -- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 msgr2=0x7fa71813dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 0x7fa71813dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 monclient: found mon.noname-c 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 -- 172.21.15.105:0/2691825284 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa718124080 con 0x7fa7180a4ab0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71e7f6640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 0x7fa71813dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71dff5640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7080097c0 tx=0x7fa708019170 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 client.?.objecter ms_handle_connect 0x7fa7180a4ab0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7080b9070 con 0x7fa7180a4ab0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: dump: 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 5 monclient: authenticate success, global_id 74614 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7080044b0 con 0x7fa7180a4ab0 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _renew_subs 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7180abc50 con 0x7fa7180a4ab0 2024-09-17T16:43:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa7080b0040 con 0x7fa7180a4ab0 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 client.74614.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70ffff640 4 set_mon_vals no callback set 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 monclient: dump: 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _renew_subs 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa718122880 con 0x7fa7180a4ab0 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 librados: init done 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718139c60 con 0x7fa7180a4ab0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa708004040 con 0x7fa7180a4ab0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 0x7fa6fc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(639..639 src has 1..639) ==== 6769+0+0 (secure 0 0 0) 0x7fa7080e9260 con 0x7fa7180a4ab0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 10 client.74614.objecter ms_dispatch 0x7fa718000ba0 osd_map(639..639 src has 1..639) 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 3 client.74614.objecter handle_osd_map got epochs [639,639] > 0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 3 client.74614.objecter handle_osd_map decoding full epoch 639 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa70e7fc640 20 client.74614.objecter dump_active .. 0 homeless 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71d7f4640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 0x7fa6fc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.211+0000 7fa71d7f4640 10 monclient: get_auth_request con 0x7fa6fc03e580 auth_method 0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.215+0000 7fa71d7f4640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 0x7fa6fc040a40 secure :-1 s=READY pgs=3627 cs=0 l=1 rev1=1 crypto rx=0x7fa7180a66d0 tx=0x7fa714008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.215+0000 7fa70e7fc640 10 client.74614.objecter ms_handle_connect 0x7fa6fc03e580 2024-09-17T16:43:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.215+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa708016110 con 0x7fa7180a4ab0 2024-09-17T16:43:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.215+0000 7fa70e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.215+0000 7fa70e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:43:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa720259640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:43:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa720259640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa720259640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa718074850 con 0x7fa7180a4ab0 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa70e7fc640 1 -- 172.21.15.105:0/2691825284 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7fa7080bc360 con 0x7fa7180a4ab0 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa70e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.451+0000 7fa70e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:43:59.458 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":102,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 10 librados: watch_flush enter 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 10 librados: watch_flush exit 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 20 client.74614.objecter shutdown clearing up homeless session... 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 10 client.74614.objecter successfully canceled tick 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 msgr2=0x7fa6fc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 0x7fa6fc040a40 secure :-1 s=READY pgs=3627 cs=0 l=1 rev1=1 crypto rx=0x7fa7180a66d0 tx=0x7fa714008040 comp rx=0 tx=0).stop 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 10 monclient: shutdown 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 msgr2=0x7fa7181386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7080097c0 tx=0x7fa708019170 comp rx=0 tx=0).stop 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 shutdown_connections 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa6fc03e580 0x7fa6fc040a40 unknown :-1 s=CLOSED pgs=3627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa7181397f0 0x7fa71813dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fa7180a6a30 0x7fa71813d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 --2- 172.21.15.105:0/2691825284 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fa7180a4ab0 0x7fa7181386e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 >> 172.21.15.105:0/2691825284 conn(0x7fa7180a8d00 msgr2=0x7fa71813fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 shutdown_connections 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 -- 172.21.15.105:0/2691825284 wait complete. 2024-09-17T16:43:59.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.455+0000 7fa720259640 1 librados: shutdown 2024-09-17T16:43:59.481 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:43:59.482 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:43:59.482 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:43:59.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: get_monmap_and_config 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: monmap: 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:43:59.596439+0000 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:43:59.596439+0000 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:43:59.602 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 Processor -- start 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 -- start start 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: init 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient: _add_conns ranks=[3,8,5,0,6,1,2,7,9,10,4] 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 0x7f46440a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-g con 0x7f46440a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-e con 0x7f46440a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:43:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46441396a0 0x7f4644139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-h con 0x7f46441396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f46440a3fc0 con 0x7f46440a68e0 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f46440a63c0 con 0x7f46441396a0 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f46440a7740 con 0x7f46440a4910 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464ba7c640 10 monclient(hunting): authenticate will time out at 1922.862466s 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33540/0 (socket says 172.21.15.105:33540) 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 -- 172.21.15.105:0/3597388360 learned_addr learned my addr 172.21.15.105:0/3597388360 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): get_auth_request con 0x7f46440a4910 auth_method 0 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649818640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 0x7f46440a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient(hunting): handle_auth_done global_id 54872 payload 274 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f464a019640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46441396a0 0x7f4644139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 -- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46441396a0 msgr2=0x7f4644139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46441396a0 0x7f4644139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 -- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 msgr2=0x7f46440a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 0x7f46440a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 monclient: found mon.noname-e 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649017640 1 -- 172.21.15.105:0/3597388360 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464406ec10 con 0x7f46440a4910 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649818640 10 monclient: get_auth_request con 0x7f46440a68e0 auth_method 0 2024-09-17T16:43:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.595+0000 7f4649818640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 0x7f46440a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46400098d0 tx=0x7f464001cea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=20aabfd551861760 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 1 -- 172.21.15.105:0/3597388360 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46400b0070 con 0x7f46440a4910 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: dump: 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.620 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.621 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 5 monclient: authenticate success, global_id 54872 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:43:59.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 1 -- 172.21.15.105:0/3597388360 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46400aa810 con 0x7f46440a4910 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: get_monmap_and_config success 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 4 set_mon_vals no callback set 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 1 -- 172.21.15.105:0/3597388360 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46400a9300 con 0x7f46440a4910 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4648816640 10 monclient: dump: 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.623 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.624 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: shutdown 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- 172.21.15.105:0/3597388360 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 msgr2=0x7f46440a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46400098d0 tx=0x7f464001cea0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- 172.21.15.105:0/3597388360 shutdown_connections 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f46441396a0 0x7f4644139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f46440a4910 0x7f46440a7200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- 172.21.15.105:0/3597388360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46440a68e0 0x7f46440a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- 172.21.15.105:0/3597388360 >> 172.21.15.105:0/3597388360 conn(0x7f4644068ba0 msgr2=0x7f4644068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- 172.21.15.105:0/3597388360 shutdown_connections 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- 172.21.15.105:0/3597388360 wait complete. 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: monmap: 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:43:59.626 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.627 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 librados: starting msgr at 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 librados: starting objecter 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 Processor -- start 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- start start 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 librados: setting wanted keys 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 librados: calling monclient init 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: init 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient: _add_conns ranks=[1,6,4,2,0,8,7,5,3] 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46440a4910 0x7f4644137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-f con 0x7f46440a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 0x7f4644137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-d con 0x7f46440a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 0x7f4644138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): picked mon.noname-c con 0x7f46441396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4644143320 con 0x7f46440a4910 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f464406e630 con 0x7f46441396a0 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f46441434a0 con 0x7f46440a68e0 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464ba7c640 10 monclient(hunting): authenticate will time out at 1922.867616s 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 0x7f4644137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 0x7f4644138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 0x7f4644137f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39198/0 (socket says 172.21.15.105:39198) 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 0x7f4644138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34040/0 (socket says 172.21.15.105:34040) 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 1 -- 172.21.15.105:0/563518950 learned_addr learned my addr 172.21.15.105:0/563518950 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): get_auth_request con 0x7f46440a68e0 auth_method 0 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): get_auth_request con 0x7f46441396a0 auth_method 0 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649017640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f464a019640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.599+0000 7f4649818640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46440a4910 0x7f4644137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): get_auth_request con 0x7f46440a4910 auth_method 0 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 10 monclient(hunting): handle_auth_done global_id 94119 payload 824 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 msgr2=0x7f4644138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 0x7f4644138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46440a4910 msgr2=0x7f4644137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46440a4910 0x7f4644137a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 monclient: found mon.noname-d 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46440a9fc0 con 0x7f46440a68e0 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464a019640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f46441396a0 0x7f4644138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649017640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 0x7f4644137f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46400ae040 tx=0x7f46400191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.?.objecter ms_handle_connect 0x7f46440a68e0 2024-09-17T16:43:59.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46400b9070 con 0x7f46440a68e0 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: dump: 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.633 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46400044b0 con 0x7f46440a68e0 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 5 monclient: authenticate success, global_id 94119 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 monclient: _renew_subs 2024-09-17T16:43:59.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4644123d70 con 0x7f46440a68e0 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46400b0040 con 0x7f46440a68e0 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463bfff640 4 set_mon_vals no callback set 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 client.94119.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: dump: 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.635 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 monclient: _renew_subs 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:43:59.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f464413f040 con 0x7f46440a68e0 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 1 librados: init done 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 0x7f46440a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4644051e80 con 0x7f46440a0a20 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 0x7f46440a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient: get_auth_request con 0x7f46440a0a20 auth_method 0 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4640004040 con 0x7f46440a68e0 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 10 monclient(hunting): handle_auth_done global_id 36486 payload 824 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46440abb70 con 0x7f46440a0a20 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f462803e540 0x7f4628040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(640..640 src has 1..640) ==== 6769+0+0 (secure 0 0 0) 0x7f46400e9330 con 0x7f46440a68e0 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.94119.objecter ms_dispatch 0x7f4644000ba0 osd_map(640..640 src has 1..640) 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 3 client.94119.objecter handle_osd_map got epochs [640,640] > 0 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 3 client.94119.objecter handle_osd_map decoding full epoch 640 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 20 client.94119.objecter dump_active .. 0 homeless 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.94119.objecter ms_handle_connect 0x7f46440a0a20 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f4649818640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 0x7f46440a0e00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f463400e770 tx=0x7f463400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.94119.objecter resend_mon_ops 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4634019070 con 0x7f46440a0a20 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: dump: 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.639 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464a019640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f462803e540 0x7f4628040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464a019640 10 monclient: get_auth_request con 0x7f462803e540 auth_method 0 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4634004030 con 0x7f46440a0a20 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 msgr2=0x7f46440a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 0x7f46440a0e00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f463400e770 tx=0x7f463400ec30 comp rx=0 tx=0).stop 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 msgr2=0x7f46440a0e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a0a20 0x7f46440a0e00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f464a019640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f462803e540 0x7f4628040a00 secure :-1 s=READY pgs=3628 cs=0 l=1 rev1=1 crypto rx=0x7f4644074620 tx=0x7f463c007670 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.603+0000 7f463a7fc640 10 client.94119.objecter ms_handle_connect 0x7f462803e540 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 0x7f464413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4644139ae0 con 0x7f46440a1340 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 0x7f464413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient: get_auth_request con 0x7f46440a1340 auth_method 0 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 10 monclient(hunting): handle_auth_done global_id 36495 payload 824 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 1 -- 172.21.15.105:0/563518950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f46440746d0 con 0x7f46440a1340 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f4649818640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 0x7f464413d640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f463400e770 tx=0x7f46340046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 client.94119.objecter ms_handle_connect 0x7f46440a1340 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 client.94119.objecter resend_mon_ops 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f463400ac50 con 0x7f46440a1340 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 monclient: dump: 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.642 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9455 (secure 0 0 0) 0x7f463400ac50 con 0x7f46440a1340 2024-09-17T16:43:59.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 msgr2=0x7f464413d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 0x7f464413d640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f463400e770 tx=0x7f46340046d0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 msgr2=0x7f464413d640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f463a7fc640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46440a1340 0x7f464413d640 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f463400e770 tx=0x7f46340046d0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.644 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:43:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:43:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:43:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:43:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:43:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 830414, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:43:59.646 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:43:59.647 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.648 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.649 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.650 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.651 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:43:59.652 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:43:59.653 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:43:59.653 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:43:59.653 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:43:59.653 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:43:59.653 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.654 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:43:59.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.656 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:43:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.660 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:43:59.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.665 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:43:59.666 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:43:59.666 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.667 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.668 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:43:59.669 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.670 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.671 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.672 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.673 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:43:59.674 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 librados: watch_flush enter 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 librados: watch_flush exit 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 20 client.94119.objecter shutdown clearing up homeless session... 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 client.94119.objecter successfully canceled tick 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f462803e540 msgr2=0x7f4628040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f462803e540 0x7f4628040a00 secure :-1 s=READY pgs=3628 cs=0 l=1 rev1=1 crypto rx=0x7f4644074620 tx=0x7f463c007670 comp rx=0 tx=0).stop 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 10 monclient: shutdown 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 msgr2=0x7f4644137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 --2- 172.21.15.105:0/563518950 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f46440a68e0 0x7f4644137f50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f46400ae040 tx=0x7f46400191e0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464a019640 1 -- 172.21.15.105:0/563518950 reap_dead start 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464a019640 1 -- 172.21.15.105:0/563518950 reap_dead start 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 shutdown_connections 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 >> 172.21.15.105:0/563518950 conn(0x7f4644068ba0 msgr2=0x7f46440a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 shutdown_connections 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 -- 172.21.15.105:0/563518950 wait complete. 2024-09-17T16:43:59.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.611+0000 7f464ba7c640 1 librados: shutdown 2024-09-17T16:43:59.679 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:43:59.679 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:43:59.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: get_monmap_and_config 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: monmap: 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:43:59.792605+0000 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:43:59.792605+0000 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.797 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:43:59.798 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 Processor -- start 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 -- start start 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: init 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient: _add_conns ranks=[4,6,5,9,10,1,2,0,8,7,3] 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-c con 0x7f4a7409b410 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 0x7f4a74135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-d con 0x7f4a740a2690 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:43:59.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4a74135e50 0x7f4a74136290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-h con 0x7f4a74135e50 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f4a740a03f0 con 0x7f4a7409b410 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4a740a4cd0 con 0x7f4a74135e50 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f4a740a0e40 con 0x7f4a740a2690 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a794c7640 10 monclient(hunting): authenticate will time out at 1923.058493s 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 0x7f4a74135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34042/0 (socket says 172.21.15.105:34042) 2024-09-17T16:43:59.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 -- 172.21.15.105:0/352544474 learned_addr learned my addr 172.21.15.105:0/352544474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request con 0x7f4a7409b410 auth_method 0 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): get_auth_request con 0x7f4a740a2690 auth_method 0 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a737fe640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4a74135e50 0x7f4a74136290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a737fe640 10 monclient(hunting): get_auth_request con 0x7f4a74135e50 auth_method 0 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_done global_id 74632 payload 274 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 -- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 msgr2=0x7f4a74135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 0x7f4a74135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 -- 172.21.15.105:0/352544474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4a74135e50 msgr2=0x7f4a74136290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4a74135e50 0x7f4a74136290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 monclient: found mon.noname-c 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a72ffd640 1 -- 172.21.15.105:0/352544474 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740a1120 con 0x7f4a7409b410 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.791+0000 7f4a727fc640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 0x7f4a74135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00d180 tx=0x7f4a6c01eae0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3c2ef7e06e6c496d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 1 -- 172.21.15.105:0/352544474 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c0a77b0 con 0x7f4a7409b410 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: dump: 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.812 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 1 -- 172.21.15.105:0/352544474 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c0a7970 con 0x7f4a7409b410 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 5 monclient: authenticate success, global_id 74632 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: get_monmap_and_config success 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 1 -- 172.21.15.105:0/352544474 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c009600 con 0x7f4a7409b410 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 4 set_mon_vals no callback set 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:43:59.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a71ffb640 10 monclient: dump: 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.815 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: shutdown 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 msgr2=0x7f4a7409b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00d180 tx=0x7f4a6c01eae0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- 172.21.15.105:0/352544474 shutdown_connections 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4a74135e50 0x7f4a74136290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f4a740a2690 0x7f4a74135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- 172.21.15.105:0/352544474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4a7409b410 0x7f4a7409b7f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- 172.21.15.105:0/352544474 >> 172.21.15.105:0/352544474 conn(0x7f4a740a83e0 msgr2=0x7f4a740a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- 172.21.15.105:0/352544474 shutdown_connections 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- 172.21.15.105:0/352544474 wait complete. 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: monmap: 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.817 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 librados: starting msgr at 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 librados: starting objecter 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 Processor -- start 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- start start 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 librados: setting wanted keys 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 librados: calling monclient init 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: init 2024-09-17T16:43:59.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient: _add_conns ranks=[7,8,1,4,5,6,3,2,0] 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4a7409b410 0x7f4a740742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-i con 0x7f4a7409b410 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 0x7f4a74132b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-e con 0x7f4a74135e50 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a741330d0 0x7f4a7413ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): picked mon.noname-f con 0x7f4a741330d0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7409beb0 con 0x7f4a741330d0 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f4a7409bd30 con 0x7f4a7409b410 2024-09-17T16:43:59.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4a7413ec90 con 0x7f4a74135e50 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a794c7640 10 monclient(hunting): authenticate will time out at 1923.063747s 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 0x7f4a74132b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 0x7f4a74132b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33546/0 (socket says 172.21.15.105:33546) 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 1 -- 172.21.15.105:0/2029670736 learned_addr learned my addr 172.21.15.105:0/2029670736 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 10 monclient(hunting): get_auth_request con 0x7f4a74135e50 auth_method 0 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.795+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a741330d0 0x7f4a7413ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4a7409b410 0x7f4a740742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 10 monclient(hunting): get_auth_request con 0x7f4a741330d0 auth_method 0 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request con 0x7f4a7409b410 auth_method 0 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 10 monclient(hunting): handle_auth_done global_id 54881 payload 824 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4a7409b410 msgr2=0x7f4a740742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4a7409b410 0x7f4a740742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a741330d0 msgr2=0x7f4a7413ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a741330d0 0x7f4a7413ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 monclient: found mon.noname-e 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740a8a30 con 0x7f4a74135e50 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a727fc640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 0x7f4a74132b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e960 tx=0x7f4a5c00ee20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.?.objecter ms_handle_connect 0x7f4a74135e50 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a74135e50 2024-09-17T16:43:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: dump: 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c013470 con 0x7f4a74135e50 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a5c013a90 con 0x7f4a74135e50 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: dump: 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a717fa640 4 set_mon_vals no callback set 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 5 monclient: authenticate success, global_id 54881 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 monclient: _renew_subs 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a74074ab0 con 0x7f4a74135e50 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 client.54881.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 monclient: _renew_subs 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7413c4e0 con 0x7f4a74135e50 2024-09-17T16:43:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 1 librados: init done 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 0x7f4a7413b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4a74051e80 con 0x7f4a740a2690 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 0x7f4a7413b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient: get_auth_request con 0x7f4a740a2690 auth_method 0 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4a5c01a070 con 0x7f4a74135e50 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4a5403e550 0x7f4a54040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_done global_id 84341 payload 824 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(640..640 src has 1..640) ==== 6769+0+0 (secure 0 0 0) 0x7f4a5c00f040 con 0x7f4a74135e50 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.54881.objecter ms_dispatch 0x7f4a74000ba0 osd_map(640..640 src has 1..640) 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 3 client.54881.objecter handle_osd_map got epochs [640,640] > 0 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 3 client.54881.objecter handle_osd_map decoding full epoch 640 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a740a3870 con 0x7f4a740a2690 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 20 client.54881.objecter dump_active .. 0 homeless 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 0x7f4a7413b2e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00d150 tx=0x7f4a6c01b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.54881.objecter ms_handle_connect 0x7f4a740a2690 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.54881.objecter resend_mon_ops 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4a5403e550 0x7f4a54040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c0bd070 con 0x7f4a740a2690 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: dump: 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 10 monclient: get_auth_request con 0x7f4a5403e550 auth_method 0 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4a6c003bb0 con 0x7f4a740a2690 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 msgr2=0x7f4a7413b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 0x7f4a7413b2e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00d150 tx=0x7f4a6c01b880 comp rx=0 tx=0).stop 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 msgr2=0x7f4a7413b2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a740a2690 0x7f4a7413b2e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a6b7fe640 10 client.54881.objecter ms_handle_connect 0x7f4a5403e550 2024-09-17T16:43:59.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.799+0000 7f4a737fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4a5403e550 0x7f4a54040a10 secure :-1 s=READY pgs=3629 cs=0 l=1 rev1=1 crypto rx=0x7f4a740a4610 tx=0x7f4a64008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:43:59.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 0x7f4a74070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7413bc30 con 0x7f4a7413b820 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 0x7f4a74070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient: get_auth_request con 0x7f4a7413b820 auth_method 0 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 10 monclient(hunting): handle_auth_done global_id 84350 payload 824 2024-09-17T16:43:59.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 1 -- 172.21.15.105:0/2029670736 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a7406a1c0 con 0x7f4a7413b820 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 client.54881.objecter ms_handle_connect 0x7f4a7413b820 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 client.54881.objecter resend_mon_ops 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a72ffd640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 0x7f4a74070070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c01eb20 tx=0x7f4a6c0a9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c0b88b0 con 0x7f4a7413b820 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 monclient: dump: 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.834 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f4a6c0b8440 con 0x7f4a7413b820 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:43:59.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 msgr2=0x7f4a74070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 0x7f4a74070070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c01eb20 tx=0x7f4a6c0a9000 comp rx=0 tx=0).stop 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 msgr2=0x7f4a74070070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a6b7fe640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4a7413b820 0x7f4a74070070 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c01eb20 tx=0x7f4a6c0a9000 comp rx=0 tx=0).stop 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:43:59.836 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 147042, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:43:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.838 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.839 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:43:59.840 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:43:59.841 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.842 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:43:59.844 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:43:59.845 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.847 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.848 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.849 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:43:59.850 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.851 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.852 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:43:59.853 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.855 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:43:59.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.857 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:43:59.858 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:43:59.859 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 librados: watch_flush enter 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 librados: watch_flush exit 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 20 client.54881.objecter shutdown clearing up homeless session... 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 client.54881.objecter successfully canceled tick 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4a5403e550 msgr2=0x7f4a54040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4a5403e550 0x7f4a54040a10 secure :-1 s=READY pgs=3629 cs=0 l=1 rev1=1 crypto rx=0x7f4a740a4610 tx=0x7f4a64008040 comp rx=0 tx=0).stop 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 10 monclient: shutdown 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:43:59.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 msgr2=0x7f4a74132b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 --2- 172.21.15.105:0/2029670736 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4a74135e50 0x7f4a74132b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e960 tx=0x7f4a5c00ee20 comp rx=0 tx=0).stop 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a737fe640 1 -- 172.21.15.105:0/2029670736 reap_dead start 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a737fe640 1 -- 172.21.15.105:0/2029670736 reap_dead start 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 shutdown_connections 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 >> 172.21.15.105:0/2029670736 conn(0x7f4a740a83e0 msgr2=0x7f4a74133820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 shutdown_connections 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 -- 172.21.15.105:0/2029670736 wait complete. 2024-09-17T16:43:59.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.807+0000 7f4a794c7640 1 librados: shutdown 2024-09-17T16:43:59.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:43:59.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:43:59.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:43:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: get_monmap_and_config 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: build_initial_monmap 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: monmap: 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:43:59.976477+0000 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:43:59.976477+0000 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:43:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:43:59.982 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 Processor -- start 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 -- start start 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: init 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: _reopen_session rank -1 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient: _add_conns ranks=[0,8,10,3,2,4,1,9,5,6,7] 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6b00a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-e con 0x7fb6b00a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 0x7fb6b0141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-f con 0x7fb6b00a73d0 addr v2:172.21.15.160:3300/0 2024-09-17T16:43:59.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b006ebc0 con 0x7fb6b00a68e0 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b00a3fc0 con 0x7fb6b00a73d0 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb6b00a63c0 con 0x7fb6b00a4910 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): _renew_subs 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6b54d0640 10 monclient(hunting): authenticate will time out at 1923.242192s 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6aeffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb6b00a68e0 0x7fb6b00a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42236/0 (socket says 172.21.15.105:42236) 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6aeffd640 1 -- 172.21.15.105:0/436711733 learned_addr learned my addr 172.21.15.105:0/436711733 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): get_auth_request con 0x7fb6b00a4910 auth_method 0 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/436711733 >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 0x7fb6b0141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 10 monclient(hunting): get_auth_request con 0x7fb6b00a73d0 auth_method 0 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_done global_id 54890 payload 274 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 -- 172.21.15.105:0/436711733 >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 msgr2=0x7fb6b0141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:43:59.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/436711733 >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 0x7fb6b0141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 -- 172.21.15.105:0/436711733 >> v1:172.21.15.105:6789/0 conn(0x7fb6b00a68e0 legacy=0x7fb6b00a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 monclient: found mon.noname-e 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6a67fc640 1 -- 172.21.15.105:0/436711733 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b00a3690 con 0x7fb6b00a4910 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.975+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/436711733 >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 0x7fb6b0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/436711733 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b013eaf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6980097c0 tx=0x7fb69801c330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2c2a845f06079110 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:43:59.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 1 -- 172.21.15.105:0/436711733 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6980b0070 con 0x7fb6b00a4910 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: dump: 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:43:59.997 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: _finish_auth 0 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:43:59.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 5 monclient: authenticate success, global_id 54890 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 1 -- 172.21.15.105:0/436711733 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980aa810 con 0x7fb6b00a4910 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: get_monmap_and_config success 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 4 set_mon_vals no callback set 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 1 -- 172.21.15.105:0/436711733 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6980a9300 con 0x7fb6b00a4910 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6ae7fc640 10 monclient: dump: 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:43:59.999 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.000 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: shutdown 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- 172.21.15.105:0/436711733 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 msgr2=0x7fb6b013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/436711733 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b013eaf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6980097c0 tx=0x7fb69801c330 comp rx=0 tx=0).stop 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- 172.21.15.105:0/436711733 shutdown_connections 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/436711733 >> v2:172.21.15.160:3300/0 conn(0x7fb6b00a73d0 0x7fb6b0141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/436711733 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b013eaf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- 172.21.15.105:0/436711733 >> 172.21.15.105:0/436711733 conn(0x7fb6b0068ba0 msgr2=0x7fb6b0068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- 172.21.15.105:0/436711733 shutdown_connections 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- 172.21.15.105:0/436711733 wait complete. 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: monmap: 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:00.002 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 librados: starting msgr at 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 librados: starting objecter 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 Processor -- start 2024-09-17T16:44:00.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- start start 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 librados: setting wanted keys 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 librados: calling monclient init 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: init 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient: _add_conns ranks=[8,5,2,6,1,3,4,7,0] 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b0128a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-e con 0x7fb6b00a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6b00a68e0 0x7fb6b01d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-h con 0x7fb6b00a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6b00a73d0 0x7fb6b0126510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): picked mon.noname-b con 0x7fb6b00a73d0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:00.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b0143f20 con 0x7fb6b00a73d0 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb6b0143da0 con 0x7fb6b00a68e0 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fb6b01440a0 con 0x7fb6b00a4910 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6b54d0640 10 monclient(hunting): authenticate will time out at 1923.247467s 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b0128a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b0128a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33556/0 (socket says 172.21.15.105:33556) 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 -- 172.21.15.105:0/4169961663 learned_addr learned my addr 172.21.15.105:0/4169961663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): get_auth_request con 0x7fb6b00a4910 auth_method 0 2024-09-17T16:44:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6b00a73d0 0x7fb6b0126510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): get_auth_request con 0x7fb6b00a73d0 auth_method 0 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient(hunting): handle_auth_done global_id 54899 payload 824 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6b00a68e0 msgr2=0x7fb6b01d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb6b00a68e0 0x7fb6b01d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6b00a73d0 msgr2=0x7fb6b0126510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb6b00a73d0 0x7fb6b0126510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 monclient: found mon.noname-e 2024-09-17T16:44:00.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.979+0000 7fb6aeffd640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b0129f40 con 0x7fb6b00a4910 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6aeffd640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b0128a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c480 tx=0x7fb6a000c940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.?.objecter ms_handle_connect 0x7fb6b00a4910 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a0017070 con 0x7fb6b00a4910 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.008 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.009 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a0007070 con 0x7fb6b00a4910 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 5 monclient: authenticate success, global_id 54899 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 monclient: _renew_subs 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b012a220 con 0x7fb6b00a4910 2024-09-17T16:44:00.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6a0007710 con 0x7fb6b00a4910 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 client.54899.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6adffb640 4 set_mon_vals no callback set 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 monclient: _renew_subs 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b006e090 con 0x7fb6b00a4910 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 1 librados: init done 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 0x7fb6b0126a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb6b0051e80 con 0x7fb6b01d1b60 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 0x7fb6b0126a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient: get_auth_request con 0x7fb6b01d1b60 auth_method 0 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb6a00072a0 con 0x7fb6b00a4910 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_done global_id 74641 payload 824 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6b00a2580 con 0x7fb6b01d1b60 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb69003e550 0x7fb690040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 0x7fb6b0126a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0074a10 tx=0x7fb698006c40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.54899.objecter ms_handle_connect 0x7fb6b01d1b60 2024-09-17T16:44:00.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.54899.objecter resend_mon_ops 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(640..640 src has 1..640) ==== 6769+0+0 (secure 0 0 0) 0x7fb6a004f340 con 0x7fb6b00a4910 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.54899.objecter ms_dispatch 0x7fb6b0000ba0 osd_map(640..640 src has 1..640) 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 3 client.54899.objecter handle_osd_map got epochs [640,640] > 0 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 3 client.54899.objecter handle_osd_map decoding full epoch 640 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 20 client.54899.objecter dump_active .. 0 homeless 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6980b8030 con 0x7fb6b01d1b60 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.015 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.016 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb69003e550 0x7fb690040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6af7fe640 10 monclient: get_auth_request con 0x7fb69003e550 auth_method 0 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb698003620 con 0x7fb6b01d1b60 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 msgr2=0x7fb6b0126a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 0x7fb6b0126a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0074a10 tx=0x7fb698006c40 comp rx=0 tx=0).stop 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 msgr2=0x7fb6b0126a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b01d1b60 0x7fb6b0126a50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb69003e550 0x7fb690040a10 secure :-1 s=READY pgs=3630 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8007330 tx=0x7fb6a8009730 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.983+0000 7fb6a7fff640 10 client.54899.objecter ms_handle_connect 0x7fb69003e550 2024-09-17T16:44:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 0x7fb6b0122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb6b0127370 con 0x7fb6b0126f90 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 0x7fb6b0122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient: get_auth_request con 0x7fb6b0126f90 auth_method 0 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 10 monclient(hunting): handle_auth_done global_id 74650 payload 824 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 1 -- 172.21.15.105:0/4169961663 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb6b01293b0 con 0x7fb6b0126f90 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 client.54899.objecter ms_handle_connect 0x7fb6b0126f90 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a67fc640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 0x7fb6b0122530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a9e0 tx=0x7fb6980a8000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 client.54899.objecter resend_mon_ops 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6980b8040 con 0x7fb6b0126f90 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 monclient: dump: 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9622 (secure 0 0 0) 0x7fb6980b4430 con 0x7fb6b0126f90 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 msgr2=0x7fb6b0122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 0x7fb6b0122530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a9e0 tx=0x7fb6980a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 msgr2=0x7fb6b0122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6a7fff640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb6b0126f90 0x7fb6b0122530 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a9e0 tx=0x7fb6980a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.023 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 279453, 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:00.024 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:00.025 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.026 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.027 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:00.029 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.030 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:00.031 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.032 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.034 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.036 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:00.037 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.038 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.039 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.040 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.041 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.042 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.045 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:00.046 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.047 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.049 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.050 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.051 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.052 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:00.053 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 librados: watch_flush enter 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 librados: watch_flush exit 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 20 client.54899.objecter shutdown clearing up homeless session... 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 client.54899.objecter successfully canceled tick 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb69003e550 msgr2=0x7fb690040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb69003e550 0x7fb690040a10 secure :-1 s=READY pgs=3630 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8007330 tx=0x7fb6a8009730 comp rx=0 tx=0).stop 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 10 monclient: shutdown 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 msgr2=0x7fb6b0128a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 --2- 172.21.15.105:0/4169961663 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fb6b00a4910 0x7fb6b0128a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c480 tx=0x7fb6a000c940 comp rx=0 tx=0).stop 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6af7fe640 1 -- 172.21.15.105:0/4169961663 reap_dead start 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6af7fe640 1 -- 172.21.15.105:0/4169961663 reap_dead start 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 shutdown_connections 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 >> 172.21.15.105:0/4169961663 conn(0x7fb6b0068ba0 msgr2=0x7fb6b00a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 shutdown_connections 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 -- 172.21.15.105:0/4169961663 wait complete. 2024-09-17T16:44:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:43:59.991+0000 7fb6b54d0640 1 librados: shutdown 2024-09-17T16:44:00.057 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:00.057 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:44:00.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:44:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: get_monmap_and_config 2024-09-17T16:44:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: monmap: 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:00.168222+0000 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:00.168222+0000 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:00.174 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 Processor -- start 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 -- start start 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: init 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient: _add_conns ranks=[2,3,4,6,1,5,9,7,0,8,10] 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-b con 0x7f9b5c0a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 0x7f9b5c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-g con 0x7f9b5c0a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-c con 0x7f9b5c1396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c0a3fc0 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5c0a63c0 con 0x7f9b5c0a4910 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9b5c0a7740 con 0x7f9b5c1396a0 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b62cc3640 10 monclient(hunting): authenticate will time out at 1923.433927s 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47120/0 (socket says 172.21.15.105:47120) 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 -- 172.21.15.105:0/1177769452 learned_addr learned my addr 172.21.15.105:0/1177769452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34064/0 (socket says 172.21.15.105:34064) 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request con 0x7f9b5c0a68e0 auth_method 0 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): get_auth_request con 0x7f9b5c1396a0 auth_method 0 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b5bfff640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 0x7f9b5c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_done global_id 84368 payload 274 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 -- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 msgr2=0x7f9b5c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 -- 172.21.15.105:0/1177769452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 msgr2=0x7f9b5c0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 0x7f9b5c0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 monclient: found mon.noname-b 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b60a5f640 1 -- 172.21.15.105:0/1177769452 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c06ec10 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b5bfff640 10 monclient: get_auth_request con 0x7f9b5c0a4910 auth_method 0 2024-09-17T16:44:00.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b5bfff640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 0x7f9b5c0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.167+0000 7f9b61260640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098d0 tx=0x7f9b4401cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fe808d655b4d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 1 -- 172.21.15.105:0/1177769452 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b440b0070 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: dump: 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 1 -- 172.21.15.105:0/1177769452 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b440aa810 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 5 monclient: authenticate success, global_id 84368 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 1 -- 172.21.15.105:0/1177769452 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b440a9300 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 4 set_mon_vals no callback set 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5b7fe640 10 monclient: dump: 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: shutdown 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 msgr2=0x7f9b5c0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098d0 tx=0x7f9b4401cea0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- 172.21.15.105:0/1177769452 shutdown_connections 2024-09-17T16:44:00.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9b5c1396a0 0x7f9b5c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9b5c0a4910 0x7f9b5c0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/1177769452 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b5c0a68e0 0x7f9b5c0a6cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- 172.21.15.105:0/1177769452 >> 172.21.15.105:0/1177769452 conn(0x7f9b5c068ba0 msgr2=0x7f9b5c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- 172.21.15.105:0/1177769452 shutdown_connections 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- 172.21.15.105:0/1177769452 wait complete. 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: monmap: 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 librados: starting msgr at 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 librados: starting objecter 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 Processor -- start 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- start start 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 librados: setting wanted keys 2024-09-17T16:44:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 librados: calling monclient init 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: init 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient: _add_conns ranks=[5,6,7,3,4,0,1,2,8] 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9b5c0a4910 0x7f9b5c1d1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-h con 0x7f9b5c0a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-d con 0x7f9b5c0a68e0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b5c1396a0 0x7f9b5c13d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): picked mon.noname-i con 0x7f9b5c1396a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f9b5c143320 con 0x7f9b5c0a4910 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9b5c06e630 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9b5c1434a0 con 0x7f9b5c1396a0 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b62cc3640 10 monclient(hunting): authenticate will time out at 1923.439190s 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39214/0 (socket says 172.21.15.105:39214) 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 -- 172.21.15.105:0/3548035395 learned_addr learned my addr 172.21.15.105:0/3548035395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request con 0x7f9b5c0a68e0 auth_method 0 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9b5c0a4910 0x7f9b5c1d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b61260640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b5c1396a0 0x7f9b5c13d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request con 0x7f9b5c0a4910 auth_method 0 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b60a5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b61260640 10 monclient(hunting): get_auth_request con 0x7f9b5c1396a0 auth_method 0 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b61260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b61260640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b61260640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient(hunting): handle_auth_done global_id 94137 payload 824 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b5c1396a0 msgr2=0x7f9b5c13d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f9b5c1396a0 0x7f9b5c13d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9b5c0a4910 msgr2=0x7f9b5c1d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f9b5c0a4910 0x7f9b5c1d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 monclient: found mon.noname-d 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c123730 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b5bfff640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c820 tx=0x7f9b4c00cce0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b597fa640 10 client.?.objecter ms_handle_connect 0x7f9b5c0a68e0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.171+0000 7f9b597fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b4c0044a0 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: dump: 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.198 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.199 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 5 monclient: authenticate success, global_id 94137 2024-09-17T16:44:00.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4c007830 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 monclient: _renew_subs 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c123a10 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b4c007e50 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b5affd640 4 set_mon_vals no callback set 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 client.94137.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: dump: 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.201 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 monclient: _renew_subs 2024-09-17T16:44:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c13fd00 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 1 librados: init done 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 0x7f9b5c13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9b5c123e40 con 0x7f9b5c13da10 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 0x7f9b5c13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient: get_auth_request con 0x7f9b5c13da10 auth_method 0 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9b4c0079e0 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_done global_id 94146 payload 824 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b5c0aa0b0 con 0x7f9b5c13da10 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 0x7f9b5c13ddf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098a0 tx=0x7f9b440191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b3c03e6f0 0x7f9b3c040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 client.94137.objecter ms_handle_connect 0x7f9b5c13da10 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 client.94137.objecter resend_mon_ops 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(640..640 src has 1..640) ==== 6769+0+0 (secure 0 0 0) 0x7f9b4c0502c0 con 0x7f9b5c0a68e0 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 client.94137.objecter ms_dispatch 0x7f9b5c000ba0 osd_map(640..640 src has 1..640) 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 3 client.94137.objecter handle_osd_map got epochs [640,640] > 0 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 3 client.94137.objecter handle_osd_map decoding full epoch 640 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 20 client.94137.objecter dump_active .. 0 homeless 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b440b9030 con 0x7f9b5c13da10 2024-09-17T16:44:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: dump: 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b61260640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b3c03e6f0 0x7f9b3c040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b61260640 10 monclient: get_auth_request con 0x7f9b3c03e6f0 auth_method 0 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9b440b1040 con 0x7f9b5c13da10 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 msgr2=0x7f9b5c13ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 0x7f9b5c13ddf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098a0 tx=0x7f9b440191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 msgr2=0x7f9b5c13ddf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13da10 0x7f9b5c13ddf0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b440098a0 tx=0x7f9b440191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b597fa640 10 client.94137.objecter ms_handle_connect 0x7f9b3c03e6f0 2024-09-17T16:44:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.175+0000 7f9b61260640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b3c03e6f0 0x7f9b3c040bb0 secure :-1 s=READY pgs=3631 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c074620 tx=0x7f9b5400b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 0x7f9b5c0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9b5c051e80 con 0x7f9b5c13e330 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 0x7f9b5c0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient: get_auth_request con 0x7f9b5c13e330 auth_method 0 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 10 monclient(hunting): handle_auth_done global_id 94155 payload 824 2024-09-17T16:44:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 1 -- 172.21.15.105:0/3548035395 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9b5c0abbe0 con 0x7f9b5c13e330 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b60a5f640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 0x7f9b5c0a0a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b4401c270 tx=0x7f9b440a6000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 client.94137.objecter ms_handle_connect 0x7f9b5c13e330 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 client.94137.objecter resend_mon_ops 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b440b9070 con 0x7f9b5c13e330 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 monclient: dump: 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f9b44003e40 con 0x7f9b5c13e330 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 msgr2=0x7f9b5c0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 0x7f9b5c0a0a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b4401c270 tx=0x7f9b440a6000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 msgr2=0x7f9b5c0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b597fa640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c13e330 0x7f9b5c0a0a20 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b4401c270 tx=0x7f9b440a6000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 16027, 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:00.212 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.213 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.214 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:00.215 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:00.216 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:00.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.218 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.220 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.221 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.222 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.226 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:00.227 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.229 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.231 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.232 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:00.233 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 librados: watch_flush enter 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 librados: watch_flush exit 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 20 client.94137.objecter shutdown clearing up homeless session... 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 client.94137.objecter successfully canceled tick 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b3c03e6f0 msgr2=0x7f9b3c040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9b3c03e6f0 0x7f9b3c040bb0 secure :-1 s=READY pgs=3631 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c074620 tx=0x7f9b5400b040 comp rx=0 tx=0).stop 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 10 monclient: shutdown 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 msgr2=0x7f9b5c1d1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c820 tx=0x7f9b4c00cce0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b61260640 1 -- 172.21.15.105:0/3548035395 reap_dead start 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 shutdown_connections 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 --2- 172.21.15.105:0/3548035395 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9b5c0a68e0 0x7f9b5c1d1bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 >> 172.21.15.105:0/3548035395 conn(0x7f9b5c068ba0 msgr2=0x7f9b5c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 shutdown_connections 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 -- 172.21.15.105:0/3548035395 wait complete. 2024-09-17T16:44:00.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.183+0000 7f9b62cc3640 1 librados: shutdown 2024-09-17T16:44:00.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:00.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:44:00.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: get_monmap_and_config 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: monmap: 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:00.355890+0000 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:00.355890+0000 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.360 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:00.361 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 Processor -- start 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 -- start start 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: init 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient: _add_conns ranks=[1,5,0,9,7,10,3,4,2,8,6] 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7faa5809f8e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7faa580a46f0 0x7faa5809b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-h con 0x7faa580a46f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faa5813b980 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:00.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa5809f3f0 con 0x7faa5813b980 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa580a27c0 con 0x7faa5809f8e0 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7faa580a1a50 con 0x7faa580a46f0 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 10 monclient(hunting): authenticate will time out at 1923.621788s 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faa5813b980 0x7faa5813dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42250/0 (socket says 172.21.15.105:42250) 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 1 -- 172.21.15.105:0/1707181474 learned_addr learned my addr 172.21.15.105:0/1707181474 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7faa580a46f0 0x7faa5809b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): get_auth_request con 0x7faa580a46f0 auth_method 0 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373216212 0 0) 0x7faa48001120 con 0x7faa5813b980 2024-09-17T16:44:00.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _negotiate 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): my global_id is 36513 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa44003530 con 0x7faa5813b980 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644695305 0 0) 0x7faa580a27c0 con 0x7faa5809f8e0 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _negotiate 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): my global_id is 84322 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa48001120 con 0x7faa5809f8e0 2024-09-17T16:44:00.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1525921686 0 0) 0x7faa44003530 con 0x7faa5813b980 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7faa580a46f0 msgr2=0x7faa5809b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7faa580a46f0 0x7faa5809b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 >> v1:172.21.15.160:6789/0 conn(0x7faa5809f8e0 legacy=0x7faa5809b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5809bca0 con 0x7faa5813b980 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 5 monclient: authenticate success, global_id 36513 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7faa48001b40 con 0x7faa5813b980 2024-09-17T16:44:00.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: dump: 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.377 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 >> v1:172.21.15.105:6789/0 conn(0x7faa5813b980 legacy=0x7faa5813dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient: _add_conns ranks=[8,7,4,3,2,5,0,1,6] 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 0x7faa44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): picked mon.e con 0x7faa440043d0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faa44004f20 0x7faa440099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): picked mon.i con 0x7faa44004f20 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:00.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faa44009f00 0x7faa4400c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): picked mon.c con 0x7faa44009f00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7faa580a1a50 con 0x7faa44009f00 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7faa4400c9c0 con 0x7faa44004f20 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7faa4400cb70 con 0x7faa440043d0 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa55ffb640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa5d56d640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 0x7faa44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): get_auth_request con 0x7faa440043d0 auth_method 0 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faa44009f00 0x7faa4400c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): get_auth_request con 0x7faa44009f00 auth_method 0 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faa44004f20 0x7faa440099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): get_auth_request con 0x7faa44004f20 auth_method 0 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa567fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient(hunting): handle_auth_done global_id 36513 payload 293 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 -- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faa44004f20 msgr2=0x7faa440099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faa44004f20 0x7faa440099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 -- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faa44009f00 msgr2=0x7faa4400c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faa44009f00 0x7faa4400c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 monclient: found mon.e 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:00.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 1 -- 172.21.15.105:0/1707181474 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa44003530 con 0x7faa440043d0 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.355+0000 7faa577fe640 1 -- 172.21.15.105:0/1707181474 reap_dead start 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa56ffd640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 0x7faa44004810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa400020b0 tx=0x7faa4001c8e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6330fd973def40a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa40001c50 con 0x7faa440043d0 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: dump: 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.382 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa40003df0 con 0x7faa440043d0 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 4 set_mon_vals no callback set 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 1 -- 172.21.15.105:0/1707181474 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa40005540 con 0x7faa440043d0 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa55ffb640 10 monclient: dump: 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: shutdown 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 msgr2=0x7faa44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 0x7faa44004810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa400020b0 tx=0x7faa4001c8e0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- 172.21.15.105:0/1707181474 shutdown_connections 2024-09-17T16:44:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 --2- 172.21.15.105:0/1707181474 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa440043d0 0x7faa44004810 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- 172.21.15.105:0/1707181474 >> 172.21.15.105:0/1707181474 conn(0x7faa580a83e0 msgr2=0x7faa580a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- 172.21.15.105:0/1707181474 shutdown_connections 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- 172.21.15.105:0/1707181474 wait complete. 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: monmap: 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 librados: starting msgr at 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 librados: starting objecter 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 Processor -- start 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- start start 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 librados: setting wanted keys 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 librados: calling monclient init 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: init 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient: _add_conns ranks=[0,1,6,8,7,3,4,5,2] 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa5812e4f0 0x7faa5812e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-a con 0x7faa5812e4f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa5812ee10 0x7faa581d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-f con 0x7faa5812ee10 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 0x7faa581d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): picked mon.noname-d con 0x7faa581d1190 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faa5809ba70 con 0x7faa5812e4f0 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faa5806b5b0 con 0x7faa5812ee10 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7faa580a0a20 con 0x7faa581d1190 2024-09-17T16:44:00.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa5d56d640 10 monclient(hunting): authenticate will time out at 1923.627787s 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 0x7faa581d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 0x7faa581d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39226/0 (socket says 172.21.15.105:39226) 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 1 -- 172.21.15.105:0/801187301 learned_addr learned my addr 172.21.15.105:0/801187301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 10 monclient(hunting): get_auth_request con 0x7faa581d1190 auth_method 0 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa56ffd640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa5812e4f0 0x7faa5812e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.359+0000 7faa577fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): get_auth_request con 0x7faa5812e4f0 auth_method 0 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa5812ee10 0x7faa581d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): get_auth_request con 0x7faa5812ee10 auth_method 0 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient(hunting): handle_auth_done global_id 36522 payload 824 2024-09-17T16:44:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 msgr2=0x7faa581d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 0x7faa581d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa5812ee10 msgr2=0x7faa581d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa5812ee10 0x7faa581d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 monclient: found mon.noname-a 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa58134b10 con 0x7faa5812e4f0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faa581d1190 0x7faa581d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa56ffd640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa5812e4f0 0x7faa5812e8d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa40004000 tx=0x7faa44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 client.?.objecter ms_handle_connect 0x7faa5812e4f0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa40007da0 con 0x7faa5812e4f0 2024-09-17T16:44:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: dump: 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa400b4070 con 0x7faa5812e4f0 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 5 monclient: authenticate success, global_id 36522 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 monclient: _renew_subs 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa58074da0 con 0x7faa5812e4f0 2024-09-17T16:44:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa40005710 con 0x7faa5812e4f0 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa557fa640 4 set_mon_vals no callback set 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 client.36522.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 monclient: dump: 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.394 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 monclient: _renew_subs 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa58134df0 con 0x7faa5812e4f0 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 1 librados: init done 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7faa58051e80 con 0x7faa58072080 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient: get_auth_request con 0x7faa58072080 auth_method 0 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7faa400bf060 con 0x7faa5812e4f0 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 msgr2=0x7faa58072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 msgr2=0x7faa58072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faa3803e540 0x7faa38040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(640..640 src has 1..640) ==== 6769+0+0 (secure 0 0 0) 0x7faa400ea190 con 0x7faa5812e4f0 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 client.36522.objecter ms_dispatch 0x7faa58000ba0 osd_map(640..640 src has 1..640) 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 3 client.36522.objecter handle_osd_map got epochs [640,640] > 0 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 3 client.36522.objecter handle_osd_map decoding full epoch 640 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 20 client.36522.objecter dump_active .. 0 homeless 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faa3803e540 0x7faa38040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 10 monclient: get_auth_request con 0x7faa3803e540 auth_method 0 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa4f7fe640 10 client.36522.objecter ms_handle_connect 0x7faa3803e540 2024-09-17T16:44:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.363+0000 7faa577fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faa3803e540 0x7faa38040a00 secure :-1 s=READY pgs=3632 cs=0 l=1 rev1=1 crypto rx=0x7faa48000c10 tx=0x7faa48001b20 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient: get_auth_request con 0x7faa58072080 auth_method 0 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 10 monclient(hunting): handle_auth_done global_id 54908 payload 824 2024-09-17T16:44:00.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa5809fcc0 con 0x7faa58072080 2024-09-17T16:44:00.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faa50016740 tx=0x7faa50016c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa4f7fe640 10 client.36522.objecter ms_handle_connect 0x7faa58072080 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.563+0000 7faa4f7fe640 10 client.36522.objecter resend_mon_ops 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa50014db0 con 0x7faa58072080 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 10 monclient: dump: 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.573 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faa5001b070 con 0x7faa58072080 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:00.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 msgr2=0x7faa58072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faa50016740 tx=0x7faa50016c00 comp rx=0 tx=0).stop 2024-09-17T16:44:00.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 msgr2=0x7faa58072460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.567+0000 7faa4f7fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa58072080 0x7faa58072460 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faa50016740 tx=0x7faa50016c00 comp rx=0 tx=0).stop 2024-09-17T16:44:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 0x7faa581dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7faa5806a1c0 con 0x7faa580729a0 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 0x7faa581dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient: get_auth_request con 0x7faa580729a0 auth_method 0 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 10 monclient(hunting): handle_auth_done global_id 54917 payload 824 2024-09-17T16:44:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 1 -- 172.21.15.105:0/801187301 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faa580a1ca0 con 0x7faa580729a0 2024-09-17T16:44:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa567fc640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 0x7faa581dbf30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faa50000c10 tx=0x7faa50004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 client.36522.objecter ms_handle_connect 0x7faa580729a0 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 client.36522.objecter resend_mon_ops 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faa50003c00 con 0x7faa580729a0 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 monclient: dump: 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9457 (secure 0 0 0) 0x7faa50003c00 con 0x7faa580729a0 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 msgr2=0x7faa581dbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 0x7faa581dbf30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faa50000c10 tx=0x7faa50004740 comp rx=0 tx=0).stop 2024-09-17T16:44:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 msgr2=0x7faa581dbf30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa4f7fe640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7faa580729a0 0x7faa581dbf30 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faa50000c10 tx=0x7faa50004740 comp rx=0 tx=0).stop 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 547572, 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:00.586 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:00.587 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:00.588 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.589 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:00.590 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.591 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.593 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.594 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.596 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.598 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.599 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:00.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.603 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.603 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.603 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.603 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.604 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.606 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:00.607 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:00.608 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:00.609 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:00.610 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 librados: watch_flush enter 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 librados: watch_flush exit 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 20 client.36522.objecter shutdown clearing up homeless session... 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 client.36522.objecter successfully canceled tick 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faa3803e540 msgr2=0x7faa38040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7faa3803e540 0x7faa38040a00 secure :-1 s=READY pgs=3632 cs=0 l=1 rev1=1 crypto rx=0x7faa48000c10 tx=0x7faa48001b20 comp rx=0 tx=0).stop 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 10 monclient: shutdown 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa5812e4f0 msgr2=0x7faa5812e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 --2- 172.21.15.105:0/801187301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa5812e4f0 0x7faa5812e8d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa40004000 tx=0x7faa44005840 comp rx=0 tx=0).stop 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa577fe640 1 -- 172.21.15.105:0/801187301 reap_dead start 2024-09-17T16:44:00.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa577fe640 1 -- 172.21.15.105:0/801187301 reap_dead start 2024-09-17T16:44:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 shutdown_connections 2024-09-17T16:44:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.575+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 >> 172.21.15.105:0/801187301 conn(0x7faa580a83e0 msgr2=0x7faa5809c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.579+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 shutdown_connections 2024-09-17T16:44:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.579+0000 7faa5d56d640 1 -- 172.21.15.105:0/801187301 wait complete. 2024-09-17T16:44:00.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.579+0000 7faa5d56d640 1 librados: shutdown 2024-09-17T16:44:00.613 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:00.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:44:00.613 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: get_monmap_and_config 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: monmap: 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:00.735934+0000 2024-09-17T16:44:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:00.735934+0000 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:00.742 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 Processor -- start 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 -- start start 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: init 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient: _add_conns ranks=[8,1,5,7,2,3,10,4,9,6,0] 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-e con 0x7f26bc0a5960 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f26bc0a42c0 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a46a0 0x7f26bc145c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-h con 0x7f26bc0a46a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26bc069500 con 0x7f26bc0a42c0 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f26bc0a5d40 con 0x7f26bc0a46a0 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f26bc09d640 con 0x7f26bc0a5960 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c32db640 10 monclient(hunting): authenticate will time out at 1924.001711s 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33596/0 (socket says 172.21.15.105:33596) 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 -- 172.21.15.105:0/300405264 learned_addr learned my addr 172.21.15.105:0/300405264 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): get_auth_request con 0x7f26bc0a5960 auth_method 0 2024-09-17T16:44:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient(hunting): handle_auth_done global_id 54926 payload 274 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 -- 172.21.15.105:0/300405264 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a46a0 msgr2=0x7f26bc145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 --2- 172.21.15.105:0/300405264 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a46a0 0x7f26bc145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 -- 172.21.15.105:0/300405264 >> v1:172.21.15.160:6789/0 conn(0x7f26bc0a42c0 legacy=0x7f26bc142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 monclient: found mon.noname-e 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.735+0000 7f26c1077640 1 -- 172.21.15.105:0/300405264 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc09d360 con 0x7f26bc0a5960 2024-09-17T16:44:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c1077640 1 --2- 172.21.15.105:0/300405264 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26b80098d0 tx=0x7f26b801c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=11a36b343918ecc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 1 -- 172.21.15.105:0/300405264 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26b80b0070 con 0x7f26bc0a5960 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: dump: 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 1 -- 172.21.15.105:0/300405264 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80aa810 con 0x7f26bc0a5960 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 1 -- 172.21.15.105:0/300405264 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26b80a9300 con 0x7f26bc0a5960 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 5 monclient: authenticate success, global_id 54926 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 4 set_mon_vals no callback set 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26b3fff640 10 monclient: dump: 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: shutdown 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- 172.21.15.105:0/300405264 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 msgr2=0x7f26bc0a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- 172.21.15.105:0/300405264 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26b80098d0 tx=0x7f26b801c410 comp rx=0 tx=0).stop 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- 172.21.15.105:0/300405264 shutdown_connections 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- 172.21.15.105:0/300405264 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a46a0 0x7f26bc145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- 172.21.15.105:0/300405264 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f26bc0a5960 0x7f26bc0a3d80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- 172.21.15.105:0/300405264 >> 172.21.15.105:0/300405264 conn(0x7f26bc0a4e70 msgr2=0x7f26bc0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- 172.21.15.105:0/300405264 shutdown_connections 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- 172.21.15.105:0/300405264 wait complete. 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: monmap: 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 librados: starting msgr at 2024-09-17T16:44:00.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 librados: starting objecter 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 Processor -- start 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- start start 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 librados: setting wanted keys 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 librados: calling monclient init 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: init 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient: _add_conns ranks=[7,4,5,3,8,0,2,6,1] 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26bc0a42c0 0x7f26bc1d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-i con 0x7f26bc0a42c0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:00.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 0x7f26bc1d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-c con 0x7f26bc0a46a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a5960 0x7f26bc0983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): picked mon.noname-h con 0x7f26bc0a5960 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f26bc09e210 con 0x7f26bc0a46a0 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f26bc09e090 con 0x7f26bc0a5960 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f26bc09c370 con 0x7f26bc0a42c0 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c32db640 10 monclient(hunting): authenticate will time out at 1924.007814s 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c0876640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 0x7f26bc1d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.739+0000 7f26c0876640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 0x7f26bc1d5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34084/0 (socket says 172.21.15.105:34084) 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 -- 172.21.15.105:0/2109141346 learned_addr learned my addr 172.21.15.105:0/2109141346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): get_auth_request con 0x7f26bc0a46a0 auth_method 0 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a5960 0x7f26bc0983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26bc0a42c0 0x7f26bc1d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient(hunting): handle_auth_done global_id 74668 payload 824 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26bc0a42c0 msgr2=0x7f26bc1d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26bc0a42c0 0x7f26bc1d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a5960 msgr2=0x7f26bc0983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a5960 0x7f26bc0983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 monclient: found mon.noname-c 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc0a3aa0 con 0x7f26bc0a46a0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 10 monclient: get_auth_request con 0x7f26bc0a5960 auth_method 0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient: get_auth_request con 0x7f26bc0a42c0 auth_method 0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f26bc0a5960 0x7f26bc0983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f26bc0a42c0 0x7f26bc1d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c0876640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 0x7f26bc1d5f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c490 tx=0x7f26ac001e80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.?.objecter ms_handle_connect 0x7f26bc0a46a0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26ac017070 con 0x7f26bc0a46a0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: dump: 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 5 monclient: authenticate success, global_id 74668 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 monclient: _renew_subs 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc0706d0 con 0x7f26bc0a46a0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ac0052e0 con 0x7f26bc0a46a0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26ac003a70 con 0x7f26bc0a46a0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b37fe640 4 set_mon_vals no callback set 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 client.74668.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: dump: 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 monclient: _renew_subs 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc0a3550 con 0x7f26bc0a46a0 2024-09-17T16:44:00.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 1 librados: init done 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 0x7f26bc098cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc070070 con 0x7f26bc0988f0 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f26ac003c10 con 0x7f26bc0a46a0 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 0x7f26bc098cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient: get_auth_request con 0x7f26bc0988f0 auth_method 0 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26a003e550 0x7f26a0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(641..641 src has 1..641) ==== 6769+0+0 (secure 0 0 0) 0x7f26ac04f400 con 0x7f26bc0a46a0 2024-09-17T16:44:00.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.74668.objecter ms_dispatch 0x7f26bc000ba0 osd_map(641..641 src has 1..641) 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 3 client.74668.objecter handle_osd_map got epochs [641,641] > 0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 3 client.74668.objecter handle_osd_map decoding full epoch 641 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 20 client.74668.objecter dump_active .. 0 homeless 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26a003e550 0x7f26a0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 10 monclient: get_auth_request con 0x7f26a003e550 auth_method 0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 10 monclient(hunting): handle_auth_done global_id 84340 payload 824 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26bc0a3d80 con 0x7f26bc0988f0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.74668.objecter ms_handle_connect 0x7f26bc0988f0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.74668.objecter resend_mon_ops 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 0x7f26bc098cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26b80ad000 tx=0x7f26b80191b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26b80b9020 con 0x7f26bc0988f0 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 monclient: dump: 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.772 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26b1ffb640 10 client.74668.objecter ms_handle_connect 0x7f26a003e550 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.743+0000 7f26c1878640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26a003e550 0x7f26a0040a10 secure :-1 s=READY pgs=3633 cs=0 l=1 rev1=1 crypto rx=0x7f26b4007e30 tx=0x7f26b4007340 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f26b80b1040 con 0x7f26bc0988f0 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 msgr2=0x7f26bc098cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 0x7f26bc098cd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26b80ad000 tx=0x7f26b80191b0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 msgr2=0x7f26bc098cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.747+0000 7f26b1ffb640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc0988f0 0x7f26bc098cd0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26b80ad000 tx=0x7f26b80191b0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c32db640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:00.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c32db640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 0x7f26bc142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c32db640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc051e80 con 0x7f26bc099210 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 0x7f26bc142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient: get_auth_request con 0x7f26bc099210 auth_method 0 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 10 monclient(hunting): handle_auth_done global_id 84349 payload 824 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 1 -- 172.21.15.105:0/2109141346 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f26bc06a1c0 con 0x7f26bc099210 2024-09-17T16:44:00.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 10 client.74668.objecter ms_handle_connect 0x7f26bc099210 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26c1077640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 0x7f26bc142ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26b80098a0 tx=0x7f26b80a8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 10 client.74668.objecter resend_mon_ops 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26b80b9040 con 0x7f26bc099210 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.751+0000 7f26b1ffb640 10 monclient: dump: 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.777 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f26b80c0860 con 0x7f26bc099210 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 msgr2=0x7f26bc142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 0x7f26bc142ce0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26b80098a0 tx=0x7f26b80a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 msgr2=0x7f26bc142ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26b1ffb640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26bc099210 0x7f26bc142ce0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26b80098a0 tx=0x7f26b80a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 147981, 2024-09-17T16:44:00.779 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.780 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.781 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.782 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:00.783 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.784 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:00.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.789 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:00.790 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.791 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.794 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.796 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.798 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.799 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:00.800 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:00.801 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 10 librados: watch_flush enter 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 10 librados: watch_flush exit 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 20 client.74668.objecter shutdown clearing up homeless session... 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 10 client.74668.objecter successfully canceled tick 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26a003e550 msgr2=0x7f26a0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f26a003e550 0x7f26a0040a10 secure :-1 s=READY pgs=3633 cs=0 l=1 rev1=1 crypto rx=0x7f26b4007e30 tx=0x7f26b4007340 comp rx=0 tx=0).stop 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 10 monclient: shutdown 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 msgr2=0x7f26bc1d5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 --2- 172.21.15.105:0/2109141346 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f26bc0a46a0 0x7f26bc1d5f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c490 tx=0x7f26ac001e80 comp rx=0 tx=0).stop 2024-09-17T16:44:00.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c1878640 1 -- 172.21.15.105:0/2109141346 reap_dead start 2024-09-17T16:44:00.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c1878640 1 -- 172.21.15.105:0/2109141346 reap_dead start 2024-09-17T16:44:00.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 shutdown_connections 2024-09-17T16:44:00.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 >> 172.21.15.105:0/2109141346 conn(0x7f26bc0a4e70 msgr2=0x7f26bc1419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 shutdown_connections 2024-09-17T16:44:00.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 -- 172.21.15.105:0/2109141346 wait complete. 2024-09-17T16:44:00.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.755+0000 7f26c32db640 1 librados: shutdown 2024-09-17T16:44:00.805 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:00.805 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:44:00.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: get_monmap_and_config 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: monmap: 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:00.920015+0000 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:00.920015+0000 2024-09-17T16:44:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:00.926 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 Processor -- start 2024-09-17T16:44:00.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 -- start start 2024-09-17T16:44:00.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: init 2024-09-17T16:44:00.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient: _add_conns ranks=[4,0,3,2,9,1,5,7,10,6,8] 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 0x7f22b80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-c con 0x7f22b80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f22b80a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 0x7f22b8139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-g con 0x7f22b80a7200 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22b806ebc0 con 0x7f22b80a4910 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f22b80a3fc0 con 0x7f22b80a7200 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f22b80a63c0 con 0x7f22b80a68e0 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22bdf27640 10 monclient(hunting): authenticate will time out at 1924.185781s 2024-09-17T16:44:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 0x7f22b80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b6ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f22b80a4910 0x7f22b81396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42266/0 (socket says 172.21.15.105:42266) 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): get_auth_request con 0x7f22b80a68e0 auth_method 0 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b6ffd640 1 -- 172.21.15.105:0/1981746655 learned_addr learned my addr 172.21.15.105:0/1981746655 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient(hunting): handle_auth_done global_id 74677 payload 274 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 -- 172.21.15.105:0/1981746655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 msgr2=0x7f22b8139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b7fff640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 0x7f22b8139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b67fc640 1 -- 172.21.15.105:0/1981746655 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096356113 0 0) 0x7f22b806ebc0 con 0x7f22b80a4910 2024-09-17T16:44:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 0x7f22b8139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 -- 172.21.15.105:0/1981746655 >> v1:172.21.15.105:6789/0 conn(0x7f22b80a4910 legacy=0x7f22b81396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 monclient: found mon.noname-c 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b77fe640 1 -- 172.21.15.105:0/1981746655 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b80a3690 con 0x7f22b80a68e0 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b67fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b7fff640 10 monclient: get_auth_request con 0x7f22b80a7200 auth_method 0 2024-09-17T16:44:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.919+0000 7f22b7fff640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 0x7f22b8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b77fe640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 0x7f22b80a6cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097f0 tx=0x7f22a801c370 comp rx=0 tx=0).ready entity=mon.4 client_cookie=969df0495b9f77c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 1 -- 172.21.15.105:0/1981746655 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a80b0070 con 0x7f22b80a68e0 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: dump: 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.939 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 1 -- 172.21.15.105:0/1981746655 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a80aa810 con 0x7f22b80a68e0 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 5 monclient: authenticate success, global_id 74677 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 4 set_mon_vals no callback set 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 1 -- 172.21.15.105:0/1981746655 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a80a9300 con 0x7f22b80a68e0 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b67fc640 10 monclient: dump: 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.941 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.942 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: shutdown 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/1981746655 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 msgr2=0x7f22b80a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 0x7f22b80a6cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22a80097f0 tx=0x7f22a801c370 comp rx=0 tx=0).stop 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/1981746655 shutdown_connections 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b80a7200 0x7f22b8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- 172.21.15.105:0/1981746655 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f22b80a68e0 0x7f22b80a6cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/1981746655 >> 172.21.15.105:0/1981746655 conn(0x7f22b8068ba0 msgr2=0x7f22b8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/1981746655 shutdown_connections 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/1981746655 wait complete. 2024-09-17T16:44:00.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: build_initial_monmap 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: monmap: 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:00.944 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 librados: starting msgr at 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 librados: starting objecter 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 Processor -- start 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- start start 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 librados: setting wanted keys 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 librados: calling monclient init 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: init 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _add_conns ranks=[5,0,1,6,2,7,4,8,3] 2024-09-17T16:44:00.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f22b80a4910 0x7f22b81403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-h con 0x7f22b80a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-a con 0x7f22b80a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22b80a7200 0x7f22b813e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): picked mon.noname-f con 0x7f22b80a7200 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22b813a2f0 con 0x7f22b80a68e0 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f22b80a7610 con 0x7f22b80a7200 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f22b813a470 con 0x7f22b80a4910 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient(hunting): authenticate will time out at 1924.190908s 2024-09-17T16:44:00.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36320/0 (socket says 172.21.15.105:36320) 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 -- 172.21.15.105:0/3529708605 learned_addr learned my addr 172.21.15.105:0/3529708605 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): get_auth_request con 0x7f22b80a68e0 auth_method 0 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b7fff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22b80a7200 0x7f22b813e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b77fe640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f22b80a4910 0x7f22b81403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient(hunting): handle_auth_done global_id 36540 payload 824 2024-09-17T16:44:00.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f22b80a4910 msgr2=0x7f22b81403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f22b80a4910 0x7f22b81403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22b80a7200 msgr2=0x7f22b813e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22b80a7200 0x7f22b813e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 monclient: found mon.noname-a 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b813f890 con 0x7f22b80a68e0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b7fff640 10 monclient: get_auth_request con 0x7f22b80a7200 auth_method 0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b7fff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22b80a7200 0x7f22b813e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b6ffd640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c7a0 tx=0x7f22a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 client.?.objecter ms_handle_connect 0x7f22b80a68e0 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:00.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a0018070 con 0x7f22b80a68e0 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: dump: 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.949 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: _finish_auth 0 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a0004d80 con 0x7f22b80a68e0 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 5 monclient: authenticate success, global_id 36540 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _renew_subs 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b80a2430 con 0x7f22b80a68e0 2024-09-17T16:44:00.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a00070d0 con 0x7f22b80a68e0 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22b5ffb640 4 set_mon_vals no callback set 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 client.36540.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22affff640 10 monclient: dump: 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.951 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _renew_subs 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b813a040 con 0x7f22b80a68e0 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.923+0000 7f22bdf27640 1 librados: init done 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22bdf27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22bdf27640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 0x7f22b813a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f22b8051e80 con 0x7f22b813e910 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f22a0007270 con 0x7f22b80a68e0 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 0x7f22b813a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f229803e550 0x7f2298040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient: get_auth_request con 0x7f22b813e910 auth_method 0 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(641..641 src has 1..641) ==== 6769+0+0 (secure 0 0 0) 0x7f22a004f5e0 con 0x7f22b80a68e0 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 client.36540.objecter ms_dispatch 0x7f22b8000ba0 osd_map(641..641 src has 1..641) 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 3 client.36540.objecter handle_osd_map got epochs [641,641] > 0 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 3 client.36540.objecter handle_osd_map decoding full epoch 641 2024-09-17T16:44:00.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 20 client.36540.objecter dump_active .. 0 homeless 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b7fff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f229803e550 0x7f2298040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b7fff640 10 monclient: get_auth_request con 0x7f229803e550 auth_method 0 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 10 monclient(hunting): handle_auth_done global_id 54867 payload 824 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22b80a1ee0 con 0x7f22b813e910 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 client.36540.objecter ms_handle_connect 0x7f22b813e910 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 client.36540.objecter resend_mon_ops 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b77fe640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 0x7f22b813a8c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22b806e5a0 tx=0x7f22a8019110 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 client.36540.objecter ms_handle_connect 0x7f229803e550 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22b7fff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f229803e550 0x7f2298040a10 secure :-1 s=READY pgs=3634 cs=0 l=1 rev1=1 crypto rx=0x7f22b000b820 tx=0x7f22b0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a80b9030 con 0x7f22b813e910 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 monclient: dump: 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.955 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f22a8003620 con 0x7f22b813e910 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 msgr2=0x7f22b813a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 0x7f22b813a8c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22b806e5a0 tx=0x7f22a8019110 comp rx=0 tx=0).stop 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 msgr2=0x7f22b813a8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.927+0000 7f22affff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813e910 0x7f22b813a8c0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22b806e5a0 tx=0x7f22a8019110 comp rx=0 tx=0).stop 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 0x7f22b813b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f22b806ddb0 con 0x7f22b813ae00 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 0x7f22b813b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient: get_auth_request con 0x7f22b813ae00 auth_method 0 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:00.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 10 monclient(hunting): handle_auth_done global_id 54876 payload 824 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 1 -- 172.21.15.105:0/3529708605 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f22b813ecf0 con 0x7f22b813ae00 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 client.36540.objecter ms_handle_connect 0x7f22b813ae00 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 client.36540.objecter resend_mon_ops 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b77fe640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 0x7f22b813b210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22a8000c10 tx=0x7f22a80a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22a80b9040 con 0x7f22b813ae00 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:00.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 monclient: dump: 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:00.960 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f22a80039e0 con 0x7f22b813ae00 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 msgr2=0x7f22b813b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 0x7f22b813b210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22a8000c10 tx=0x7f22a80a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 msgr2=0x7f22b813b210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:00.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22affff640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22b813ae00 0x7f22b813b210 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22a8000c10 tx=0x7f22a80a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 547941, 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:00.962 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.963 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.964 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.965 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:00.966 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:00.967 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:00.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.969 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.972 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.973 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.976 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:00.977 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.978 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.979 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:00.980 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:00.981 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.983 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:00.984 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:00.985 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:00.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 librados: watch_flush enter 2024-09-17T16:44:00.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 librados: watch_flush exit 2024-09-17T16:44:00.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 20 client.36540.objecter shutdown clearing up homeless session... 2024-09-17T16:44:00.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 client.36540.objecter successfully canceled tick 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f229803e550 msgr2=0x7f2298040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f229803e550 0x7f2298040a10 secure :-1 s=READY pgs=3634 cs=0 l=1 rev1=1 crypto rx=0x7f22b000b820 tx=0x7f22b0008040 comp rx=0 tx=0).stop 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 10 monclient: shutdown 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 msgr2=0x7f22b813de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c7a0 tx=0x7f22a000cc60 comp rx=0 tx=0).stop 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22b7fff640 1 -- 172.21.15.105:0/3529708605 reap_dead start 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 shutdown_connections 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 --2- 172.21.15.105:0/3529708605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b80a68e0 0x7f22b813de90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 >> 172.21.15.105:0/3529708605 conn(0x7f22b8068ba0 msgr2=0x7f22b81413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 shutdown_connections 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.935+0000 7f22bdf27640 1 -- 172.21.15.105:0/3529708605 wait complete. 2024-09-17T16:44:00.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:00.939+0000 7f22bdf27640 1 librados: shutdown 2024-09-17T16:44:00.988 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:00.988 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:44:00.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:44:01.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: get_monmap_and_config 2024-09-17T16:44:01.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: monmap: 2024-09-17T16:44:01.108 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.108 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:01.104191+0000 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:01.104191+0000 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.109 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:01.110 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:01.110 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 Processor -- start 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 -- start start 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: init 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient: _add_conns ranks=[3,1,4,2,5,8,10,6,0,7,9] 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6104139b20 0x7f610409ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-g con 0x7f6104139b20 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6104139f00 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-c con 0x7f61040a4e60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f610406c4e0 con 0x7f6104139f00 2024-09-17T16:44:01.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f610406c310 con 0x7f6104139b20 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f610409e070 con 0x7f61040a4e60 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6108bad640 10 monclient(hunting): authenticate will time out at 1924.369971s 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34090/0 (socket says 172.21.15.105:34090) 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 -- 172.21.15.105:0/429034610 learned_addr learned my addr 172.21.15.105:0/429034610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): get_auth_request con 0x7f61040a4e60 auth_method 0 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f610259c640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6104139b20 0x7f610409ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f610259c640 10 monclient(hunting): get_auth_request con 0x7f6104139b20 auth_method 0 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f610259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f610259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f610259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient(hunting): handle_auth_done global_id 74686 payload 274 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 -- 172.21.15.105:0/429034610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6104139b20 msgr2=0x7f610409ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6104139b20 0x7f610409ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 -- 172.21.15.105:0/429034610 >> v1:172.21.15.160:6789/0 conn(0x7f6104139f00 legacy=0x7f61040a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 monclient: found mon.noname-c 2024-09-17T16:44:01.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:01.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.103+0000 7f6102d9d640 1 -- 172.21.15.105:0/429034610 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610409dd90 con 0x7f61040a4e60 2024-09-17T16:44:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6102d9d640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009e80 tx=0x7f60fc01c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=48be0be2c705a82f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 1 -- 172.21.15.105:0/429034610 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60fc0090d0 con 0x7f61040a4e60 2024-09-17T16:44:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: dump: 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.123 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 5 monclient: authenticate success, global_id 74686 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 1 -- 172.21.15.105:0/429034610 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc0a5b10 con 0x7f61040a4e60 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 4 set_mon_vals no callback set 2024-09-17T16:44:01.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 1 -- 172.21.15.105:0/429034610 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60fc0a5d00 con 0x7f61040a4e60 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610159a640 10 monclient: dump: 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.125 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: shutdown 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- 172.21.15.105:0/429034610 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 msgr2=0x7f6104141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009e80 tx=0x7f60fc01c910 comp rx=0 tx=0).stop 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- 172.21.15.105:0/429034610 shutdown_connections 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f61040a4e60 0x7f6104141870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- 172.21.15.105:0/429034610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6104139b20 0x7f610409ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- 172.21.15.105:0/429034610 >> 172.21.15.105:0/429034610 conn(0x7f61040a93f0 msgr2=0x7f61040a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- 172.21.15.105:0/429034610 shutdown_connections 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- 172.21.15.105:0/429034610 wait complete. 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: monmap: 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.127 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 librados: starting msgr at 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 librados: starting objecter 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 Processor -- start 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- start start 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 librados: setting wanted keys 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 librados: calling monclient init 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: init 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient: _add_conns ranks=[6,1,5,8,3,7,4,2,0] 2024-09-17T16:44:01.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-d con 0x7f61040a4e60 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6104139b20 0x7f61041d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-f con 0x7f6104139b20 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6104139f00 0x7f610413f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): picked mon.noname-h con 0x7f6104139f00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f61041445d0 con 0x7f6104139b20 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f610406cde0 con 0x7f6104139f00 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f6104144750 con 0x7f61040a4e60 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6108bad640 10 monclient(hunting): authenticate will time out at 1924.375040s 2024-09-17T16:44:01.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39242/0 (socket says 172.21.15.105:39242) 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 -- 172.21.15.105:0/2503573389 learned_addr learned my addr 172.21.15.105:0/2503573389 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): get_auth_request con 0x7f61040a4e60 auth_method 0 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6102d9d640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6104139f00 0x7f610413f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6104139b20 0x7f61041d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient(hunting): handle_auth_done global_id 94173 payload 824 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6104139f00 msgr2=0x7f610413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6104139f00 0x7f610413f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6104139b20 msgr2=0x7f61041d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6104139b20 0x7f61041d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 monclient: found mon.noname-d 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61041389a0 con 0x7f61040a4e60 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6101d9b640 10 monclient: get_auth_request con 0x7f6104139b20 auth_method 0 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6104139b20 0x7f61041d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f610259c640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60f400d800 tx=0x7f60f400dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 client.?.objecter ms_handle_connect 0x7f61040a4e60 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60f400bb30 con 0x7f61040a4e60 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: dump: 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f4019070 con 0x7f61040a4e60 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60f400bcf0 con 0x7f61040a4e60 2024-09-17T16:44:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f3fff640 4 set_mon_vals no callback set 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.107+0000 7f60f2ffd640 10 monclient: dump: 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 5 monclient: authenticate success, global_id 94173 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 monclient: _renew_subs 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61041206f0 con 0x7f61040a4e60 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 client.94173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 monclient: _renew_subs 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:01.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f6104120cb0 con 0x7f61040a4e60 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 1 librados: init done 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 0x7f610413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6104051e80 con 0x7f610413f9d0 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f60f400b6c0 con 0x7f61040a4e60 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 0x7f610413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient: get_auth_request con 0x7f610413f9d0 auth_method 0 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f60e403e550 0x7f60e4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(641..641 src has 1..641) ==== 6769+0+0 (secure 0 0 0) 0x7f60f400f040 con 0x7f61040a4e60 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 client.94173.objecter ms_dispatch 0x7f6104000ba0 osd_map(641..641 src has 1..641) 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 3 client.94173.objecter handle_osd_map got epochs [641,641] > 0 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 3 client.94173.objecter handle_osd_map decoding full epoch 641 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 20 client.94173.objecter dump_active .. 0 homeless 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 10 monclient(hunting): handle_auth_done global_id 74390 payload 824 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f610411fdf0 con 0x7f610413f9d0 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6102d9d640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f60e403e550 0x7f60e4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6102d9d640 10 monclient: get_auth_request con 0x7f60e403e550 auth_method 0 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 client.94173.objecter ms_handle_connect 0x7f610413f9d0 2024-09-17T16:44:01.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 client.94173.objecter resend_mon_ops 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 0x7f610413fdb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61040a6800 tx=0x7f60f800d880 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60f8017070 con 0x7f610413f9d0 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 monclient: dump: 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.138 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 client.94173.objecter ms_handle_connect 0x7f60e403e550 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f6102d9d640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f60e403e550 0x7f60e4040a10 secure :-1 s=READY pgs=3635 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0ae000 tx=0x7f60fc0a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f60f80052e0 con 0x7f610413f9d0 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 msgr2=0x7f610413fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 0x7f610413fdb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61040a6800 tx=0x7f60f800d880 comp rx=0 tx=0).stop 2024-09-17T16:44:01.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 msgr2=0x7f610413fdb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:01.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.111+0000 7f60f2ffd640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f610413f9d0 0x7f610413fdb0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f61040a6800 tx=0x7f60f800d880 comp rx=0 tx=0).stop 2024-09-17T16:44:01.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 0x7f610413b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f6104138d60 con 0x7f61041402f0 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 0x7f610413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient: get_auth_request con 0x7f61041402f0 auth_method 0 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 10 monclient(hunting): handle_auth_done global_id 74399 payload 824 2024-09-17T16:44:01.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 1 -- 172.21.15.105:0/2503573389 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f610409ea50 con 0x7f61041402f0 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 client.94173.objecter ms_handle_connect 0x7f61041402f0 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 client.94173.objecter resend_mon_ops 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6101d9b640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 0x7f610413b630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f8000fc0 tx=0x7f60f8000ee0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60f8017040 con 0x7f61041402f0 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 monclient: dump: 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.143 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.144 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f60f80070b0 con 0x7f61041402f0 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 msgr2=0x7f610413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 0x7f610413b630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f8000fc0 tx=0x7f60f8000ee0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 msgr2=0x7f610413b630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f60f2ffd640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f61041402f0 0x7f610413b630 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f8000fc0 tx=0x7f60f8000ee0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.145 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 280588, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:01.146 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.147 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.148 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:01.149 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.150 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:01.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:01.156 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.157 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:01.158 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:01.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.161 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:01.162 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.164 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.166 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:01.167 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 10 librados: watch_flush enter 2024-09-17T16:44:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 10 librados: watch_flush exit 2024-09-17T16:44:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 20 client.94173.objecter shutdown clearing up homeless session... 2024-09-17T16:44:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.119+0000 7f6108bad640 10 client.94173.objecter successfully canceled tick 2024-09-17T16:44:01.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f60e403e550 msgr2=0x7f60e4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f60e403e550 0x7f60e4040a10 secure :-1 s=READY pgs=3635 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0ae000 tx=0x7f60fc0a7070 comp rx=0 tx=0).stop 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 10 monclient: shutdown 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 msgr2=0x7f61041d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60f400d800 tx=0x7f60f400dcc0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6102d9d640 1 -- 172.21.15.105:0/2503573389 reap_dead start 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 shutdown_connections 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 --2- 172.21.15.105:0/2503573389 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f61040a4e60 0x7f61041d18c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 >> 172.21.15.105:0/2503573389 conn(0x7f61040a93f0 msgr2=0x7f61040a76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 shutdown_connections 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 -- 172.21.15.105:0/2503573389 wait complete. 2024-09-17T16:44:01.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.123+0000 7f6108bad640 1 librados: shutdown 2024-09-17T16:44:01.171 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:01.171 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:44:01.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:44:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: get_monmap_and_config 2024-09-17T16:44:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: monmap: 2024-09-17T16:44:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.289 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:01.285338+0000 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:01.285338+0000 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:01.290 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:01.291 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 Processor -- start 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 -- start start 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: init 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient: _add_conns ranks=[2,5,3,8,7,1,4,10,0,9,6] 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): picked mon.noname-b con 0x7f35640a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f35640a4910 0x7f35640a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): picked mon.noname-h con 0x7f35640a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:01.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35641396a0 0x7f3564139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): picked mon.noname-g con 0x7f35641396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f35640a3fc0 con 0x7f35640a68e0 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f35640a63c0 con 0x7f35641396a0 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f35640a7740 con 0x7f35640a4910 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3569081640 10 monclient(hunting): authenticate will time out at 1924.551145s 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47134/0 (socket says 172.21.15.105:47134) 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 -- 172.21.15.105:0/3015201185 learned_addr learned my addr 172.21.15.105:0/3015201185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): get_auth_request con 0x7f35640a68e0 auth_method 0 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356359e640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35641396a0 0x7f3564139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356259c640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f35640a4910 0x7f35640a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356259c640 10 monclient(hunting): get_auth_request con 0x7f35640a4910 auth_method 0 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356359e640 10 monclient(hunting): get_auth_request con 0x7f35641396a0 auth_method 0 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356359e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f356359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient(hunting): handle_auth_done global_id 84377 payload 274 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 -- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f35640a4910 msgr2=0x7f35640a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f35640a4910 0x7f35640a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 -- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35641396a0 msgr2=0x7f3564139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35641396a0 0x7f3564139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 monclient: found mon.noname-b 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:01.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.283+0000 7f3562d9d640 1 -- 172.21.15.105:0/3015201185 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356406ec10 con 0x7f35640a68e0 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3562d9d640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35540098d0 tx=0x7f355401cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db47cee6a1a57531 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 1 -- 172.21.15.105:0/3015201185 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35540b0070 con 0x7f35640a68e0 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: dump: 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.304 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 5 monclient: authenticate success, global_id 84377 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 1 -- 172.21.15.105:0/3015201185 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35540aa810 con 0x7f35640a68e0 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 4 set_mon_vals no callback set 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 1 -- 172.21.15.105:0/3015201185 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35540a9300 con 0x7f35640a68e0 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3561d9b640 10 monclient: dump: 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.306 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 10 monclient: shutdown 2024-09-17T16:44:01.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 -- 172.21.15.105:0/3015201185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 msgr2=0x7f35640a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35540098d0 tx=0x7f355401cea0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 -- 172.21.15.105:0/3015201185 shutdown_connections 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35641396a0 0x7f3564139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f35640a4910 0x7f35640a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 --2- 172.21.15.105:0/3015201185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f35640a68e0 0x7f35640a6cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 -- 172.21.15.105:0/3015201185 >> 172.21.15.105:0/3015201185 conn(0x7f3564068ba0 msgr2=0x7f3564068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 -- 172.21.15.105:0/3015201185 shutdown_connections 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 -- 172.21.15.105:0/3015201185 wait complete. 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 10 monclient: monmap: 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:01.308 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 librados: starting msgr at 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 librados: starting objecter 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.287+0000 7f3569081640 1 Processor -- start 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- start start 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 librados: setting wanted keys 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 librados: calling monclient init 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: init 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _add_conns ranks=[8,3,0,6,7,2,5,1,4] 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 0x7f35641379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): picked mon.noname-e con 0x7f35640a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 0x7f3564137f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): picked mon.noname-g con 0x7f35640a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35641396a0 0x7f3564138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): picked mon.noname-a con 0x7f35641396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3564143320 con 0x7f35641396a0 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f356406e630 con 0x7f35640a68e0 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f35641434a0 con 0x7f35640a4910 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): authenticate will time out at 1924.556737s 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 0x7f35641379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 0x7f35641379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33602/0 (socket says 172.21.15.105:33602) 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 -- 172.21.15.105:0/2255014960 learned_addr learned my addr 172.21.15.105:0/2255014960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): get_auth_request con 0x7f35640a4910 auth_method 0 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35641396a0 0x7f3564138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): get_auth_request con 0x7f35641396a0 auth_method 0 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 0x7f3564137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 msgr2=0x7f3564137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 msgr2=0x7f3564137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 0x7f3564137f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 0x7f3564137f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient(hunting): handle_auth_done global_id 54935 payload 824 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 msgr2=0x7f3564137f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35640a68e0 0x7f3564137f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35641396a0 msgr2=0x7f3564138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35641396a0 0x7f3564138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 monclient: found mon.noname-e 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3564123750 con 0x7f35640a4910 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35641396a0 0x7f3564138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3562d9d640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 0x7f35641379d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f35540ae040 tx=0x7f35540191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.?.objecter ms_handle_connect 0x7f35640a4910 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35540b9070 con 0x7f35640a4910 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: dump: 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35540044b0 con 0x7f35640a4910 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35540b0040 con 0x7f35640a4910 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 5 monclient: authenticate success, global_id 54935 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _renew_subs 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356413fbd0 con 0x7f35640a4910 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356159a640 4 set_mon_vals no callback set 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: dump: 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.317 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 client.54935.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _renew_subs 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3564123470 con 0x7f35640a4910 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 librados: init done 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 0x7f35640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3564051e80 con 0x7f35640a0a20 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f3554004040 con 0x7f35640a4910 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f355003e570 0x7f3550040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 0x7f35640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(641..641 src has 1..641) ==== 6769+0+0 (secure 0 0 0) 0x7f35540e9310 con 0x7f35640a4910 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.54935.objecter ms_dispatch 0x7f3564000ba0 osd_map(641..641 src has 1..641) 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 3 client.54935.objecter handle_osd_map got epochs [641,641] > 0 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 3 client.54935.objecter handle_osd_map decoding full epoch 641 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient: get_auth_request con 0x7f35640a0a20 auth_method 0 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 20 client.54935.objecter dump_active .. 0 homeless 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f355003e570 0x7f3550040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 10 monclient: get_auth_request con 0x7f355003e570 auth_method 0 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 10 monclient(hunting): handle_auth_done global_id 64645 payload 824 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3564139ae0 con 0x7f35640a0a20 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.54935.objecter ms_handle_connect 0x7f35640a0a20 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.54935.objecter resend_mon_ops 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 0x7f35640a0e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f355800e9a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 client.54935.objecter ms_handle_connect 0x7f355003e570 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f356359e640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f355003e570 0x7f3550040a30 secure :-1 s=READY pgs=3636 cs=0 l=1 rev1=1 crypto rx=0x7f356413ecf0 tx=0x7f355c009780 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3558017070 con 0x7f35640a0a20 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.291+0000 7f354b7fe640 10 monclient: dump: 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.321 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f355801c450 con 0x7f35640a0a20 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 msgr2=0x7f35640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 0x7f35640a0e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f355800e9a0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 msgr2=0x7f35640a0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:01.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.295+0000 7f354b7fe640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a0a20 0x7f35640a0e00 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f355800e9a0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f3569081640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f3569081640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 0x7f356413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f3569081640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f3564123c70 con 0x7f35640a1340 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 0x7f356413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient: get_auth_request con 0x7f35640a1340 auth_method 0 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 10 monclient(hunting): handle_auth_done global_id 64654 payload 824 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.299+0000 7f356259c640 1 -- 172.21.15.105:0/2255014960 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f356413f820 con 0x7f35640a1340 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f356259c640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 0x7f356413d600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f3558004740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 client.54935.objecter ms_handle_connect 0x7f35640a1340 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 client.54935.objecter resend_mon_ops 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f355800f040 con 0x7f35640a1340 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:01.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 monclient: dump: 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.326 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9455 (secure 0 0 0) 0x7f355800efc0 con 0x7f35640a1340 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 msgr2=0x7f356413d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 0x7f356413d600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f3558004740 comp rx=0 tx=0).stop 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 msgr2=0x7f356413d600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f354b7fe640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f35640a1340 0x7f356413d600 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f355800e4e0 tx=0x7f3558004740 comp rx=0 tx=0).stop 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:01.327 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 414403, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:01.328 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.329 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.330 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:01.331 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:01.332 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.334 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.336 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.337 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.339 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.340 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:01.342 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.344 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.345 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.346 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:01.347 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.348 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:01.349 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:01.350 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 10 librados: watch_flush enter 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 10 librados: watch_flush exit 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 20 client.54935.objecter shutdown clearing up homeless session... 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 10 client.54935.objecter successfully canceled tick 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f355003e570 msgr2=0x7f3550040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f355003e570 0x7f3550040a30 secure :-1 s=READY pgs=3636 cs=0 l=1 rev1=1 crypto rx=0x7f356413ecf0 tx=0x7f355c009780 comp rx=0 tx=0).stop 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 10 monclient: shutdown 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 msgr2=0x7f35641379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 --2- 172.21.15.105:0/2255014960 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f35640a4910 0x7f35641379d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f35540ae040 tx=0x7f35540191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f356359e640 1 -- 172.21.15.105:0/2255014960 reap_dead start 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f356359e640 1 -- 172.21.15.105:0/2255014960 reap_dead start 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 shutdown_connections 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 >> 172.21.15.105:0/2255014960 conn(0x7f3564068ba0 msgr2=0x7f35640a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 shutdown_connections 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 -- 172.21.15.105:0/2255014960 wait complete. 2024-09-17T16:44:01.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.303+0000 7f3569081640 1 librados: shutdown 2024-09-17T16:44:01.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:01.353 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:44:01.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:44:01.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: get_monmap_and_config 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: monmap: 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:01.470364+0000 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:01.470364+0000 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:01.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 1 Processor -- start 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 1 -- start start 2024-09-17T16:44:01.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: init 2024-09-17T16:44:01.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.467+0000 7fbb17585640 10 monclient: _add_conns ranks=[0,4,7,1,5,8,10,6,2,3,9] 2024-09-17T16:44:01.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb100a68e0 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 0x7fbb1013eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-c con 0x7fbb100a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb100a73d0 0x7fbb10141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-i con 0x7fbb100a73d0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1006ebc0 con 0x7fbb100a68e0 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fbb100a3fc0 con 0x7fbb100a4910 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fbb100a63c0 con 0x7fbb100a73d0 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient(hunting): authenticate will time out at 1924.736095s 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15321640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbb100a68e0 0x7fbb100a6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42268/0 (socket says 172.21.15.105:42268) 2024-09-17T16:44:01.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15321640 1 -- 172.21.15.105:0/3882439546 learned_addr learned my addr 172.21.15.105:0/3882439546 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 0x7fbb1013eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): get_auth_request con 0x7fbb100a4910 auth_method 0 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15b22640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb100a73d0 0x7fbb10141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15b22640 10 monclient(hunting): get_auth_request con 0x7fbb100a73d0 auth_method 0 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15b22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15b22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb15b22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 1 -- 172.21.15.105:0/3882439546 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716542338 0 0) 0x7fbb1006ebc0 con 0x7fbb100a68e0 2024-09-17T16:44:01.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient(hunting): _negotiate 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient(hunting): my global_id is 36558 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 1 -- 172.21.15.105:0/3882439546 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbafc003530 con 0x7fbb100a68e0 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient(hunting): handle_auth_done global_id 74695 payload 274 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 -- 172.21.15.105:0/3882439546 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb100a73d0 msgr2=0x7fbb10141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb100a73d0 0x7fbb10141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 -- 172.21.15.105:0/3882439546 >> v1:172.21.15.105:6789/0 conn(0x7fbb100a68e0 legacy=0x7fbb100a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 monclient: found mon.noname-c 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 -- 172.21.15.105:0/3882439546 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb100a3690 con 0x7fbb100a4910 2024-09-17T16:44:01.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb14b20640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 0x7fbb1013eaf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb000097f0 tx=0x7fbb0001c370 comp rx=0 tx=0).ready entity=mon.4 client_cookie=647bbe905e4a74f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 1 -- 172.21.15.105:0/3882439546 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb000b0070 con 0x7fbb100a4910 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: dump: 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.488 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 5 monclient: authenticate success, global_id 74695 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 1 -- 172.21.15.105:0/3882439546 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb000aa3a0 con 0x7fbb100a4910 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:01.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 4 set_mon_vals no callback set 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 1 -- 172.21.15.105:0/3882439546 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb000aa9c0 con 0x7fbb100a4910 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb07fff640 10 monclient: dump: 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.490 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 10 monclient: shutdown 2024-09-17T16:44:01.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- 172.21.15.105:0/3882439546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 msgr2=0x7fbb1013eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 0x7fbb1013eaf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb000097f0 tx=0x7fbb0001c370 comp rx=0 tx=0).stop 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- 172.21.15.105:0/3882439546 shutdown_connections 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb100a73d0 0x7fbb10141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 --2- 172.21.15.105:0/3882439546 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fbb100a4910 0x7fbb1013eaf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- 172.21.15.105:0/3882439546 >> 172.21.15.105:0/3882439546 conn(0x7fbb10068ba0 msgr2=0x7fbb10068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- 172.21.15.105:0/3882439546 shutdown_connections 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.471+0000 7fbb17585640 1 -- 172.21.15.105:0/3882439546 wait complete. 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: build_initial_monmap 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: monmap: 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:01.492 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 librados: starting msgr at 2024-09-17T16:44:01.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 librados: starting objecter 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 Processor -- start 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- start start 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 librados: setting wanted keys 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 librados: calling monclient init 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: init 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _add_conns ranks=[0,5,6,4,7,3,2,8,1] 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-a con 0x7fbb100a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 0x7fbb10126730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-h con 0x7fbb100a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 0x7fbb10126c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): picked mon.noname-d con 0x7fbb100a73d0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:01.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10143f20 con 0x7fbb100a4910 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fbb10143da0 con 0x7fbb100a68e0 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fbb101440a0 con 0x7fbb100a73d0 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient(hunting): authenticate will time out at 1924.741432s 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 0x7fbb10126c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36342/0 (socket says 172.21.15.105:36342) 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 -- 172.21.15.105:0/1699201946 learned_addr learned my addr 172.21.15.105:0/1699201946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): get_auth_request con 0x7fbb100a4910 auth_method 0 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): get_auth_request con 0x7fbb100a73d0 auth_method 0 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient(hunting): handle_auth_done global_id 36567 payload 824 2024-09-17T16:44:01.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb14b20640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 0x7fbb10126730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 -- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 msgr2=0x7fbb10126c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 0x7fbb10126c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 -- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 msgr2=0x7fbb10126730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 0x7fbb10126730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient: _finish_hunting 0 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 monclient: found mon.noname-a 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 -- 172.21.15.105:0/1699201946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb10129100 con 0x7fbb100a4910 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb14b20640 10 monclient: get_auth_request con 0x7fbb100a68e0 auth_method 0 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15b22640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 0x7fbb10126c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb14b20640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 0x7fbb10126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:01.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb15321640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00c4d0 tx=0x7fbb0c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 client.?.objecter ms_handle_connect 0x7fbb100a4910 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb0c017070 con 0x7fbb100a4910 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: dump: 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.498 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 5 monclient: authenticate success, global_id 36567 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c007070 con 0x7fbb100a4910 2024-09-17T16:44:01.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _renew_subs 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb10127c10 con 0x7fbb100a4910 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb0c0076d0 con 0x7fbb100a4910 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb06ffd640 4 set_mon_vals no callback set 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 client.36567.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 monclient: dump: 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:01.500 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _renew_subs 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:01.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb101289d0 con 0x7fbb100a4910 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 librados: init done 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fbb0c007260 con 0x7fbb100a4910 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb100a6cc0 con 0x7fbb100a4910 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 0x7fbaf4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(641..641 src has 1..641) ==== 6769+0+0 (secure 0 0 0) 0x7fbb0c04f2e0 con 0x7fbb100a4910 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 10 client.36567.objecter ms_dispatch 0x7fbb10000ba0 osd_map(641..641 src has 1..641) 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 3 client.36567.objecter handle_osd_map got epochs [641,641] > 0 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 3 client.36567.objecter handle_osd_map decoding full epoch 641 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb05ffb640 20 client.36567.objecter dump_active .. 0 homeless 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.475+0000 7fbb14b20640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 0x7fbaf4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:01.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.479+0000 7fbb14b20640 10 monclient: get_auth_request con 0x7fbaf403e550 auth_method 0 2024-09-17T16:44:01.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.479+0000 7fbb05ffb640 10 client.36567.objecter ms_handle_connect 0x7fbaf403e550 2024-09-17T16:44:01.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.479+0000 7fbb14b20640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 0x7fbaf4040a10 secure :-1 s=READY pgs=3637 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ad000 tx=0x7fbb000a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:01.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.483+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb0c011d50 con 0x7fbb100a4910 2024-09-17T16:44:01.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.483+0000 7fbb05ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:01.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.483+0000 7fbb05ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.735+0000 7fbb17585640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:44:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.735+0000 7fbb17585640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:44:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.735+0000 7fbb17585640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:44:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.735+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fbb100746d0 con 0x7fbb100a4910 2024-09-17T16:44:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb05ffb640 1 -- 172.21.15.105:0/1699201946 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fbb0c04f0d0 con 0x7fbb100a4910 2024-09-17T16:44:01.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb05ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:44:01.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb05ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:01.748 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.748 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1252873509,config=1228700967,health=2856257528,kv=2863082231,logm=3012276590}) 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=18,config=2,health=12,kv=10,logm=58} crc {auth=2881920865,config=692619927,health=2856257528,kv=2863082231,logm=148828030}) 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=12,kv=9,logm=58} crc {auth=1252873509,config=692619927,health=2856257528,kv=566996664,logm=1304223379}) 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=11,kv=10,logm=60} crc {auth=806974975,config=692619927,health=4177975799,kv=2863082231,logm=1419315737}) 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=19,config=2,health=11,kv=10,logm=58} crc {auth=1252873509,config=692619927,health=2990453344,kv=2863082231,logm=1434514360}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=11,kv=9,logm=62} crc {auth=917825994,config=692619927,health=417084525,kv=382906235,logm=2181409984}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=19,config=2,health=12,kv=9,logm=58} crc {auth=2497200037,config=692619927,health=2856257528,kv=3449169938,logm=2775783018}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=4178210802,config=692619927,health=2856257528,kv=2863082231,logm=2914210015}) 2024-09-17T16:44:01.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.739+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1285645405,config=692619927,health=2856257528,kv=1522469405,logm=391708327}) 2024-09-17T16:44:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 10 librados: watch_flush enter 2024-09-17T16:44:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 10 librados: watch_flush exit 2024-09-17T16:44:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 20 client.36567.objecter shutdown clearing up homeless session... 2024-09-17T16:44:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 10 client.36567.objecter successfully canceled tick 2024-09-17T16:44:01.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 msgr2=0x7fbaf4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 0x7fbaf4040a10 secure :-1 s=READY pgs=3637 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ad000 tx=0x7fbb000a6000 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 10 monclient: shutdown 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 msgr2=0x7fbb101261f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.739+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00c4d0 tx=0x7fbb0c00c990 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 shutdown_connections 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbaf403e550 0x7fbaf4040a10 unknown :-1 s=CLOSED pgs=3637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fbb100a73d0 0x7fbb10126c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb100a68e0 0x7fbb10126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 --2- 172.21.15.105:0/1699201946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb100a4910 0x7fbb101261f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 >> 172.21.15.105:0/1699201946 conn(0x7fbb10068ba0 msgr2=0x7fbb1009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 shutdown_connections 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 -- 172.21.15.105:0/1699201946 wait complete. 2024-09-17T16:44:01.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:01.743+0000 7fbb17585640 1 librados: shutdown 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3144446843}) 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4096592367}) 2024-09-17T16:44:01.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3468638229}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=366103984}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2203328832}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3170862219}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3609399300}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2708016186}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.743+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3108116896}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1842871477}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2301496900}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4210655577}) 2024-09-17T16:44:01.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1945826932}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2416091182}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4253137225}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909568215}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.747+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.751+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92262292}) 2024-09-17T16:44:01.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.751+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3635971828}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=605868190}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3895852993}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4133388437}) 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3413152715}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=637425625}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4050754101}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2311325250}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066424571}) 2024-09-17T16:44:01.765 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.755+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3411607281}) 2024-09-17T16:44:01.769 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949003489}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2062768819}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1119273116}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1809859729}) 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.770 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=878193616}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1303118939}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1123291443}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1597406121}) 2024-09-17T16:44:01.771 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.763+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1783030078}) 2024-09-17T16:44:01.772 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3671569473}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3115898040}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2861860844}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3569080419}) 2024-09-17T16:44:01.774 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2224978045}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2884612862}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=300645399}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1047644990}) 2024-09-17T16:44:01.775 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=661954565}) 2024-09-17T16:44:01.777 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=792878303}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2702245634}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3796771429}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.767+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3765373996}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2891587703}) 2024-09-17T16:44:01.778 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2421614348}) 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2699676419}) 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246730247}) 2024-09-17T16:44:01.779 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=115996195}) 2024-09-17T16:44:01.781 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.781 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.781 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3131986549}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.771+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=901136713}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1987363327}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1066206324}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=637577115}) 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.782 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1966743301}) 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1567494063}) 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2159406252}) 2024-09-17T16:44:01.783 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.775+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1705985633}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839081932}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1347852847}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691257328}) 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3678623277}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2545958967}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3508494585}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4128873913}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270751261}) 2024-09-17T16:44:01.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.779+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3214819854}) 2024-09-17T16:44:01.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145366518}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2814995839}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3742999876}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1045894848}) 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4098754258}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3611188473}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=520504376}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1718807413}) 2024-09-17T16:44:01.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.783+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=991226049}) 2024-09-17T16:44:01.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1142545145}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2568624760}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3755046194}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=839982269}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1634244725}) 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=826548465}) 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2898494484}) 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232007495}) 2024-09-17T16:44:01.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.787+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2210691517}) 2024-09-17T16:44:01.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1644153466}) 2024-09-17T16:44:01.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2139972574}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=665630664}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1794673157}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=484652063}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2223293675}) 2024-09-17T16:44:01.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2479671102}) 2024-09-17T16:44:01.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1377489280}) 2024-09-17T16:44:01.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.791+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1580771635}) 2024-09-17T16:44:01.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193626581}) 2024-09-17T16:44:01.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737306988}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4035624772}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=697469745}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2085432021}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=868006262}) 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1487185934}) 2024-09-17T16:44:01.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807260025}) 2024-09-17T16:44:01.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.795+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1197838349}) 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3009188209}) 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.808 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659434154}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3529697900}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=633256917}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4129272987}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1240770219}) 2024-09-17T16:44:01.809 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.810 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.810 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=687366168}) 2024-09-17T16:44:01.810 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.810 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3136590229}) 2024-09-17T16:44:01.810 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.799+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659470199}) 2024-09-17T16:44:01.812 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.812 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.812 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584471632}) 2024-09-17T16:44:01.812 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.812 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2815497433}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=464832378}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2774326632}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2444575545}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=205420459}) 2024-09-17T16:44:01.813 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.814 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.814 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1524731691}) 2024-09-17T16:44:01.814 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.814 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916220}) 2024-09-17T16:44:01.814 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.803+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1558188998}) 2024-09-17T16:44:01.816 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.816 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.816 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3132953651}) 2024-09-17T16:44:01.816 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.816 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2238005594}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1849524633}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=993472661}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1081790496}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602401878}) 2024-09-17T16:44:01.817 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.818 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.818 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3429472420}) 2024-09-17T16:44:01.818 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.818 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4015618849}) 2024-09-17T16:44:01.818 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.807+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=201196855}) 2024-09-17T16:44:01.820 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.820 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.820 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20659960}) 2024-09-17T16:44:01.820 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.820 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125740971}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3429993715}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4082708758}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=906016578}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.821 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3985792925}) 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1915570213}) 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339342822}) 2024-09-17T16:44:01.822 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.811+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=18313163}) 2024-09-17T16:44:01.824 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.824 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.824 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3086272512}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3591779838}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473650107}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1380486145}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3152568359}) 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.825 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2702562203}) 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3569657929}) 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589271275}) 2024-09-17T16:44:01.826 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.815+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1711924315}) 2024-09-17T16:44:01.828 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.828 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3458302782}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2959265250}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2691311261}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=498346392}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.829 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1783013214}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2835572675}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3982572615}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2454950753}) 2024-09-17T16:44:01.830 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.819+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2593443265}) 2024-09-17T16:44:01.832 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4204011380}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3205459107}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2402880740}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3277975496}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.833 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352087379}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2824887625}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4041026763}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133301242}) 2024-09-17T16:44:01.834 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.823+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2616686287}) 2024-09-17T16:44:01.836 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=843250093}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3723617119}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2350924914}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=856088049}) 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.837 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3074629601}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802653391}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.827+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=564088643}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445962573}) 2024-09-17T16:44:01.838 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3498715275}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3555506593}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=31431215}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1858330578}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.831+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1032549420}) 2024-09-17T16:44:01.841 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2148217134}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3213762871}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3370416026}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662824010}) 2024-09-17T16:44:01.842 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3254019033}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3900828179}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1724080581}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=378216546}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=675843445}) 2024-09-17T16:44:01.845 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.835+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2153685580}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3871125711}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1090183860}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883455350}) 2024-09-17T16:44:01.846 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=713001530}) 2024-09-17T16:44:01.848 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4044320336}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2168163144}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2554797409}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2192569973}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.849 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1643132573}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3398126780}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=308509620}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305006080}) 2024-09-17T16:44:01.850 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.839+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3460352824}) 2024-09-17T16:44:01.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618810478}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=768194456}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4012838144}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1032132592}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1797956225}) 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2212434495}) 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1223813321}) 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3102068330}) 2024-09-17T16:44:01.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.843+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1065396911}) 2024-09-17T16:44:01.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2064379941}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=850258184}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1407535133}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2260452948}) 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1635699794}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1997922030}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3300728351}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572974576}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.847+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1915610751}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1117604139}) 2024-09-17T16:44:01.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=985638626}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4120274078}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3953602653}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2214336473}) 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2774196310}) 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029955438}) 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1462189890}) 2024-09-17T16:44:01.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2866483952}) 2024-09-17T16:44:01.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1673707025}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1777351304}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=900971548}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3788078737}) 2024-09-17T16:44:01.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3304823372}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=581515071}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1955736015}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4267706613}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.851+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2969064101}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=105469559}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082924529}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3984414711}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3030438287}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2474862830}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=356778517}) 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2047428891}) 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1785358233}) 2024-09-17T16:44:01.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.855+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4179665641}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=851980935}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3981793534}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1243820671}) 2024-09-17T16:44:01.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3594727678}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2635836874}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2581047088}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916237834}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1182037121}) 2024-09-17T16:44:01.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.859+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3363273127}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4149028651}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2810973538}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1645581726}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2442264218}) 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2844261898}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1782866533}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1752256126}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902581317}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=72824214}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404231369}) 2024-09-17T16:44:01.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3308048965}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2676369926}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1435332693}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1682400769}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3806649464}) 2024-09-17T16:44:01.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2004189150}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4114813511}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.863+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1037456502}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=464171962}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2564238560}) 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3770741486}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2349584414}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1045535889}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3853404271}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3826360616}) 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1493721589}) 2024-09-17T16:44:01.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.867+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=603128782}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073178292}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3643054252}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3577542763}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1769192675}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=818350598}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=439547218}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1257237104}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1303881338}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383274077}) 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3798084617}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4067177907}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062311609}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1511310415}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4140549646}) 2024-09-17T16:44:01.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3843208485}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1586485186}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1308559027}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.871+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2834420063}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279922820}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3658819417}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=173940506}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2668445217}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1828946431}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3431111317}) 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3322857659}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=639220380}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.875+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3218160871}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595366644}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1460895710}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794879992}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=68373921}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2431503052}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4179780922}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3425041385}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044241559}) 2024-09-17T16:44:01.899 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.879+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4003799916}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2007674600}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1060374087}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3411774155}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1028723721}) 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.900 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3751996526}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3675053838}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1532724531}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659616098}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2603880334}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1942136997}) 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.901 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1565946372}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3284237727}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4277925313}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2252560254}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226829364}) 2024-09-17T16:44:01.902 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.903 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.903 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1090359485}) 2024-09-17T16:44:01.903 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.903 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=744414679}) 2024-09-17T16:44:01.903 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.883+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=772806532}) 2024-09-17T16:44:01.920 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.920 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.920 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2511681868}) 2024-09-17T16:44:01.920 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.920 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1942263407}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796328361}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4023797769}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2413747271}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=827289519}) 2024-09-17T16:44:01.921 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3285079650}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=413356637}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.887+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=957122439}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=267923889}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1125331242}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.922 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3007486724}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2712734579}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3457834467}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2036103942}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3404850960}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811591416}) 2024-09-17T16:44:01.923 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1282309700}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640814154}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=407709809}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=354785933}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1938216932}) 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.924 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=561812067}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=182896052}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3126201085}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194497991}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.891+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3196003130}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=454665064}) 2024-09-17T16:44:01.925 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2150047127}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1183643628}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3959617855}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2439414993}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.926 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3125096438}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1598384487}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559186000}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.895+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4134896890}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3678235140}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2439651524}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.927 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2857051166}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3327394454}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1292477854}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3549488454}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3801120946}) 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.928 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=628578788}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1415964642}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1164220943}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1959425871}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=451023374}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4078558423}) 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.929 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1182730078}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=421122120}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2445861296}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1171686081}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.899+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2532295900}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=997231898}) 2024-09-17T16:44:01.930 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1613223977}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4150865427}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3424185044}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3146338921}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3612097923}) 2024-09-17T16:44:01.931 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877832666}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1501048944}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.903+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3968100140}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465016373}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3418021098}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.932 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=897771291}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2524921512}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=344218249}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3598855454}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2984814310}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=103285959}) 2024-09-17T16:44:01.933 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2412451475}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668816628}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=172212366}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2241048648}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.907+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=493255733}) 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.934 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=767080211}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=323239808}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3829894620}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998627956}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3299113107}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=453607543}) 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.935 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3594293463}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1494267526}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6771448}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2776183247}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=476377995}) 2024-09-17T16:44:01.936 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.937 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.937 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1843677380}) 2024-09-17T16:44:01.937 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.937 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1871813095}) 2024-09-17T16:44:01.937 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.911+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3759497172}) 2024-09-17T16:44:01.972 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=173788540}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3797029290}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3936767661}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.973 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=159310339}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2539903070}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2942990341}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=643866854}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3179633314}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.915+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3679525343}) 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.974 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211974190}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3999800905}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540381201}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1617548363}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3733208232}) 2024-09-17T16:44:01.975 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=668693575}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=896926064}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4164621759}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2960678785}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1182610561}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.976 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022478684}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1968304367}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.919+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1727019819}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1549661506}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1346179378}) 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.977 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1406779404}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573115021}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2930111612}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402285711}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=808820159}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.978 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2414767694}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=765860033}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2969877961}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1773993030}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3233284827}) 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.979 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197602878}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.923+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3568417653}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2876840940}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3787197892}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1825540593}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4284416253}) 2024-09-17T16:44:01.980 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2272388020}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2705334943}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1886717431}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691936035}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.927+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1268874899}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.981 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1920800227}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3237923586}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2710865375}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3606430830}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3478151525}) 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.982 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2386346771}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3559311128}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468036940}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=945120107}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2148479326}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2756730566}) 2024-09-17T16:44:01.983 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=427263056}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1051609879}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3085163913}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=394313359}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.984 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3920885069}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=156407176}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.931+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1543653234}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169453314}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1884359100}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1307524893}) 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.985 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2824386907}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1195675308}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2087761659}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4045524130}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4286243739}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.935+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2198277598}) 2024-09-17T16:44:01.986 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2579187937}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1685458331}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1746520441}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=293231867}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.987 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=792190417}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4225467161}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=729249599}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=272609847}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1454716894}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1423682732}) 2024-09-17T16:44:01.988 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1863376134}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1004449502}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.939+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3673702452}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2134495956}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2756224657}) 2024-09-17T16:44:01.989 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803817252}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2145988175}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3691012464}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890432848}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3671193545}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.990 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=383924414}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4094616701}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1585659074}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2974583182}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=767041246}) 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.991 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462381802}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.943+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3675281365}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2507240326}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3998652531}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4167219375}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3128861295}) 2024-09-17T16:44:01.992 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2323842852}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4263503977}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3677436629}) 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.993 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=954536425}) 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1581904533}) 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=747813989}) 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.996 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1921528617}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=539452339}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3281819597}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1533093715}) 2024-09-17T16:44:01.997 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2151112218}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2818283149}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47230271}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.947+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3617125376}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:01.998 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3784308155}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2249093088}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1110705232}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3394673916}) 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:01.999 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1562730030}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1840518280}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=844472252}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=743839039}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.951+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1059573853}) 2024-09-17T16:44:02.000 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3662782732}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2980694764}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2458686345}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.001 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2644747260}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2405261697}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3613220568}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3408024783}) 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.002 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=374205773}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4211957617}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=185241574}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3561291184}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859387149}) 2024-09-17T16:44:02.003 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056210890}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1976314391}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1166168994}) 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.004 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.005 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2018828914}) 2024-09-17T16:44:02.005 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.005 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3811652098}) 2024-09-17T16:44:02.005 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.955+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3407312723}) 2024-09-17T16:44:02.005 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1680323375}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2797254137}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1709793258}) 2024-09-17T16:44:02.006 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3308030852}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2495951480}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=159041289}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.007 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1604695893}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543004861}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2376889557}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3673931975}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2905775011}) 2024-09-17T16:44:02.008 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2286767342}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=726113902}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3947420285}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3774298624}) 2024-09-17T16:44:02.009 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=402419511}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2196202209}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.959+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=505014436}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1188112004}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.010 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1081091889}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=125945560}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3255710493}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=317792601}) 2024-09-17T16:44:02.011 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=545543173}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1026318983}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128691842}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=533706241}) 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.012 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1113468054}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2291812849}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=108197890}) 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.013 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3906018630}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1696247827}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=661288078}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1473220472}) 2024-09-17T16:44:02.014 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.015 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.963+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1790478236}) 2024-09-17T16:44:02.015 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2408754208}) 2024-09-17T16:44:02.189 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.189 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4055064495}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3960910447}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1325469762}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2737235557}) 2024-09-17T16:44:02.190 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=166857128}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=283051749}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3768038220}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207738147}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.967+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3629940264}) 2024-09-17T16:44:02.191 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092085018}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3952749763}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3143857964}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2923238816}) 2024-09-17T16:44:02.192 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3842626183}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1885933065}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4097630027}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2104901540}) 2024-09-17T16:44:02.193 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3613047508}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3231345601}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3664152591}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2644222333}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.194 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=770731613}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3402579608}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3282623491}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1885754650}) 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.195 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2143391810}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.971+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2131292202}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3975372986}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1368713837}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811473910}) 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.196 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1620324054}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1845297290}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489770340}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=684562492}) 2024-09-17T16:44:02.197 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2652356482}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.975+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2732509002}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1300812013}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1411160206}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1175427065}) 2024-09-17T16:44:02.198 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=476905534}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3207424510}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691843971}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1123021916}) 2024-09-17T16:44:02.199 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692949340}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3341314581}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1285785436}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=301305882}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3013795681}) 2024-09-17T16:44:02.200 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3500414275}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3199950218}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2337277138}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.201 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3783428620}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=877450649}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.979+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1618033462}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021660168}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=349395279}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4173608370}) 2024-09-17T16:44:02.202 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.203 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.203 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=283709043}) 2024-09-17T16:44:02.203 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.203 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=439863973}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2258547880}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2362415362}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629776386}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.983+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1173553402}) 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.204 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2436614329}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3138507675}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2737353144}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.205 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4259034516}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3545760000}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2291997302}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=736797639}) 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.206 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309838862}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1991023467}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432528738}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.987+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4065530753}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435392563}) 2024-09-17T16:44:02.207 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2984029598}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2386698753}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619253384}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.208 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4263831229}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=862346808}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3646237987}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118510281}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3362630988}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.209 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4190851338}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2652439927}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2891287819}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2139143470}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.210 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=485607501}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751243472}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.991+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1410236000}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3707204881}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1193329858}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.211 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3869948702}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2062839183}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=648387683}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1958988832}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.212 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2080087091}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=993916154}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1275172312}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2492807020}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1742269316}) 2024-09-17T16:44:02.213 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1805196618}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1232380777}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=483122282}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3690949999}) 2024-09-17T16:44:02.214 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1933121882}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.995+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1996653433}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=353702252}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=255343831}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4222773906}) 2024-09-17T16:44:02.215 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3105233180}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1542375145}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2134075039}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=684905393}) 2024-09-17T16:44:02.216 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1656369316}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3425777137}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:01.999+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3235695396}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1543628837}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3047584679}) 2024-09-17T16:44:02.217 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=141251306}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4247533289}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=143890592}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2830480428}) 2024-09-17T16:44:02.218 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1628056629}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=435587715}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2370147120}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2100302627}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.219 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2279810213}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3390695611}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=922584788}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3164240831}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.220 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=609946818}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2773976004}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.003+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448909669}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=103658031}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1562190147}) 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.221 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2829529356}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908377873}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1550353696}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1476660102}) 2024-09-17T16:44:02.222 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=80061470}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3804267615}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3644925044}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.007+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3352212401}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=131055128}) 2024-09-17T16:44:02.223 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3881303930}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4238200335}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=328697446}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=465907919}) 2024-09-17T16:44:02.224 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3444759918}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2219324102}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2245055221}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.011+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1448465811}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=307731842}) 2024-09-17T16:44:02.225 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1472228267}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1423252805}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2605849854}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.226 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1392438505}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=617512171}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4264292753}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2185832700}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3261926989}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.227 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1805181738}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1786689401}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3994753243}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2790909239}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.228 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1538662863}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=359129667}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1138245867}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428315903}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.015+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3172666553}) 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.229 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173073262}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3648421107}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=951053683}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3678414994}) 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.230 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3410654587}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3186113920}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3645759442}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77607125}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.019+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1836886487}) 2024-09-17T16:44:02.231 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2016287347}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=222938437}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3348700748}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.232 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1763832281}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1287012516}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=142923169}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=196123934}) 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.233 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1911907639}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=806620880}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3424190796}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619529356}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=181902912}) 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.234 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2390691272}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3012103221}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.023+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=571774586}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1639613798}) 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.235 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3111677109}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3231412520}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1572655079}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=872446931}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3121507714}) 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.236 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3060615118}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3080758904}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=480389435}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2390678803}) 2024-09-17T16:44:02.237 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232693757}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.027+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3017258089}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2817389412}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=416783813}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.238 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2855301379}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=249287137}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2751421613}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2899607101}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.239 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1514701109}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2942999758}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2909382747}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158180194}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053382800}) 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.240 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4096389869}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1680669142}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.031+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4160942015}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=496143889}) 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.241 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2119059706}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=91380497}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3205358416}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3095278966}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2443687118}) 2024-09-17T16:44:02.242 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1138265039}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2087402127}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3115978317}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.243 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3500124052}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1681759301}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226761473}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.035+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1335345157}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3771900913}) 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.244 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1862572868}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1729387169}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1280518732}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3607211959}) 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.245 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2837076767}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=996772405}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771337872}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.039+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3716609008}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146873689}) 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.246 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3149926076}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=800394130}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=441971262}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3991984273}) 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.247 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424566487}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1928296802}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277472807}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3062804320}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3710796332}) 2024-09-17T16:44:02.248 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2731167488}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=327281589}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3063465402}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.249 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3419373938}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2560908424}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007585993}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2929375506}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.043+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2282975740}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.250 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=566493271}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2628814465}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=803060641}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2077692766}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.251 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1845024102}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3190505680}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4136212949}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753901828}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=280775708}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.252 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3358765215}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1075438225}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3277815870}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4051302675}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.253 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.047+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4230922453}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=231590511}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2747179007}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592307534}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4047922194}) 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.254 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1237159394}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4267417259}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3997692114}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2419475586}) 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.255 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1598587200}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089186718}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4084661308}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4196166540}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.051+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1629132180}) 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.256 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4161968219}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245645856}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1983751317}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=356327785}) 2024-09-17T16:44:02.257 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3295087160}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1827846178}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=76812776}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1313734501}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1116179941}) 2024-09-17T16:44:02.258 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1737754021}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1701732335}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2941995823}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.259 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2217358870}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=868067814}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2422190932}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=888456288}) 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.260 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3768057734}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.055+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1515233667}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=776728610}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1030204369}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=391760542}) 2024-09-17T16:44:02.261 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3832673806}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1770392013}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=56365689}) 2024-09-17T16:44:02.262 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1439428785}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269466825}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=515937056}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=771483353}) 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.263 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2818873760}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3488623674}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=526347750}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3240058601}) 2024-09-17T16:44:02.264 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2056237055}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1728567590}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1188355336}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.059+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=900828644}) 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.265 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2365574660}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1413203730}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2713904059}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4156294098}) 2024-09-17T16:44:02.266 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2413928640}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=100730442}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1101214957}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=750653632}) 2024-09-17T16:44:02.267 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2001178036}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000408764}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=171214732}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3517602658}) 2024-09-17T16:44:02.268 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3139726823}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2297510119}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1416738673}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.063+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.269 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1271177869}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269225185}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=968145028}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3851353280}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4026186380}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.270 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1071749891}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=339606421}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4013800114}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218936796}) 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.271 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=148208146}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900020134}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.067+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2724906993}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=993824402}) 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.272 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=617225677}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750647477}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1653370068}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.273 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=220187672}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4145539576}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=426751432}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2817060391}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3491382128}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.274 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3710575191}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=583503726}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4201518121}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=238422778}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.275 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2054683387}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=314494789}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2050426323}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2118555795}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.071+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2357164308}) 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.276 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3368628658}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2675129598}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2728737722}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3271261083}) 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.277 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3846711526}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951867241}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=786051250}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683472634}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2190180451}) 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.278 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=716918479}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3628095963}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1874417134}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1562983201}) 2024-09-17T16:44:02.279 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1624558316}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=643660412}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3711654160}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1212326389}) 2024-09-17T16:44:02.280 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.075+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2763265931}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3025848042}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1192178860}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1092451927}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.281 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3389779919}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2848944123}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=923874777}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1415709204}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3266491143}) 2024-09-17T16:44:02.282 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=732087316}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1259926627}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=568998247}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1434178415}) 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.283 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3604051893}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3326391609}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1514336897}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=455781358}) 2024-09-17T16:44:02.284 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557073968}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.079+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3925331546}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2465136562}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=955049608}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.285 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=770594789}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1064590317}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=634027883}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2538415699}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.286 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096382128}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935375288}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1193512021}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4113859787}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2655607621}) 2024-09-17T16:44:02.287 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=254017909}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4064487211}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1945189102}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.288 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3164377816}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1424288087}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=737875374}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=48309839}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.083+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1131863629}) 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.289 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1629321774}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2933991040}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3309838684}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=395010516}) 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.290 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1575490519}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2162902879}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2783118119}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3681037733}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735083894}) 2024-09-17T16:44:02.291 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3813544068}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753669909}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1062093171}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.292 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=887081957}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4050189682}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3835284528}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3552972157}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.087+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=296193236}) 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.293 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2258544790}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=410943197}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1063167526}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3741281360}) 2024-09-17T16:44:02.294 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1748099183}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269692820}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2568164596}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1397285687}) 2024-09-17T16:44:02.295 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2560370066}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=648977281}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1743303677}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279920422}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.296 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=881741751}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1333049096}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3403247062}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2180368071}) 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.297 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1820928197}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.091+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3177833925}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625614649}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907492422}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3360097498}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.298 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=947028936}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3452049307}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3386722177}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1143910215}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2070533317}) 2024-09-17T16:44:02.299 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1229062300}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1277051874}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1530771028}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2647397309}) 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.300 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4187194639}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1965625829}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2287367039}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3872557322}) 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.301 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2702753736}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.095+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=974393990}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2709840251}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3010547191}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=843572681}) 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.302 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2766047101}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1175173324}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=220818627}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2481298454}) 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.303 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986048534}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4178766732}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=365453024}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3979395103}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2636061976}) 2024-09-17T16:44:02.304 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4259994047}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2496601659}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1951600809}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.305 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=56076116}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769732464}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.099+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2190067864}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4256581477}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=50278706}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.306 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=966625898}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3390870987}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3836027304}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1943517391}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.307 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1017586508}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=483600661}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1348724544}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1035184391}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4229432973}) 2024-09-17T16:44:02.308 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=387654945}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=215918547}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2038996192}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2195022198}) 2024-09-17T16:44:02.309 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1696587884}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2495165294}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.103+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3745862663}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043319711}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2995615189}) 2024-09-17T16:44:02.310 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1915333458}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3908986263}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1169760466}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1452048929}) 2024-09-17T16:44:02.311 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3809016308}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238764580}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=422919115}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2474369423}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.312 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1546957598}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3128791753}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=525051264}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2080866219}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.313 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1339238761}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=429110177}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942064887}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.107+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2141759016}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322127941}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.314 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2064417520}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1081978668}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4184405805}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2537610633}) 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.315 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=946390277}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4084098290}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=958024960}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3892637961}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1615277516}) 2024-09-17T16:44:02.316 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1723602690}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3582184552}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3653781127}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3252631626}) 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.317 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1419827197}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1584783829}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255710708}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.111+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1836376943}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1934658990}) 2024-09-17T16:44:02.318 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2952666112}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3462797076}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4144254362}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4183026316}) 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.319 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=701393298}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=394007394}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963328882}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2977912371}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2562292883}) 2024-09-17T16:44:02.320 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3049441026}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1219941203}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3396501813}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2637447412}) 2024-09-17T16:44:02.321 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1229549103}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=905869106}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2679247608}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.115+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1085633788}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.322 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=4,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=11} crc {logm=385923954,mdsmap=952268285,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=732475678,mon_config_key=346208105,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=3719057555,osdmap=2754558469}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=6,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=10} crc {logm=522377848,mdsmap=2435593270,mgr=51957313,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1101488944,mon_config_key=4257234632,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2109140028,osdmap=1680357548}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=6,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=9} crc {logm=522377848,mdsmap=952268285,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1101488944,mon_config_key=2112433942,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=1960432555,osdmap=2449695256}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=8} crc {logm=522377848,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2855578965,mon_config_key=2681608797,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=3264471105,osdmap=3668461622}) 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.323 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=9} crc {logm=1545679444,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=4132097838,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=3523298669,osdmap=2060163404}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=10} crc {logm=3784268637,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=1771288005,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=4255104038,osdmap=3563981898}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=5,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=12} crc {logm=3272789069,mdsmap=1248375647,mgr=3567252286,mgr_metadata=3792311847,mgrstat=3382675010,mon_config_key=346208105,monmap=835142563,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2786937641,osdmap=3442561041}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=14} crc {logm=522377848,mdsmap=1248375647,mgr=1283586406,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=2117672606,mon_config_key=4027989049,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=2112738550,osdmap=1870156084}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=9} crc {logm=522377848,mdsmap=1248375647,mgr=3370157582,mgr_metadata=3792311847,mgrstat=947529578,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=3481485897,osd_snap=3117425281,osdmap=1307960058}) 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.324 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=908614500}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3322026993}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2620048637}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=376772656}) 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.325 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1265142489}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3056121937}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1282488549}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3499474135}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.119+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1515143713}) 2024-09-17T16:44:02.326 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2492909479}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3854086687}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=370318152}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.327 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2173740888}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2111643969}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2492712256}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=54293063}) 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.328 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=121295441}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.123+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1346264977}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=384839674}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2202699653}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=428224226}) 2024-09-17T16:44:02.329 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=530904574}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=498657337}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=908972277}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1318611476}) 2024-09-17T16:44:02.330 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2430756412}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=727997318}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1222933823}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=189577461}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1958154019}) 2024-09-17T16:44:02.331 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1859284515}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4165554023}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=224750101}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.332 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2388348788}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3874603094}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.127+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2555320087}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1348247836}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3921273721}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.333 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4266181470}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2323983450}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=397313507}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3483112982}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.334 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3704530174}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2431740011}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3019413932}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2169301645}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1728897841}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.335 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=219167421}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3565804389}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=973132841}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3535915867}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.131+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.336 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.135+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3569281167}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.135+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.135+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2539521282}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.135+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2013633037}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3926408399}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2913001048}) 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.337 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=875743240}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1687517473}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=654419404}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.338 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2527735118}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=960235860}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=648374516}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.139+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3570498937}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1676518828}) 2024-09-17T16:44:02.339 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4283511161}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4035433691}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=439776875}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.340 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2944199042}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1406959141}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3196744451}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1665151920}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.143+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3819901284}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.341 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3696929805}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=909405207}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=882068413}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3835973523}) 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.342 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2350692362}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=414475827}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1492783149}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=531260312}) 2024-09-17T16:44:02.343 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.147+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2196705552}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=21} crc {osdmap=4154031832}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=21} crc {osdmap=3477140298}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=20} crc {osdmap=537961015}) 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.344 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=22} crc {osdmap=619135465}) 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=21} crc {osdmap=2525007644}) 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=1769310884}) 2024-09-17T16:44:02.345 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:44:02.163+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=20} crc {osdmap=1202263923}) 2024-09-17T16:44:02.773 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:44:02.774 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:44:02.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: get_monmap_and_config 2024-09-17T16:44:02.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: build_initial_monmap 2024-09-17T16:44:02.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: monmap: 2024-09-17T16:44:02.885 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:02.881096+0000 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:02.881096+0000 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:02.886 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:02.887 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 Processor -- start 2024-09-17T16:44:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 -- start start 2024-09-17T16:44:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: init 2024-09-17T16:44:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:02.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient: _add_conns ranks=[9,5,4,7,10,1,8,6,0,3,2] 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-a con 0x7ff424138860 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138c40 0x7ff4240a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-h con 0x7ff424138c40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 0x7ff4240a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-c con 0x7ff424139080 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff42409da00 con 0x7ff424138860 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff42406c320 con 0x7ff424139080 2024-09-17T16:44:02.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff42409deb0 con 0x7ff424138c40 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42befb640 10 monclient(hunting): authenticate will time out at 1926.146843s 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 0x7ff4240a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36344/0 (socket says 172.21.15.105:36344) 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): get_auth_request con 0x7ff424139080 auth_method 0 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 1 -- 172.21.15.105:0/4212338232 learned_addr learned my addr 172.21.15.105:0/4212338232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:02.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): get_auth_request con 0x7ff424138860 auth_method 0 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:02.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient(hunting): handle_auth_done global_id 74704 payload 274 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 -- 172.21.15.105:0/4212338232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138c40 msgr2=0x7ff4240a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429496640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138c40 0x7ff4240a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138c40 0x7ff4240a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 -- 172.21.15.105:0/4212338232 >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 msgr2=0x7ff42406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 --2- 172.21.15.105:0/4212338232 >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient: _finish_hunting 0 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 monclient: found mon.noname-c 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:02.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff42a498640 1 -- 172.21.15.105:0/4212338232 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42409e190 con 0x7ff424139080 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.879+0000 7ff429c97640 1 --2- 172.21.15.105:0/4212338232 >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42a498640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 0x7ff4240a4b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0097f0 tx=0x7ff41c01c360 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2d17f6604c04a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 1 -- 172.21.15.105:0/4212338232 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff41c0b0070 con 0x7ff424139080 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: dump: 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:02.901 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: _finish_auth 0 2024-09-17T16:44:02.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: _check_auth_tickets 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 1 -- 172.21.15.105:0/4212338232 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c0aa810 con 0x7ff424139080 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 5 monclient: authenticate success, global_id 74704 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 1 -- 172.21.15.105:0/4212338232 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff41c0a9300 con 0x7ff424139080 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 4 set_mon_vals no callback set 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff428c95640 10 monclient: dump: 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:02.903 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:02.904 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 10 monclient: shutdown 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- 172.21.15.105:0/4212338232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 msgr2=0x7ff4240a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 0x7ff4240a4b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0097f0 tx=0x7ff41c01c360 comp rx=0 tx=0).stop 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- 172.21.15.105:0/4212338232 shutdown_connections 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff424139080 0x7ff4240a4b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 --2- 172.21.15.105:0/4212338232 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138c40 0x7ff4240a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 --2- 172.21.15.105:0/4212338232 >> v2:172.21.15.105:3300/0 conn(0x7ff424138860 0x7ff42406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- 172.21.15.105:0/4212338232 >> 172.21.15.105:0/4212338232 conn(0x7ff424068dd0 msgr2=0x7ff4240691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:02.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- 172.21.15.105:0/4212338232 shutdown_connections 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- 172.21.15.105:0/4212338232 wait complete. 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 10 monclient: build_initial_monmap 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 10 monclient: monmap: 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:02.906 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 librados: starting msgr at 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 librados: starting objecter 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 Processor -- start 2024-09-17T16:44:02.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 -- start start 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 librados: setting wanted keys 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 1 librados: calling monclient init 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.883+0000 7ff42befb640 10 monclient: init 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _add_conns ranks=[5,1,2,4,3,7,0,8,6] 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138860 0x7ff42413bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-h con 0x7ff424138860 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 0x7ff42413c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-f con 0x7ff424138c40 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): picked mon.noname-b con 0x7ff424139080 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:02.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241433e0 con 0x7ff424138c40 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff424143260 con 0x7ff424139080 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff424143560 con 0x7ff424138860 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient(hunting): authenticate will time out at 1926.152273s 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47144/0 (socket says 172.21.15.105:47144) 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 -- 172.21.15.105:0/2066728246 learned_addr learned my addr 172.21.15.105:0/2066728246 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:02.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): get_auth_request con 0x7ff424139080 auth_method 0 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient(hunting): handle_auth_done global_id 84386 payload 824 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429496640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 0x7ff42413c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429c97640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138860 0x7ff42413bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 -- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138860 msgr2=0x7ff42413bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138860 0x7ff42413bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 -- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 msgr2=0x7ff42413c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 0x7ff42413c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient: _finish_hunting 0 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 monclient: found mon.noname-b 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 -- 172.21.15.105:0/2066728246 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42413ed30 con 0x7ff424139080 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429496640 10 monclient: get_auth_request con 0x7ff424138c40 auth_method 0 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429496640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 0x7ff42413c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42a498640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff41c01cc00 tx=0x7ff41c019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 client.?.objecter ms_handle_connect 0x7ff424139080 2024-09-17T16:44:02.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff41c0b0050 con 0x7ff424139080 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: dump: 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:02.912 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:02.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c0b1040 con 0x7ff424139080 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 5 monclient: authenticate success, global_id 84386 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _renew_subs 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42413f2b0 con 0x7ff424139080 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff41c0044b0 con 0x7ff424139080 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 monclient: dump: 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:02.914 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:02.915 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff417fff640 4 set_mon_vals no callback set 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 client.84386.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _renew_subs 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42413e5b0 con 0x7ff424139080 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 librados: init done 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:02.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42406c5d0 con 0x7ff424139080 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff41c0be030 con 0x7ff424139080 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 0x7ff408040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(642..642 src has 1..642) ==== 6761+0+0 (secure 0 0 0) 0x7ff41c0bc020 con 0x7ff424139080 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 client.84386.objecter ms_dispatch 0x7ff424000ba0 osd_map(642..642 src has 1..642) 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 3 client.84386.objecter handle_osd_map got epochs [642,642] > 0 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 3 client.84386.objecter handle_osd_map decoding full epoch 642 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 20 client.84386.objecter dump_active .. 0 homeless 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429c97640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 0x7ff408040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429c97640 10 monclient: get_auth_request con 0x7ff40803e550 auth_method 0 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff4167fc640 10 client.84386.objecter ms_handle_connect 0x7ff40803e550 2024-09-17T16:44:02.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.887+0000 7ff429c97640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 0x7ff408040a10 secure :-1 s=READY pgs=3638 cs=0 l=1 rev1=1 crypto rx=0x7ff420009630 tx=0x7ff420008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:02.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.891+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff41c0c5060 con 0x7ff424139080 2024-09-17T16:44:02.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.891+0000 7ff4167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:02.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:02.891+0000 7ff4167fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff42befb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff42befb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff42befb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff42413f760 con 0x7ff424139080 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff4167fc640 1 -- 172.21.15.105:0/2066728246 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7ff41c016070 con 0x7ff424139080 2024-09-17T16:44:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff4167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:44:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.131+0000 7ff4167fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":102,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:44:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 10 librados: watch_flush enter 2024-09-17T16:44:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 10 librados: watch_flush exit 2024-09-17T16:44:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 20 client.84386.objecter shutdown clearing up homeless session... 2024-09-17T16:44:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 10 client.84386.objecter successfully canceled tick 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 msgr2=0x7ff408040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 0x7ff408040a10 secure :-1 s=READY pgs=3638 cs=0 l=1 rev1=1 crypto rx=0x7ff420009630 tx=0x7ff420008040 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 10 monclient: shutdown 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 msgr2=0x7ff42413c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff41c01cc00 tx=0x7ff41c019100 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 shutdown_connections 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff40803e550 0x7ff408040a10 unknown :-1 s=CLOSED pgs=3638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff424139080 0x7ff42413c780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff424138c40 0x7ff42413c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 --2- 172.21.15.105:0/2066728246 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff424138860 0x7ff42413bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 >> 172.21.15.105:0/2066728246 conn(0x7ff424068dd0 msgr2=0x7ff4240a7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 shutdown_connections 2024-09-17T16:44:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 -- 172.21.15.105:0/2066728246 wait complete. 2024-09-17T16:44:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.135+0000 7ff42befb640 1 librados: shutdown 2024-09-17T16:44:03.157 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:44:03.157 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-17T16:44:03.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:44:03.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 10 monclient: get_monmap_and_config 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 10 monclient: monmap: 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.307 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:03.302929+0000 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:03.302929+0000 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.308 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:03.309 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:03.309 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 1 Processor -- start 2024-09-17T16:44:03.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 1 -- start start 2024-09-17T16:44:03.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.299+0000 7fe90a04a640 10 monclient: init 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient: _add_conns ranks=[9,3,4,7,5,0,2,1,8,10,6] 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe904137660 0x7fe904137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-a con 0x7fe904137660 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 0x7fe9040a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-g con 0x7fe904137f80 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe9040a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-c con 0x7fe9040a1d50 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe90406dbe0 con 0x7fe904137660 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe9040774a0 con 0x7fe904137f80 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe90409e790 con 0x7fe9040a1d50 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient(hunting): authenticate will time out at 1926.568427s 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe904137660 0x7fe904137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe904137660 0x7fe904137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36356/0 (socket says 172.21.15.105:36356) 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe9040a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 -- 172.21.15.105:0/4082425086 learned_addr learned my addr 172.21.15.105:0/4082425086 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): get_auth_request con 0x7fe904137660 auth_method 0 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): get_auth_request con 0x7fe9040a1d50 auth_method 0 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe902ffd640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 0x7fe9040a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient(hunting): handle_auth_done global_id 36585 payload 274 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 -- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 msgr2=0x7fe9040a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe9040a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 -- 172.21.15.105:0/4082425086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 msgr2=0x7fe9040a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 0x7fe9040a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 monclient: found mon.noname-a 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 -- 172.21.15.105:0/4082425086 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90409c300 con 0x7fe904137660 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe902ffd640 10 monclient: get_auth_request con 0x7fe904137f80 auth_method 0 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe902ffd640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 0x7fe9040a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9037fe640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe904137660 0x7fe904137a40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40098a0 tx=0x7fe8f401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0efa01fa43719e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 1 -- 172.21.15.105:0/4082425086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f40b0070 con 0x7fe904137660 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: dump: 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.322 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 1 -- 172.21.15.105:0/4082425086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40aa810 con 0x7fe904137660 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 5 monclient: authenticate success, global_id 36585 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:03.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 1 -- 172.21.15.105:0/4082425086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f40a9300 con 0x7fe904137660 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 4 set_mon_vals no callback set 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe9027fc640 10 monclient: dump: 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.324 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 10 monclient: shutdown 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe904137660 msgr2=0x7fe904137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe904137660 0x7fe904137a40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40098a0 tx=0x7fe8f401c400 comp rx=0 tx=0).stop 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- 172.21.15.105:0/4082425086 shutdown_connections 2024-09-17T16:44:03.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe9040a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe904137f80 0x7fe9040a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 --2- 172.21.15.105:0/4082425086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe904137660 0x7fe904137a40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- 172.21.15.105:0/4082425086 >> 172.21.15.105:0/4082425086 conn(0x7fe9040a4bf0 msgr2=0x7fe9040a4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- 172.21.15.105:0/4082425086 shutdown_connections 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.303+0000 7fe90a04a640 1 -- 172.21.15.105:0/4082425086 wait complete. 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: monmap: 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:03.326 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 librados: starting msgr at 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 librados: starting objecter 2024-09-17T16:44:03.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 Processor -- start 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- start start 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 librados: setting wanted keys 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 librados: calling monclient init 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: init 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _add_conns ranks=[4,6,1,2,7,5,8,3,0] 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe904136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-c con 0x7fe9040a1d50 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe904137660 0x7fe90409f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-d con 0x7fe904137660 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe904137f80 0x7fe90409fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): picked mon.noname-f con 0x7fe904137f80 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:03.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe904141020 con 0x7fe904137f80 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe9041411a0 con 0x7fe9040a1d50 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe904141320 con 0x7fe904137660 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): authenticate will time out at 1926.573384s 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe904136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe904137660 0x7fe90409f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe904136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34126/0 (socket says 172.21.15.105:34126) 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 -- 172.21.15.105:0/603414324 learned_addr learned my addr 172.21.15.105:0/603414324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): get_auth_request con 0x7fe9040a1d50 auth_method 0 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): get_auth_request con 0x7fe904137660 auth_method 0 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe904137f80 0x7fe90409fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): get_auth_request con 0x7fe904137f80 auth_method 0 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient(hunting): handle_auth_done global_id 74722 payload 824 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe904137660 msgr2=0x7fe90409f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe904137660 0x7fe90409f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe904137f80 msgr2=0x7fe90409fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe904137f80 0x7fe90409fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 monclient: found mon.noname-c 2024-09-17T16:44:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe904074250 con 0x7fe9040a1d50 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe9037fe640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe904136aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ae040 tx=0x7fe8f40191a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 client.?.objecter ms_handle_connect 0x7fe9040a1d50 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f40b9070 con 0x7fe9040a1d50 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: dump: 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40044b0 con 0x7fe9040a1d50 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f40b0040 con 0x7fe9040a1d50 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 5 monclient: authenticate success, global_id 74722 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _renew_subs 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90413b8a0 con 0x7fe9040a1d50 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe901ffb640 4 set_mon_vals no callback set 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: dump: 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.334 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 client.74722.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _renew_subs 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9040a3a10 con 0x7fe9040a1d50 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 librados: init done 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 0x7fe9040a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040a12d0 con 0x7fe9040a0070 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 0x7fe9040a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient: get_auth_request con 0x7fe9040a0070 auth_method 0 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe8f4004040 con 0x7fe9040a1d50 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe8f003e550 0x7fe8f0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 10 monclient(hunting): handle_auth_done global_id 36594 payload 824 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(643..643 src has 1..643) ==== 6388+0+0 (secure 0 0 0) 0x7fe8f40e9160 con 0x7fe9040a1d50 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 client.74722.objecter ms_dispatch 0x7fe904000ba0 osd_map(643..643 src has 1..643) 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 3 client.74722.objecter handle_osd_map got epochs [643,643] > 0 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 3 client.74722.objecter handle_osd_map decoding full epoch 643 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe904137a40 con 0x7fe9040a0070 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 20 client.74722.objecter dump_active .. 0 homeless 2024-09-17T16:44:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 client.74722.objecter ms_handle_connect 0x7fe9040a0070 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe902ffd640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 0x7fe9040a0450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 client.74722.objecter resend_mon_ops 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe8f003e550 0x7fe8f0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f800de60 con 0x7fe9040a0070 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe8ebfff640 10 monclient: dump: 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.338 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.307+0000 7fe903fff640 10 monclient: get_auth_request con 0x7fe8f003e550 auth_method 0 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe8f800fe50 con 0x7fe9040a0070 2024-09-17T16:44:03.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 msgr2=0x7fe9040a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 0x7fe9040a0450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f800fc00 comp rx=0 tx=0).stop 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 msgr2=0x7fe9040a0450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9040a0070 0x7fe9040a0450 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f800fc00 comp rx=0 tx=0).stop 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe903fff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe8f003e550 0x7fe8f0040a10 secure :-1 s=READY pgs=3639 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a2150 tx=0x7fe8fc00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.311+0000 7fe8ebfff640 10 client.74722.objecter ms_handle_connect 0x7fe8f003e550 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 0x7fe90409a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041374d0 con 0x7fe90409a250 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 0x7fe90409a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient: get_auth_request con 0x7fe90409a250 auth_method 0 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 10 monclient(hunting): handle_auth_done global_id 36603 payload 824 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 1 -- 172.21.15.105:0/603414324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe9040a0990 con 0x7fe90409a250 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 client.74722.objecter ms_handle_connect 0x7fe90409a250 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 client.74722.objecter resend_mon_ops 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe902ffd640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 0x7fe90409a660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8f80064b0 con 0x7fe90409a250 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 monclient: dump: 2024-09-17T16:44:03.343 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.344 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9455 (secure 0 0 0) 0x7fe8f8005fa0 con 0x7fe90409a250 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 msgr2=0x7fe90409a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 0x7fe90409a660 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f8004740 comp rx=0 tx=0).stop 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 msgr2=0x7fe90409a660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe8ebfff640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe90409a250 0x7fe90409a660 secure :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a0c60 tx=0x7fe8f8004740 comp rx=0 tx=0).stop 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:03.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 834118, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:03.346 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.347 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.348 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:03.349 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:03.350 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:03.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:03.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:03.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.356 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:03.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.361 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.362 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:03.363 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.364 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.365 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.366 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.367 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:03.368 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:03.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 librados: watch_flush enter 2024-09-17T16:44:03.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 librados: watch_flush exit 2024-09-17T16:44:03.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 20 client.74722.objecter shutdown clearing up homeless session... 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 client.74722.objecter successfully canceled tick 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe8f003e550 msgr2=0x7fe8f0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe8f003e550 0x7fe8f0040a10 secure :-1 s=READY pgs=3639 cs=0 l=1 rev1=1 crypto rx=0x7fe9040a2150 tx=0x7fe8fc00b040 comp rx=0 tx=0).stop 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 10 monclient: shutdown 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 msgr2=0x7fe904136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 --2- 172.21.15.105:0/603414324 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe9040a1d50 0x7fe904136aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40ae040 tx=0x7fe8f40191a0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe903fff640 1 -- 172.21.15.105:0/603414324 reap_dead start 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe903fff640 1 -- 172.21.15.105:0/603414324 reap_dead start 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 shutdown_connections 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 >> 172.21.15.105:0/603414324 conn(0x7fe9040a4bf0 msgr2=0x7fe9040a5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 shutdown_connections 2024-09-17T16:44:03.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.315+0000 7fe90a04a640 1 -- 172.21.15.105:0/603414324 wait complete. 2024-09-17T16:44:03.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.319+0000 7fe90a04a640 1 librados: shutdown 2024-09-17T16:44:03.372 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:03.372 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:44:03.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:44:03.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: get_monmap_and_config 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: monmap: 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:03.488456+0000 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:03.488456+0000 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.493 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:03.494 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 Processor -- start 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 -- start start 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: init 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient: _add_conns ranks=[7,0,2,9,4,1,8,6,5,10,3] 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff9580a5860 0x7ff9580a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-i con 0x7ff9580a5860 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff9580a6180 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:03.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 0x7ff958141c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-b con 0x7ff95813a110 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95806d8a0 con 0x7ff9580a6180 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff95806d460 con 0x7ff95813a110 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff95806ebc0 con 0x7ff9580a5860 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95f0c3640 10 monclient(hunting): authenticate will time out at 1926.754560s 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 0x7ff958141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff957fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff9580a6180 0x7ff958139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42276/0 (socket says 172.21.15.105:42276) 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff957fff640 1 -- 172.21.15.105:0/4157067684 learned_addr learned my addr 172.21.15.105:0/4157067684 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): get_auth_request con 0x7ff95813a110 auth_method 0 2024-09-17T16:44:03.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95ce5f640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff9580a5860 0x7ff9580a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95ce5f640 10 monclient(hunting): get_auth_request con 0x7ff9580a5860 auth_method 0 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95ce5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95ce5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95ce5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 1 -- 172.21.15.105:0/4157067684 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044256807 0 0) 0x7ff95806d8a0 con 0x7ff9580a6180 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 10 monclient(hunting): _negotiate 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 10 monclient(hunting): my global_id is 36612 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff9577fe640 1 -- 172.21.15.105:0/4157067684 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff940003530 con 0x7ff9580a6180 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient(hunting): handle_auth_done global_id 84395 payload 274 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 -- 172.21.15.105:0/4157067684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff9580a5860 msgr2=0x7ff9580a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff9580a5860 0x7ff9580a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 -- 172.21.15.105:0/4157067684 >> v1:172.21.15.105:6789/0 conn(0x7ff9580a6180 legacy=0x7ff958139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 monclient: found mon.noname-b 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.487+0000 7ff95d660640 1 -- 172.21.15.105:0/4157067684 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9580a7320 con 0x7ff95813a110 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 0x7ff958141c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff950009dc0 tx=0x7ff95001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bd3bbf7649ab7c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 1 -- 172.21.15.105:0/4157067684 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9500b2070 con 0x7ff95813a110 2024-09-17T16:44:03.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: dump: 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.501 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 1 -- 172.21.15.105:0/4157067684 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9500a5a50 con 0x7ff95813a110 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 1 -- 172.21.15.105:0/4157067684 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9500a5c40 con 0x7ff95813a110 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 5 monclient: authenticate success, global_id 84395 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:03.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 4 set_mon_vals no callback set 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff9577fe640 10 monclient: dump: 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.503 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.504 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.504 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.504 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.504 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.504 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: shutdown 2024-09-17T16:44:03.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- 172.21.15.105:0/4157067684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 msgr2=0x7ff958141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 0x7ff958141c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff950009dc0 tx=0x7ff95001c850 comp rx=0 tx=0).stop 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- 172.21.15.105:0/4157067684 shutdown_connections 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95813a110 0x7ff958141c60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/4157067684 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff9580a5860 0x7ff9580a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- 172.21.15.105:0/4157067684 >> 172.21.15.105:0/4157067684 conn(0x7ff9580aa450 msgr2=0x7ff9580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- 172.21.15.105:0/4157067684 shutdown_connections 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- 172.21.15.105:0/4157067684 wait complete. 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: monmap: 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:03.512 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 librados: starting msgr at 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 librados: starting objecter 2024-09-17T16:44:03.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 Processor -- start 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- start start 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 librados: setting wanted keys 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 librados: calling monclient init 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: init 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient: _add_conns ranks=[2,8,4,3,0,1,5,7,6] 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 0x7ff958138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-b con 0x7ff9580a5860 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9580a6180 0x7ff958138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-e con 0x7ff9580a6180 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff95813a110 0x7ff9581394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): picked mon.noname-c con 0x7ff95813a110 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff95806f390 con 0x7ff9580a5860 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff9580a0f00 con 0x7ff95813a110 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff9581449c0 con 0x7ff9580a6180 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95f0c3640 10 monclient(hunting): authenticate will time out at 1926.759572s 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 0x7ff958138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 0x7ff958138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47170/0 (socket says 172.21.15.105:47170) 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 1 -- 172.21.15.105:0/952885809 learned_addr learned my addr 172.21.15.105:0/952885809 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9580a6180 0x7ff958138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): get_auth_request con 0x7ff9580a5860 auth_method 0 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff95813a110 0x7ff9581394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): get_auth_request con 0x7ff9580a6180 auth_method 0 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): get_auth_request con 0x7ff95813a110 auth_method 0 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95ce5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff95d660640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.491+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 10 monclient(hunting): handle_auth_done global_id 84404 payload 824 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9580a6180 msgr2=0x7ff958138f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9580a6180 0x7ff958138f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff95813a110 msgr2=0x7ff9581394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff95813a110 0x7ff9581394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 monclient: found mon.noname-b 2024-09-17T16:44:03.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9580a38e0 con 0x7ff9580a5860 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95ce5f640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 0x7ff958138a40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00d800 tx=0x7ff94c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.?.objecter ms_handle_connect 0x7ff9580a5860 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff94c00bad0 con 0x7ff9580a5860 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: dump: 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.518 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.519 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 5 monclient: authenticate success, global_id 84404 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94c019070 con 0x7ff9580a5860 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 monclient: _renew_subs 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95813d110 con 0x7ff9580a5860 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff94c00bc90 con 0x7ff9580a5860 2024-09-17T16:44:03.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff956ffd640 4 set_mon_vals no callback set 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 client.84404.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: dump: 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.521 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 monclient: _renew_subs 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95813c870 con 0x7ff9580a5860 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 1 librados: init done 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 0x7ff95809dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff958051e80 con 0x7ff95809da00 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 0x7ff95809dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient: get_auth_request con 0x7ff95809da00 auth_method 0 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff94c00b660 con 0x7ff9580a5860 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 10 monclient(hunting): handle_auth_done global_id 84413 payload 824 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff95813cd90 con 0x7ff95809da00 2024-09-17T16:44:03.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff94403e550 0x7ff944040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(643..643 src has 1..643) ==== 6388+0+0 (secure 0 0 0) 0x7ff94c00f040 con 0x7ff9580a5860 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.84404.objecter ms_dispatch 0x7ff958000ba0 osd_map(643..643 src has 1..643) 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 3 client.84404.objecter handle_osd_map got epochs [643,643] > 0 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 3 client.84404.objecter handle_osd_map decoding full epoch 643 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff957fff640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 0x7ff95809dde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff95813a0e0 tx=0x7ff9480019e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 20 client.84404.objecter dump_active .. 0 homeless 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.84404.objecter ms_handle_connect 0x7ff95809da00 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.84404.objecter resend_mon_ops 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff948019070 con 0x7ff95809da00 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: dump: 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.524 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.525 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95d660640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff94403e550 0x7ff944040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95d660640 10 monclient: get_auth_request con 0x7ff94403e550 auth_method 0 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff948005ad0 con 0x7ff95809da00 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 msgr2=0x7ff95809dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 0x7ff95809dde0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff95813a0e0 tx=0x7ff9480019e0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 msgr2=0x7ff95809dde0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff95809da00 0x7ff95809dde0 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff95813a0e0 tx=0x7ff9480019e0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff9557fa640 10 client.84404.objecter ms_handle_connect 0x7ff94403e550 2024-09-17T16:44:03.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.495+0000 7ff95d660640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff94403e550 0x7ff944040a10 secure :-1 s=READY pgs=3640 cs=0 l=1 rev1=1 crypto rx=0x7ff9500af000 tx=0x7ff9500a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 0x7ff9581db920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff95813cbb0 con 0x7ff9580a9330 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 0x7ff9581db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient: get_auth_request con 0x7ff9580a9330 auth_method 0 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 10 monclient(hunting): handle_auth_done global_id 84422 payload 824 2024-09-17T16:44:03.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 1 -- 172.21.15.105:0/952885809 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff9580a5c40 con 0x7ff9580a9330 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff957fff640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 0x7ff9581db920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff958069dd0 tx=0x7ff948005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 client.84404.objecter ms_handle_connect 0x7ff9580a9330 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 client.84404.objecter resend_mon_ops 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff948019030 con 0x7ff9580a9330 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 monclient: dump: 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.528 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7ff948014450 con 0x7ff9580a9330 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:03.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 msgr2=0x7ff9581db920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 0x7ff9581db920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff958069dd0 tx=0x7ff948005800 comp rx=0 tx=0).stop 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 msgr2=0x7ff9581db920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff9557fa640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a9330 0x7ff9581db920 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff958069dd0 tx=0x7ff948005800 comp rx=0 tx=0).stop 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 150738, 2024-09-17T16:44:03.530 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:03.531 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.532 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.533 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:03.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.535 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.536 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.537 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.538 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:03.539 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.541 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.542 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:03.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.545 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:03.546 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.547 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:03.548 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.550 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:03.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:03.552 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 librados: watch_flush enter 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 librados: watch_flush exit 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 20 client.84404.objecter shutdown clearing up homeless session... 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 client.84404.objecter successfully canceled tick 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff94403e550 msgr2=0x7ff944040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff94403e550 0x7ff944040a10 secure :-1 s=READY pgs=3640 cs=0 l=1 rev1=1 crypto rx=0x7ff9500af000 tx=0x7ff9500a7070 comp rx=0 tx=0).stop 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 10 monclient: shutdown 2024-09-17T16:44:03.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 msgr2=0x7ff958138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 --2- 172.21.15.105:0/952885809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9580a5860 0x7ff958138a40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00d800 tx=0x7ff94c00dcc0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95d660640 1 -- 172.21.15.105:0/952885809 reap_dead start 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95d660640 1 -- 172.21.15.105:0/952885809 reap_dead start 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 shutdown_connections 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 >> 172.21.15.105:0/952885809 conn(0x7ff9580aa450 msgr2=0x7ff9580732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 shutdown_connections 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 -- 172.21.15.105:0/952885809 wait complete. 2024-09-17T16:44:03.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.503+0000 7ff95f0c3640 1 librados: shutdown 2024-09-17T16:44:03.555 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:03.556 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:44:03.556 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:44:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: get_monmap_and_config 2024-09-17T16:44:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: monmap: 2024-09-17T16:44:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:03.672347+0000 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:03.672347+0000 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:03.678 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 Processor -- start 2024-09-17T16:44:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 -- start start 2024-09-17T16:44:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: init 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient: _add_conns ranks=[0,2,7,9,10,5,4,3,1,6,8] 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60b00a3e60 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 0x7f60b013ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-b con 0x7f60b00a01f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f60b00a4950 0x7f60b013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-i con 0x7f60b00a4950 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b009ebe0 con 0x7f60b00a3e60 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f60b006d390 con 0x7f60b00a01f0 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f60b00a1df0 con 0x7f60b00a4950 2024-09-17T16:44:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b7dee640 10 monclient(hunting): authenticate will time out at 1926.938119s 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5b8a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f60b00a3e60 0x7f60b00a4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42280/0 (socket says 172.21.15.105:42280) 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5b8a640 1 -- 172.21.15.105:0/3217189822 learned_addr learned my addr 172.21.15.105:0/3217189822 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 0x7f60b013ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): get_auth_request con 0x7f60b00a01f0 auth_method 0 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient(hunting): handle_auth_done global_id 84431 payload 274 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 -- 172.21.15.105:0/3217189822 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f60b00a4950 msgr2=0x7f60b013ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f60b00a4950 0x7f60b013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b4b88640 1 -- 172.21.15.105:0/3217189822 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032477074 0 0) 0x7f60b009ebe0 con 0x7f60b00a3e60 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 -- 172.21.15.105:0/3217189822 >> v1:172.21.15.105:6789/0 conn(0x7f60b00a3e60 legacy=0x7f60b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 monclient: found mon.noname-b 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b5389640 1 -- 172.21.15.105:0/3217189822 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b009f220 con 0x7f60b00a01f0 2024-09-17T16:44:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.671+0000 7f60b4b88640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:44:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 0x7f60b013ca00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097f0 tx=0x7f60a001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333c8c234b37649e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 1 -- 172.21.15.105:0/3217189822 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a00b0070 con 0x7f60b00a01f0 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: dump: 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 1 -- 172.21.15.105:0/3217189822 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a00aa810 con 0x7f60b00a01f0 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 1 -- 172.21.15.105:0/3217189822 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a00aae30 con 0x7f60b00a01f0 2024-09-17T16:44:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 5 monclient: authenticate success, global_id 84431 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 4 set_mon_vals no callback set 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b4b88640 10 monclient: dump: 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.693 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: shutdown 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- 172.21.15.105:0/3217189822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 msgr2=0x7f60b013ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 0x7f60b013ca00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097f0 tx=0x7f60a001c3a0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- 172.21.15.105:0/3217189822 shutdown_connections 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f60b00a4950 0x7f60b013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3217189822 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f60b00a01f0 0x7f60b013ca00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- 172.21.15.105:0/3217189822 >> 172.21.15.105:0/3217189822 conn(0x7f60b00a4f50 msgr2=0x7f60b00a5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- 172.21.15.105:0/3217189822 shutdown_connections 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- 172.21.15.105:0/3217189822 wait complete. 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: monmap: 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.695 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 librados: starting msgr at 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 librados: starting objecter 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 Processor -- start 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- start start 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 librados: setting wanted keys 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 librados: calling monclient init 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: init 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient: _add_conns ranks=[3,6,1,0,4,7,2,8,5] 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60b00a01f0 0x7f60b01cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-g con 0x7f60b00a01f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-d con 0x7f60b00a3e60 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60b00a4950 0x7f60b01cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): picked mon.noname-f con 0x7f60b00a4950 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0142040 con 0x7f60b00a4950 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f60b0141ec0 con 0x7f60b00a01f0 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f60b01421c0 con 0x7f60b00a3e60 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b7dee640 10 monclient(hunting): authenticate will time out at 1926.943220s 2024-09-17T16:44:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39270/0 (socket says 172.21.15.105:39270) 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 -- 172.21.15.105:0/3705814483 learned_addr learned my addr 172.21.15.105:0/3705814483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): get_auth_request con 0x7f60b00a3e60 auth_method 0 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5b8a640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60b00a01f0 0x7f60b01cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5b8a640 10 monclient(hunting): get_auth_request con 0x7f60b00a01f0 auth_method 0 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5b8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5b8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5b8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b638b640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60b00a4950 0x7f60b01cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b638b640 10 monclient(hunting): get_auth_request con 0x7f60b00a4950 auth_method 0 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b638b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b638b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b638b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient(hunting): handle_auth_done global_id 94200 payload 824 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60b00a01f0 msgr2=0x7f60b01cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60b00a01f0 0x7f60b01cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60b00a4950 msgr2=0x7f60b01cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f60b00a4950 0x7f60b01cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 monclient: found mon.noname-d 2024-09-17T16:44:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b013de40 con 0x7f60b00a3e60 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60b5389640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097c0 tx=0x7f60a0019140 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60a67fc640 10 client.?.objecter ms_handle_connect 0x7f60b00a3e60 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.675+0000 7f60a67fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a00b0040 con 0x7f60b00a3e60 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: dump: 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a00044b0 con 0x7f60b00a3e60 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 5 monclient: authenticate success, global_id 94200 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 monclient: _renew_subs 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b012d510 con 0x7f60b00a3e60 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a0003a90 con 0x7f60b00a3e60 2024-09-17T16:44:03.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a7fff640 4 set_mon_vals no callback set 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 client.94200.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: dump: 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 monclient: _renew_subs 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b013ed90 con 0x7f60b00a3e60 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 1 librados: init done 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 0x7f60b01387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f60b0051e80 con 0x7f60b012e600 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 0x7f60b01387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient: get_auth_request con 0x7f60b012e600 auth_method 0 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f60a0003c50 con 0x7f60b00a3e60 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_done global_id 74740 payload 824 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60b009fe40 con 0x7f60b012e600 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f609403e550 0x7f6094040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(643..643 src has 1..643) ==== 6388+0+0 (secure 0 0 0) 0x7f60a00e9c10 con 0x7f60b00a3e60 2024-09-17T16:44:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 client.94200.objecter ms_dispatch 0x7f60b0000ba0 osd_map(643..643 src has 1..643) 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 3 client.94200.objecter handle_osd_map got epochs [643,643] > 0 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 3 client.94200.objecter handle_osd_map decoding full epoch 643 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b5b8a640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 0x7f60b01387d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000f70 tx=0x7f60ac0019c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 20 client.94200.objecter dump_active .. 0 homeless 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 client.94200.objecter ms_handle_connect 0x7f60b012e600 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 client.94200.objecter resend_mon_ops 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60ac019070 con 0x7f60b012e600 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: dump: 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b638b640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f609403e550 0x7f6094040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b638b640 10 monclient: get_auth_request con 0x7f609403e550 auth_method 0 2024-09-17T16:44:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f60ac010510 con 0x7f60b012e600 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 msgr2=0x7f60b01387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 0x7f60b01387d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000f70 tx=0x7f60ac0019c0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 msgr2=0x7f60b01387d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b012e600 0x7f60b01387d0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000f70 tx=0x7f60ac0019c0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60b638b640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f609403e550 0x7f6094040a10 secure :-1 s=READY pgs=3641 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009ea0 tx=0x7f60a8009410 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.679+0000 7f60a67fc640 10 client.94200.objecter ms_handle_connect 0x7f609403e550 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 0x7f60b0139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f60b00a4240 con 0x7f60b0138d10 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 0x7f60b0139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient: get_auth_request con 0x7f60b0138d10 auth_method 0 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 10 monclient(hunting): handle_auth_done global_id 74749 payload 824 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 1 -- 172.21.15.105:0/3705814483 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f60b012e9e0 con 0x7f60b0138d10 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 client.94200.objecter ms_handle_connect 0x7f60b0138d10 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 client.94200.objecter resend_mon_ops 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b5b8a640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 0x7f60b0139120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f60ac016000 tx=0x7f60ac000ee0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60ac019040 con 0x7f60b0138d10 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 monclient: dump: 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9622 (secure 0 0 0) 0x7f60ac014480 con 0x7f60b0138d10 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 msgr2=0x7f60b0139120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 0x7f60b0139120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f60ac016000 tx=0x7f60ac000ee0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 msgr2=0x7f60b0139120 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60a67fc640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f60b0138d10 0x7f60b0139120 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f60ac016000 tx=0x7f60ac000ee0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.713 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 283149, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:03.714 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.715 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.716 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:03.717 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.718 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.719 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.720 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:03.721 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:03.723 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:03.724 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.725 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.728 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.729 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.730 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.731 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.732 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.733 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.734 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:44:03.735 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:03.736 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 librados: watch_flush enter 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 librados: watch_flush exit 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 20 client.94200.objecter shutdown clearing up homeless session... 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 client.94200.objecter successfully canceled tick 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f609403e550 msgr2=0x7f6094040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f609403e550 0x7f6094040a10 secure :-1 s=READY pgs=3641 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009ea0 tx=0x7f60a8009410 comp rx=0 tx=0).stop 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 10 monclient: shutdown 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 msgr2=0x7f60b01cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60a00097c0 tx=0x7f60a0019140 comp rx=0 tx=0).stop 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b638b640 1 -- 172.21.15.105:0/3705814483 reap_dead start 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 shutdown_connections 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 --2- 172.21.15.105:0/3705814483 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f60b00a3e60 0x7f60b01cf610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 >> 172.21.15.105:0/3705814483 conn(0x7f60b00a4f50 msgr2=0x7f60b00724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 shutdown_connections 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 -- 172.21.15.105:0/3705814483 wait complete. 2024-09-17T16:44:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.687+0000 7f60b7dee640 1 librados: shutdown 2024-09-17T16:44:03.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:03.739 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:44:03.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:44:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: get_monmap_and_config 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: monmap: 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:03.852710+0000 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:03.852710+0000 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:03.858 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 Processor -- start 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 -- start start 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: init 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient: _add_conns ranks=[2,6,4,10,5,9,7,0,3,1,8] 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9140a68e0 0x7ff9140a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-b con 0x7ff9140a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-d con 0x7ff9140a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9141396a0 0x7ff914139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-c con 0x7ff9141396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff9140a3fc0 con 0x7ff9140a68e0 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff9140a63c0 con 0x7ff9141396a0 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff9140a7740 con 0x7ff9140a4910 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91889a640 10 monclient(hunting): authenticate will time out at 1927.118991s 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39282/0 (socket says 172.21.15.105:39282) 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 -- 172.21.15.105:0/3451756857 learned_addr learned my addr 172.21.15.105:0/3451756857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): get_auth_request con 0x7ff9140a4910 auth_method 0 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9140a68e0 0x7ff9140a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9141396a0 0x7ff914139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): get_auth_request con 0x7ff9140a68e0 auth_method 0 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): get_auth_request con 0x7ff9141396a0 auth_method 0 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient(hunting): handle_auth_done global_id 94209 payload 274 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 -- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9141396a0 msgr2=0x7ff914139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9141396a0 0x7ff914139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 -- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9140a68e0 msgr2=0x7ff9140a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9140a68e0 0x7ff9140a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 monclient: found mon.noname-d 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.851+0000 7ff911d9b640 1 -- 172.21.15.105:0/3451756857 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91406ec10 con 0x7ff9140a4910 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff911d9b640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9080098d0 tx=0x7ff90801cea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e39af80217b8b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 1 -- 172.21.15.105:0/3451756857 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9080b0070 con 0x7ff9140a4910 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: dump: 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.866 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 1 -- 172.21.15.105:0/3451756857 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9080aa810 con 0x7ff9140a4910 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 5 monclient: authenticate success, global_id 94209 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 1 -- 172.21.15.105:0/3451756857 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9080a9300 con 0x7ff9140a4910 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 4 set_mon_vals no callback set 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91159a640 10 monclient: dump: 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.868 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.869 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 10 monclient: shutdown 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 msgr2=0x7ff9140a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9080098d0 tx=0x7ff90801cea0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- 172.21.15.105:0/3451756857 shutdown_connections 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9141396a0 0x7ff914139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9140a4910 0x7ff9140a7200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 --2- 172.21.15.105:0/3451756857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9140a68e0 0x7ff9140a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- 172.21.15.105:0/3451756857 >> 172.21.15.105:0/3451756857 conn(0x7ff914068ba0 msgr2=0x7ff914068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- 172.21.15.105:0/3451756857 shutdown_connections 2024-09-17T16:44:03.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- 172.21.15.105:0/3451756857 wait complete. 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 10 monclient: build_initial_monmap 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 10 monclient: monmap: 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:03.879 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 librados: starting msgr at 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 librados: starting objecter 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 Processor -- start 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 -- start start 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 librados: setting wanted keys 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 1 librados: calling monclient init 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.855+0000 7ff91889a640 10 monclient: init 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _add_conns ranks=[0,4,8,5,6,3,2,1,7] 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9140a4910 0x7ff914122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-a con 0x7ff9140a4910 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-c con 0x7ff9140a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9141396a0 0x7ff91413d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): picked mon.noname-e con 0x7ff9141396a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff914143320 con 0x7ff9140a4910 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7ff91406e630 con 0x7ff9140a68e0 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff9141434a0 con 0x7ff9141396a0 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): authenticate will time out at 1927.124142s 2024-09-17T16:44:03.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9140a4910 0x7ff914122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9141396a0 0x7ff91413d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34176/0 (socket says 172.21.15.105:34176) 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9140a4910 0x7ff914122570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36388/0 (socket says 172.21.15.105:36388) 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 -- 172.21.15.105:0/133063309 learned_addr learned my addr 172.21.15.105:0/133063309 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): get_auth_request con 0x7ff9140a68e0 auth_method 0 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): get_auth_request con 0x7ff9141396a0 auth_method 0 2024-09-17T16:44:03.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): get_auth_request con 0x7ff9140a4910 auth_method 0 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient(hunting): handle_auth_done global_id 74767 payload 824 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9141396a0 msgr2=0x7ff91413d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff9141396a0 0x7ff91413d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9140a4910 msgr2=0x7ff914122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9140a4910 0x7ff914122570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 monclient: found mon.noname-c 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:03.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff914123a90 con 0x7ff9140a68e0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff911d9b640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ae040 tx=0x7ff9080191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.?.objecter ms_handle_connect 0x7ff9140a68e0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9080b9070 con 0x7ff9140a68e0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: dump: 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.886 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.887 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9080044b0 con 0x7ff9140a68e0 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9080b0040 con 0x7ff9140a68e0 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: dump: 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.888 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.889 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff910d99640 4 set_mon_vals no callback set 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 5 monclient: authenticate success, global_id 74767 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _renew_subs 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff914123d70 con 0x7ff9140a68e0 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 client.74767.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _renew_subs 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9140a9fc0 con 0x7ff9140a68e0 2024-09-17T16:44:03.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 librados: init done 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 0x7ff91413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff9141c8330 con 0x7ff9141d1f70 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 0x7ff91413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient: get_auth_request con 0x7ff9141d1f70 auth_method 0 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff9080aac00 con 0x7ff9140a68e0 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 10 monclient(hunting): handle_auth_done global_id 94218 payload 824 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 0x7ff8f4040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9140abb70 con 0x7ff9141d1f70 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(643..643 src has 1..643) ==== 6388+0+0 (secure 0 0 0) 0x7ff9080e9420 con 0x7ff9140a68e0 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.74767.objecter ms_dispatch 0x7ff914000ba0 osd_map(643..643 src has 1..643) 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 3 client.74767.objecter handle_osd_map got epochs [643,643] > 0 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 3 client.74767.objecter handle_osd_map decoding full epoch 643 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 20 client.74767.objecter dump_active .. 0 homeless 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.74767.objecter ms_handle_connect 0x7ff9141d1f70 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.74767.objecter resend_mon_ops 2024-09-17T16:44:03.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 0x7ff91413db60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff904011dc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff904019070 con 0x7ff9141d1f70 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: dump: 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.893 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 0x7ff8f4040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 10 monclient: get_auth_request con 0x7ff8f403e540 auth_method 0 2024-09-17T16:44:03.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff904004300 con 0x7ff9141d1f70 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 msgr2=0x7ff91413db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 0x7ff91413db60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff904011dc0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 msgr2=0x7ff91413db60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff9141d1f70 0x7ff91413db60 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff904011dc0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff912d9d640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 0x7ff8f4040a00 secure :-1 s=READY pgs=3642 cs=0 l=1 rev1=1 crypto rx=0x7ff914074620 tx=0x7ff90c00b040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.859+0000 7ff902ffd640 10 client.74767.objecter ms_handle_connect 0x7ff8f403e540 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff91413e4b0 con 0x7ff91413e0a0 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient: get_auth_request con 0x7ff91413e0a0 auth_method 0 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_done global_id 94227 payload 824 2024-09-17T16:44:03.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff914139ae0 con 0x7ff91413e0a0 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 client.74767.objecter ms_handle_connect 0x7ff91413e0a0 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 client.74767.objecter resend_mon_ops 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff9040046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 msgr2=0x7ff9140a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 msgr2=0x7ff9140a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff9040046d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff904011900 tx=0x7ff9040046d0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 msgr2=0x7ff9140a0a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 0x7ff8f4049140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 msgr2=0x7ff9140a0a20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff91413e0a0 0x7ff9140a0a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7ff8f40467c0 con 0x7ff8f4046d50 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 0x7ff8f4049140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient: get_auth_request con 0x7ff8f4046d50 auth_method 0 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:03.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 10 monclient(hunting): handle_auth_done global_id 94236 payload 824 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 -- 172.21.15.105:0/133063309 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff904004300 con 0x7ff8f4046d50 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91259c640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 0x7ff8f4049140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff904000c10 tx=0x7ff904009190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 client.74767.objecter ms_handle_connect 0x7ff8f4046d50 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 client.74767.objecter resend_mon_ops 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff904038030 con 0x7ff8f4046d50 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient: dump: 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:03.899 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:03.900 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7ff9040147c0 con 0x7ff8f4046d50 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 msgr2=0x7ff8f4049140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 0x7ff8f4049140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff904000c10 tx=0x7ff904009190 comp rx=0 tx=0).stop 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 msgr2=0x7ff8f4049140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff902ffd640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7ff8f4046d50 0x7ff8f4049140 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff904000c10 tx=0x7ff904009190 comp rx=0 tx=0).stop 2024-09-17T16:44:03.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff912d9d640 1 -- 172.21.15.105:0/133063309 reap_dead start 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 19711, 2024-09-17T16:44:03.902 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:03.903 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.904 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.905 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:03.906 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:03.907 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:03.908 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.909 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.910 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:03.911 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.912 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.914 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.915 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.916 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:03.917 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.918 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:03.919 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.920 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.921 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.924 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.926 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.927 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:03.928 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:03.929 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 librados: watch_flush enter 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 librados: watch_flush exit 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 20 client.74767.objecter shutdown clearing up homeless session... 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 client.74767.objecter successfully canceled tick 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 msgr2=0x7ff8f4040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 0x7ff8f4040a00 secure :-1 s=READY pgs=3642 cs=0 l=1 rev1=1 crypto rx=0x7ff914074620 tx=0x7ff90c00b040 comp rx=0 tx=0).stop 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 10 monclient: shutdown 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 msgr2=0x7ff9141d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ae040 tx=0x7ff9080191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 shutdown_connections 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff8f403e540 0x7ff8f4040a00 unknown :-1 s=CLOSED pgs=3642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 --2- 172.21.15.105:0/133063309 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7ff9140a68e0 0x7ff9141d1a30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 >> 172.21.15.105:0/133063309 conn(0x7ff914068ba0 msgr2=0x7ff9140a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 shutdown_connections 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 -- 172.21.15.105:0/133063309 wait complete. 2024-09-17T16:44:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:03.867+0000 7ff91889a640 1 librados: shutdown 2024-09-17T16:44:03.932 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:03.932 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:44:03.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:44:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: monmap: 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.053152+0000 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.053152+0000 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.060 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 Processor -- start 2024-09-17T16:44:04.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 -- start start 2024-09-17T16:44:04.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: init 2024-09-17T16:44:04.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient: _add_conns ranks=[1,3,7,6,9,10,0,8,4,5,2] 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0180069220 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 0x7f018013e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-g con 0x7f01800a1ed0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f0180141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-i con 0x7f018013eea0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01800a25a0 con 0x7f0180069220 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0180069600 con 0x7f01800a1ed0 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f01800a37f0 con 0x7f018013eea0 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.051+0000 7f0186a51640 10 monclient(hunting): authenticate will time out at 1927.319803s 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017ffff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0180069220 0x7f01800a17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:58860/0 (socket says 172.21.15.105:58860) 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f0180141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017ffff640 1 -- 172.21.15.105:0/296239599 learned_addr learned my addr 172.21.15.105:0/296239599 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 0x7f018013e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): get_auth_request con 0x7f01800a1ed0 auth_method 0 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): get_auth_request con 0x7f018013eea0 auth_method 0 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0184fee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient(hunting): handle_auth_done global_id 54912 payload 274 2024-09-17T16:44:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 -- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 msgr2=0x7f0180141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f0180141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 -- 172.21.15.105:0/296239599 >> v1:172.21.15.160:6789/0 conn(0x7f0180069220 legacy=0x7f01800a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 monclient: found mon.noname-g 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 -- 172.21.15.105:0/296239599 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018009f9f0 con 0x7f01800a1ed0 2024-09-17T16:44:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017f7fe640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 0x7f018013e960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0168009820 tx=0x7f016801bf20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=10764ed5cc8a9e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 1 -- 172.21.15.105:0/296239599 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01680ab070 con 0x7f01800a1ed0 2024-09-17T16:44:04.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: dump: 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.076 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 1 -- 172.21.15.105:0/296239599 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01680a5900 con 0x7f01800a1ed0 2024-09-17T16:44:04.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 5 monclient: authenticate success, global_id 54912 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 4 set_mon_vals no callback set 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 1 -- 172.21.15.105:0/296239599 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01680a5b30 con 0x7f01800a1ed0 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f017effd640 10 monclient: dump: 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.078 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.079 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 10 monclient: shutdown 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 -- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 msgr2=0x7f018013e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 0x7f018013e960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0168009820 tx=0x7f016801bf20 comp rx=0 tx=0).stop 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 -- 172.21.15.105:0/296239599 shutdown_connections 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f0180141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 --2- 172.21.15.105:0/296239599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01800a1ed0 0x7f018013e960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 -- 172.21.15.105:0/296239599 >> 172.21.15.105:0/296239599 conn(0x7f01800a8620 msgr2=0x7f01800a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 -- 172.21.15.105:0/296239599 shutdown_connections 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.055+0000 7f0186a51640 1 -- 172.21.15.105:0/296239599 wait complete. 2024-09-17T16:44:04.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: monmap: 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.081 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 librados: starting msgr at 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 librados: starting objecter 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 Processor -- start 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- start start 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 librados: setting wanted keys 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 librados: calling monclient init 2024-09-17T16:44:04.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: init 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _add_conns ranks=[3,6,7,8,4,1,5,0,2] 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0180069220 0x7f01801d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-g con 0x7f0180069220 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 0x7f01801d2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-d con 0x7f01800a1ed0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f018013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): picked mon.noname-i con 0x7f018013eea0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0180144ba0 con 0x7f0180069220 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f0180144a20 con 0x7f01800a1ed0 2024-09-17T16:44:04.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0180144d20 con 0x7f018013eea0 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): authenticate will time out at 1927.325376s 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 0x7f01801d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 0x7f01801d2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:39312/0 (socket says 172.21.15.105:39312) 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 -- 172.21.15.105:0/2347777784 learned_addr learned my addr 172.21.15.105:0/2347777784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): get_auth_request con 0x7f01800a1ed0 auth_method 0 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f018013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0180069220 0x7f01801d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 10 monclient(hunting): get_auth_request con 0x7f018013eea0 auth_method 0 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): get_auth_request con 0x7f0180069220 auth_method 0 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient(hunting): handle_auth_done global_id 94245 payload 824 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 msgr2=0x7f018013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f018013cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0180069220 msgr2=0x7f01801d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0180069220 0x7f01801d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 monclient: found mon.noname-d 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018012edc0 con 0x7f01800a1ed0 2024-09-17T16:44:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0184fee640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f018013eea0 0x7f018013cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017f7fe640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 0x7f01801d2430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f016801bd50 tx=0x7f0168019010 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 client.?.objecter ms_handle_connect 0x7f01800a1ed0 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01680ab070 con 0x7f01800a1ed0 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: dump: 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.087 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01680044b0 con 0x7f01800a1ed0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 5 monclient: authenticate success, global_id 94245 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _renew_subs 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01800a7360 con 0x7f01800a1ed0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01680a5de0 con 0x7f01800a1ed0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 client.94245.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017e7fc640 4 set_mon_vals no callback set 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 monclient: dump: 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _renew_subs 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f018013e3a0 con 0x7f01800a1ed0 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 librados: init done 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 0x7f018013d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0180051e80 con 0x7f018013d0b0 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 0x7f018013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient: get_auth_request con 0x7f018013d0b0 auth_method 0 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f01680ae070 con 0x7f01800a1ed0 2024-09-17T16:44:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 10 monclient(hunting): handle_auth_done global_id 54962 payload 824 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01800008e0 con 0x7f018013d0b0 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017ffff640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 0x7f018013d490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f017000c4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f016003e580 0x7f0160040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 client.94245.objecter ms_handle_connect 0x7f018013d0b0 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 client.94245.objecter resend_mon_ops 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(643..643 src has 1..643) ==== 6388+0+0 (secure 0 0 0) 0x7f01680e91b0 con 0x7f01800a1ed0 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 10 client.94245.objecter ms_dispatch 0x7f0180000ba0 osd_map(643..643 src has 1..643) 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 3 client.94245.objecter handle_osd_map got epochs [643,643] > 0 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.059+0000 7f017cff9640 3 client.94245.objecter handle_osd_map decoding full epoch 643 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 20 client.94245.objecter dump_active .. 0 homeless 2024-09-17T16:44:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0170005df0 con 0x7f018013d0b0 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 monclient: dump: 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f0184fee640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f016003e580 0x7f0160040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f0184fee640 10 monclient: get_auth_request con 0x7f016003e580 auth_method 0 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0170018070 con 0x7f018013d0b0 2024-09-17T16:44:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 msgr2=0x7f018013d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 0x7f018013d490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f017000c4b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 msgr2=0x7f018013d490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d0b0 0x7f018013d490 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f017000c4b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f0184fee640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f016003e580 0x7f0160040a40 secure :-1 s=READY pgs=3643 cs=0 l=1 rev1=1 crypto rx=0x7f018006db20 tx=0x7f0178008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.063+0000 7f017cff9640 10 client.94245.objecter ms_handle_connect 0x7f016003e580 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 0x7f018009c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f018006a1c0 con 0x7f018013d9d0 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 0x7f018009c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient: get_auth_request con 0x7f018013d9d0 auth_method 0 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 10 monclient(hunting): handle_auth_done global_id 54971 payload 824 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 1 -- 172.21.15.105:0/2347777784 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f01800a17c0 con 0x7f018013d9d0 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 client.94245.objecter ms_handle_connect 0x7f018013d9d0 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017ffff640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 0x7f018009c350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f0170001260 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 client.94245.objecter resend_mon_ops 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f017001dc80 con 0x7f018013d9d0 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 monclient: dump: 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9457 (secure 0 0 0) 0x7f01700070d0 con 0x7f018013d9d0 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 msgr2=0x7f018009c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 0x7f018009c350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f0170001260 comp rx=0 tx=0).stop 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 msgr2=0x7f018009c350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f017cff9640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f018013d9d0 0x7f018009c350 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0180074f90 tx=0x7f0170001260 comp rx=0 tx=0).stop 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:04.101 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 551064, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.102 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.103 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.105 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:04.106 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.108 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.109 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.110 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:04.112 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:04.116 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.117 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.121 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.122 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.123 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.125 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.126 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.127 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:04.128 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 10 librados: watch_flush enter 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 10 librados: watch_flush exit 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 20 client.94245.objecter shutdown clearing up homeless session... 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.067+0000 7f0186a51640 10 client.94245.objecter successfully canceled tick 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f016003e580 msgr2=0x7f0160040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f016003e580 0x7f0160040a40 secure :-1 s=READY pgs=3643 cs=0 l=1 rev1=1 crypto rx=0x7f018006db20 tx=0x7f0178008040 comp rx=0 tx=0).stop 2024-09-17T16:44:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 10 monclient: shutdown 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 msgr2=0x7f01801d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 --2- 172.21.15.105:0/2347777784 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f01800a1ed0 0x7f01801d2430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f016801bd50 tx=0x7f0168019010 comp rx=0 tx=0).stop 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0184fee640 1 -- 172.21.15.105:0/2347777784 reap_dead start 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0184fee640 1 -- 172.21.15.105:0/2347777784 reap_dead start 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 shutdown_connections 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 >> 172.21.15.105:0/2347777784 conn(0x7f01800a8620 msgr2=0x7f018013f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 shutdown_connections 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 -- 172.21.15.105:0/2347777784 wait complete. 2024-09-17T16:44:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.071+0000 7f0186a51640 1 librados: shutdown 2024-09-17T16:44:04.132 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:04.132 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:44:04.132 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: monmap: 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.247969+0000 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.247969+0000 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.253 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 Processor -- start 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 -- start start 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: init 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient: _add_conns ranks=[2,3,0,8,5,10,6,1,7,9,4] 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-b con 0x7fbb780a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb780a4ab0 0x7fbb780a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-g con 0x7fbb780a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb781397f0 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb780a4080 con 0x7fbb781397f0 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780a7c40 con 0x7fbb780a6a30 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780a7890 con 0x7fbb780a4ab0 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7ed82640 10 monclient(hunting): authenticate will time out at 1927.513621s 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47214/0 (socket says 172.21.15.105:47214) 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1093354557 learned_addr learned my addr 172.21.15.105:0/1093354557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request con 0x7fbb780a6a30 auth_method 0 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_done global_id 84449 payload 274 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1093354557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb780a4ab0 msgr2=0x7fbb780a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb77fff640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb780a4ab0 0x7fbb780a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb777fe640 1 -- 172.21.15.105:0/1093354557 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545766121 0 0) 0x7fbb780a4080 con 0x7fbb781397f0 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb780a4ab0 0x7fbb780a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1093354557 >> v1:172.21.15.105:6789/0 conn(0x7fbb781397f0 legacy=0x7fbb78139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 monclient: found mon.noname-b 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1093354557 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb780a37e0 con 0x7fbb780a6a30 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.247+0000 7fbb777fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb600098a0 tx=0x7fbb6001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18dca91c1a8bc0e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 1 -- 172.21.15.105:0/1093354557 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb600ab8b0 con 0x7fbb780a6a30 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: dump: 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 1 -- 172.21.15.105:0/1093354557 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb600b0070 con 0x7fbb780a6a30 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 5 monclient: authenticate success, global_id 84449 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 4 set_mon_vals no callback set 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 1 -- 172.21.15.105:0/1093354557 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb600aa3a0 con 0x7fbb780a6a30 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb777fe640 10 monclient: dump: 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: shutdown 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1093354557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 msgr2=0x7fbb780a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb600098a0 tx=0x7fbb6001c4a0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1093354557 shutdown_connections 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb780a4ab0 0x7fbb780a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1093354557 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb780a6a30 0x7fbb780a6e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1093354557 >> 172.21.15.105:0/1093354557 conn(0x7fbb780a8d00 msgr2=0x7fbb780a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1093354557 shutdown_connections 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1093354557 wait complete. 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: monmap: 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 librados: starting msgr at 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 librados: starting objecter 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 Processor -- start 2024-09-17T16:44:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- start start 2024-09-17T16:44:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 librados: setting wanted keys 2024-09-17T16:44:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 librados: calling monclient init 2024-09-17T16:44:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: init 2024-09-17T16:44:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient: _add_conns ranks=[1,7,5,2,8,3,6,4,0] 2024-09-17T16:44:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb780a4ab0 0x7fbb781d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-f con 0x7fbb780a4ab0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 0x7fbb781d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-i con 0x7fbb780a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 0x7fbb781d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): picked mon.noname-h con 0x7fbb781397f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78144270 con 0x7fbb780a4ab0 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fbb7806e770 con 0x7fbb781397f0 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fbb781443f0 con 0x7fbb780a6a30 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7ed82640 10 monclient(hunting): authenticate will time out at 1927.518701s 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 0x7fbb781d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 0x7fbb781d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb780a4ab0 0x7fbb781d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 0x7fbb781d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:41156/0 (socket says 172.21.15.105:41156) 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 1 -- 172.21.15.105:0/1582520029 learned_addr learned my addr 172.21.15.105:0/1582520029 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): get_auth_request con 0x7fbb780a6a30 auth_method 0 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 0x7fbb781d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3302/0 says I am v2:172.21.15.105:59146/0 (socket says 172.21.15.105:59146) 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): get_auth_request con 0x7fbb781397f0 auth_method 0 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request con 0x7fbb780a4ab0 auth_method 0 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient(hunting): handle_auth_done global_id 74417 payload 824 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 msgr2=0x7fbb781d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 0x7fbb781d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb780a4ab0 msgr2=0x7fbb781d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb780a4ab0 0x7fbb781d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 monclient: found mon.noname-h 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb7d31f640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb780abcf0 con 0x7fbb781397f0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.251+0000 7fbb77fff640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fbb780a6a30 0x7fbb781d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7d31f640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 0x7fbb781d1d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c730 tx=0x7fbb7000cbf0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.?.objecter ms_handle_connect 0x7fbb781397f0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb70003df0 con 0x7fbb781397f0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: dump: 2024-09-17T16:44:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 5 monclient: authenticate success, global_id 74417 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70005070 con 0x7fbb781397f0 2024-09-17T16:44:04.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 monclient: _renew_subs 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb78122a00 con 0x7fbb781397f0 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb70005220 con 0x7fbb781397f0 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb76ffd640 4 set_mon_vals no callback set 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 client.74417.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: dump: 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.281 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 monclient: _renew_subs 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb78122720 con 0x7fbb781397f0 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 1 librados: init done 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 0x7fbb7813abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78140820 con 0x7fbb7813a800 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 0x7fbb7813abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient: get_auth_request con 0x7fbb7813a800 auth_method 0 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fbb70003980 con 0x7fbb781397f0 2024-09-17T16:44:04.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb5803e550 0x7fbb58040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7fbb7004fd30 con 0x7fbb781397f0 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.74417.objecter ms_dispatch 0x7fbb78000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 3 client.74417.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 3 client.74417.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_done global_id 84394 payload 824 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb781395e0 con 0x7fbb7813a800 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 20 client.74417.objecter dump_active .. 0 homeless 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb77fff640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb5803e550 0x7fbb58040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb77fff640 10 monclient: get_auth_request con 0x7fbb5803e550 auth_method 0 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.74417.objecter ms_handle_connect 0x7fbb7813a800 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.74417.objecter resend_mon_ops 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 0x7fbb7813abe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb7806eab0 tx=0x7fbb60019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb60003a90 con 0x7fbb7813a800 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: dump: 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.285 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 client.74417.objecter ms_handle_connect 0x7fbb5803e550 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb77fff640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb5803e550 0x7fbb58040a10 secure :-1 s=READY pgs=3644 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbb600b0040 con 0x7fbb7813a800 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 msgr2=0x7fbb7813abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 0x7fbb7813abe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb7806eab0 tx=0x7fbb60019240 comp rx=0 tx=0).stop 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 msgr2=0x7fbb7813abe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.255+0000 7fbb757fa640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813a800 0x7fbb7813abe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7ed82640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 0x7fbb7813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7ed82640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb78051e80 con 0x7fbb7813b120 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 0x7fbb7813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient: get_auth_request con 0x7fbb7813b120 auth_method 0 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 10 monclient(hunting): handle_auth_done global_id 84403 payload 824 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 1 -- 172.21.15.105:0/1582520029 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbb780ab780 con 0x7fbb7813b120 2024-09-17T16:44:04.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb7cb1e640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 0x7fbb7813e220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb600090b0 tx=0x7fbb600a9000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 10 client.74417.objecter ms_handle_connect 0x7fbb7813b120 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 10 client.74417.objecter resend_mon_ops 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb600ca070 con 0x7fbb7813b120 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.263+0000 7fbb757fa640 10 monclient: dump: 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.288 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7fbb600ab950 con 0x7fbb7813b120 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 msgr2=0x7fbb7813e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 0x7fbb7813e220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb600090b0 tx=0x7fbb600a9000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 msgr2=0x7fbb7813e220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb757fa640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb7813b120 0x7fbb7813e220 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb600090b0 tx=0x7fbb600a9000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 151493, 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.291 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.292 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:04.294 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:04.295 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.296 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.297 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.298 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:04.302 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:04.304 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.305 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.306 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.309 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:04.311 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 10 librados: watch_flush enter 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 10 librados: watch_flush exit 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 20 client.74417.objecter shutdown clearing up homeless session... 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 10 client.74417.objecter successfully canceled tick 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb5803e550 msgr2=0x7fbb58040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fbb5803e550 0x7fbb58040a10 secure :-1 s=READY pgs=3644 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68008040 comp rx=0 tx=0).stop 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 10 monclient: shutdown 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 msgr2=0x7fbb781d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 --2- 172.21.15.105:0/1582520029 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fbb781397f0 0x7fbb781d1d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c730 tx=0x7fbb7000cbf0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7d31f640 1 -- 172.21.15.105:0/1582520029 reap_dead start 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7d31f640 1 -- 172.21.15.105:0/1582520029 reap_dead start 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 shutdown_connections 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 >> 172.21.15.105:0/1582520029 conn(0x7fbb780a8d00 msgr2=0x7fbb780a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 shutdown_connections 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 -- 172.21.15.105:0/1582520029 wait complete. 2024-09-17T16:44:04.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.267+0000 7fbb7ed82640 1 librados: shutdown 2024-09-17T16:44:04.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:04.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:44:04.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: monmap: 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.428115+0000 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.428115+0000 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.434 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 Processor -- start 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 -- start start 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: init 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient: _add_conns ranks=[10,2,6,3,0,7,1,5,8,4,9] 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f70dc0a6a30 0x7f70dc0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-f con 0x7f70dc0a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-b con 0x7f70dc0a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc1397f0 0x7f70dc139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-d con 0x7f70dc1397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc0a7c40 con 0x7f70dc0a6a30 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0a7890 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f70dc0a4080 con 0x7f70dc1397f0 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e2311640 10 monclient(hunting): authenticate will time out at 1927.693865s 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47222/0 (socket says 172.21.15.105:47222) 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 -- 172.21.15.105:0/2192594093 learned_addr learned my addr 172.21.15.105:0/2192594093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): get_auth_request con 0x7f70dc0a4ab0 auth_method 0 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc1397f0 0x7f70dc139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): get_auth_request con 0x7f70dc1397f0 auth_method 0 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70dbfff640 1 --2- 172.21.15.105:0/2192594093 >> v2:172.21.15.160:3300/0 conn(0x7f70dc0a6a30 0x7f70dc0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70dbfff640 10 monclient(hunting): get_auth_request con 0x7f70dc0a6a30 auth_method 0 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70e08ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient(hunting): handle_auth_done global_id 84458 payload 274 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 -- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc1397f0 msgr2=0x7f70dc139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc1397f0 0x7f70dc139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 -- 172.21.15.105:0/2192594093 >> v2:172.21.15.160:3300/0 conn(0x7f70dc0a6a30 msgr2=0x7f70dc0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 --2- 172.21.15.105:0/2192594093 >> v2:172.21.15.160:3300/0 conn(0x7f70dc0a6a30 0x7f70dc0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 monclient: found mon.noname-b 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.427+0000 7f70db7fe640 1 -- 172.21.15.105:0/2192594093 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0a37e0 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098a0 tx=0x7f70c401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9e391446f449e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 1 -- 172.21.15.105:0/2192594093 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70c40b0070 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:04.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: dump: 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 1 -- 172.21.15.105:0/2192594093 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c40aa810 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 5 monclient: authenticate success, global_id 84458 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 1 -- 172.21.15.105:0/2192594093 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70c40a9300 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 4 set_mon_vals no callback set 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70daffd640 10 monclient: dump: 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.449 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: shutdown 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 msgr2=0x7f70dc0a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098a0 tx=0x7f70c401c3d0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/2192594093 shutdown_connections 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc1397f0 0x7f70dc139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- 172.21.15.105:0/2192594093 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f70dc0a4ab0 0x7f70dc0a7350 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- 172.21.15.105:0/2192594093 >> v2:172.21.15.160:3300/0 conn(0x7f70dc0a6a30 0x7f70dc0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/2192594093 >> 172.21.15.105:0/2192594093 conn(0x7f70dc0a8d00 msgr2=0x7f70dc0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/2192594093 shutdown_connections 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/2192594093 wait complete. 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: monmap: 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:04.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 librados: starting msgr at 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 librados: starting objecter 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 Processor -- start 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- start start 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 librados: setting wanted keys 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 librados: calling monclient init 2024-09-17T16:44:04.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: init 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _add_conns ranks=[8,6,7,5,2,0,4,1,3] 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 0x7f70dc137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-e con 0x7f70dc0a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc0a6a30 0x7f70dc1384b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-d con 0x7f70dc0a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f70dc1397f0 0x7f70dc13d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): picked mon.noname-i con 0x7f70dc1397f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f70dc06e770 con 0x7f70dc0a6a30 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f70dc0a2fc0 con 0x7f70dc1397f0 2024-09-17T16:44:04.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f70dc143260 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient(hunting): authenticate will time out at 1927.698938s 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc0a6a30 0x7f70dc1384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 0x7f70dc137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 0x7f70dc137f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33684/0 (socket says 172.21.15.105:33684) 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 -- 172.21.15.105:0/1086019287 learned_addr learned my addr 172.21.15.105:0/1086019287 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): get_auth_request con 0x7f70dc0a4ab0 auth_method 0 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): get_auth_request con 0x7f70dc0a6a30 auth_method 0 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e08ae640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f70dc1397f0 0x7f70dc13d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient(hunting): handle_auth_done global_id 54980 payload 824 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f70dc1397f0 msgr2=0x7f70dc13d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f70dc1397f0 0x7f70dc13d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc0a6a30 msgr2=0x7f70dc1384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f70dc0a6a30 0x7f70dc1384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 monclient: found mon.noname-e 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0aa100 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e08ae640 10 monclient: get_auth_request con 0x7f70dc1397f0 auth_method 0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e08ae640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f70dc1397f0 0x7f70dc13d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70dbfff640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 0x7f70dc137f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007ef0 tx=0x7f70d400e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 client.?.objecter ms_handle_connect 0x7f70dc0a4ab0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70d4014890 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: dump: 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4019070 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 5 monclient: authenticate success, global_id 54980 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _renew_subs 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc0abcb0 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70d4013380 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70da7fc640 4 set_mon_vals no callback set 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70d8ff9640 10 monclient: dump: 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 client.54980.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _renew_subs 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc123c30 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.431+0000 7f70e2311640 1 librados: init done 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e2311640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e2311640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 0x7f70dc13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc051e80 con 0x7f70dc13dbd0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f70d4014420 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 0x7f70dc13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient: get_auth_request con 0x7f70dc13dbd0 auth_method 0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70c003e580 0x7f70c0040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7f70d40508c0 con 0x7f70dc0a4ab0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 client.54980.objecter ms_dispatch 0x7f70dc000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 3 client.54980.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 3 client.54980.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 20 client.54980.objecter dump_active .. 0 homeless 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e08ae640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70c003e580 0x7f70c0040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e08ae640 10 monclient: get_auth_request con 0x7f70c003e580 auth_method 0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 10 monclient(hunting): handle_auth_done global_id 54930 payload 824 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70dc13fb80 con 0x7f70dc13dbd0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 client.54980.objecter ms_handle_connect 0x7f70dc13dbd0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 client.54980.objecter resend_mon_ops 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70db7fe640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 0x7f70dc13dfb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0a6570 tx=0x7f70c4019170 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 client.54980.objecter ms_handle_connect 0x7f70c003e580 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70e08ae640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70c003e580 0x7f70c0040a40 secure :-1 s=READY pgs=3645 cs=0 l=1 rev1=1 crypto rx=0x7f70dc1243e0 tx=0x7f70cc008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70c40ba030 con 0x7f70dc13dbd0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 monclient: dump: 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.463 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f70c4003620 con 0x7f70dc13dbd0 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 msgr2=0x7f70dc13dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 0x7f70dc13dfb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0a6570 tx=0x7f70c4019170 comp rx=0 tx=0).stop 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 msgr2=0x7f70dc13dfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.435+0000 7f70d8ff9640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13dbd0 0x7f70dc13dfb0 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0a6570 tx=0x7f70c4019170 comp rx=0 tx=0).stop 2024-09-17T16:44:04.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70e2311640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:04.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70e2311640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 0x7f70dc0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70e2311640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc06a200 con 0x7f70dc13e4f0 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 0x7f70dc0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient: get_auth_request con 0x7f70dc13e4f0 auth_method 0 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 10 monclient(hunting): handle_auth_done global_id 54939 payload 824 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 1 -- 172.21.15.105:0/1086019287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70dc139ca0 con 0x7f70dc13e4f0 2024-09-17T16:44:04.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70db7fe640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 0x7f70dc0a0a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70c401cc70 tx=0x7f70c40a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 client.54980.objecter ms_handle_connect 0x7f70dc13e4f0 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 client.54980.objecter resend_mon_ops 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70c40ba040 con 0x7f70dc13e4f0 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 monclient: dump: 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.468 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f70c40039e0 con 0x7f70dc13e4f0 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 msgr2=0x7f70dc0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 0x7f70dc0a0a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70c401cc70 tx=0x7f70c40a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 msgr2=0x7f70dc0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.443+0000 7f70d8ff9640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70dc13e4f0 0x7f70dc0a0a20 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70c401cc70 tx=0x7f70c40a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 551449, 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:04.472 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:04.473 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:04.474 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:04.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:04.476 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.478 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:04.479 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.480 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:04.481 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:04.483 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.484 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.485 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:04.486 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.487 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.488 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.489 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:04.490 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.491 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.492 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.493 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.493 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:04.493 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 10 librados: watch_flush enter 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 10 librados: watch_flush exit 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 20 client.54980.objecter shutdown clearing up homeless session... 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 10 client.54980.objecter successfully canceled tick 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70c003e580 msgr2=0x7f70c0040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f70c003e580 0x7f70c0040a40 secure :-1 s=READY pgs=3645 cs=0 l=1 rev1=1 crypto rx=0x7f70dc1243e0 tx=0x7f70cc008040 comp rx=0 tx=0).stop 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 10 monclient: shutdown 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 msgr2=0x7f70dc137f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 --2- 172.21.15.105:0/1086019287 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f70dc0a4ab0 0x7f70dc137f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70d4007ef0 tx=0x7f70d400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e08ae640 1 -- 172.21.15.105:0/1086019287 reap_dead start 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e08ae640 1 -- 172.21.15.105:0/1086019287 reap_dead start 2024-09-17T16:44:04.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 shutdown_connections 2024-09-17T16:44:04.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 >> 172.21.15.105:0/1086019287 conn(0x7f70dc0a8d00 msgr2=0x7f70dc0a4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 shutdown_connections 2024-09-17T16:44:04.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 -- 172.21.15.105:0/1086019287 wait complete. 2024-09-17T16:44:04.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.447+0000 7f70e2311640 1 librados: shutdown 2024-09-17T16:44:04.496 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:04.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:44:04.496 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:44:04.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: monmap: 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.608535+0000 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.608535+0000 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.613 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.614 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 Processor -- start 2024-09-17T16:44:04.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 -- start start 2024-09-17T16:44:04.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: init 2024-09-17T16:44:04.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient: _add_conns ranks=[4,5,8,7,10,0,1,3,2,9,6] 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-c con 0x7f4e5809b3a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5809bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-h con 0x7f4e5809b780 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-e con 0x7f4e580a2de0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f4e580a0160 con 0x7f4e5809b3a0 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4e5809f8e0 con 0x7f4e5809b780 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4e58074a00 con 0x7f4e580a2de0 2024-09-17T16:44:04.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e5d53e640 10 monclient(hunting): authenticate will time out at 1927.874888s 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34190/0 (socket says 172.21.15.105:34190) 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33690/0 (socket says 172.21.15.105:33690) 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 -- 172.21.15.105:0/2558204670 learned_addr learned my addr 172.21.15.105:0/2558204670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request con 0x7f4e5809b3a0 auth_method 0 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): get_auth_request con 0x7f4e580a2de0 auth_method 0 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e4e7fc640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5809bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e4e7fc640 10 monclient(hunting): get_auth_request con 0x7f4e5809b780 auth_method 0 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e4e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e4e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e4e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_done global_id 74776 payload 274 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 -- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 msgr2=0x7f4e580a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 -- 172.21.15.105:0/2558204670 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 msgr2=0x7f4e5809bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5809bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 monclient: found mon.noname-c 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e56ffd640 1 -- 172.21.15.105:0/2558204670 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58074c80 con 0x7f4e5809b3a0 2024-09-17T16:44:04.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.607+0000 7f4e577fe640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098a0 tx=0x7f4e4801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d259f66f571fb2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 1 -- 172.21.15.105:0/2558204670 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e480b0070 con 0x7f4e5809b3a0 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: dump: 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 1 -- 172.21.15.105:0/2558204670 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e480aa810 con 0x7f4e5809b3a0 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 1 -- 172.21.15.105:0/2558204670 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e480a9300 con 0x7f4e5809b3a0 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e567fc640 10 monclient: dump: 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 5 monclient: authenticate success, global_id 74776 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 4 set_mon_vals no callback set 2024-09-17T16:44:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 10 monclient: shutdown 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 msgr2=0x7f4e580a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e480098a0 tx=0x7f4e4801c440 comp rx=0 tx=0).stop 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- 172.21.15.105:0/2558204670 shutdown_connections 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5809bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4e580a2de0 0x7f4e580a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/2558204670 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4e5809b3a0 0x7f4e580a05f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- 172.21.15.105:0/2558204670 >> 172.21.15.105:0/2558204670 conn(0x7f4e580a83e0 msgr2=0x7f4e580a87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- 172.21.15.105:0/2558204670 shutdown_connections 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- 172.21.15.105:0/2558204670 wait complete. 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 10 monclient: monmap: 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.632 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 librados: starting msgr at 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 librados: starting objecter 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 Processor -- start 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 -- start start 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 librados: setting wanted keys 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 1 librados: calling monclient init 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.611+0000 7f4e5d53e640 10 monclient: init 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _add_conns ranks=[7,5,2,8,4,6,3,0,1] 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4e5809b3a0 0x7f4e5813b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-i con 0x7f4e5809b3a0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5813bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-h con 0x7f4e5809b780 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 0x7f4e58090de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): picked mon.noname-b con 0x7f4e580a2de0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4e581433f0 con 0x7f4e580a2de0 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4e58143270 con 0x7f4e5809b780 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f4e58143570 con 0x7f4e5809b3a0 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): authenticate will time out at 1927.880152s 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 0x7f4e58090de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 0x7f4e58090de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47230/0 (socket says 172.21.15.105:47230) 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 learned_addr learned my addr 172.21.15.105:0/88990342 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): get_auth_request con 0x7f4e580a2de0 auth_method 0 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4e5809b3a0 0x7f4e5813b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5813bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request con 0x7f4e5809b3a0 auth_method 0 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient(hunting): handle_auth_done global_id 84467 payload 824 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4e5809b3a0 msgr2=0x7f4e5813b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4e5809b3a0 0x7f4e5813b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 msgr2=0x7f4e5813bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5813bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 monclient: found mon.noname-b 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5813c6d0 con 0x7f4e580a2de0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 10 monclient: get_auth_request con 0x7f4e5809b780 auth_method 0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e5809b780 0x7f4e5813bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e577fe640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 0x7f4e58090de0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000ef90 tx=0x7f4e5000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.?.objecter ms_handle_connect 0x7f4e580a2de0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e50018020 con 0x7f4e580a2de0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: dump: 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5000cd90 con 0x7f4e580a2de0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e50014d60 con 0x7f4e580a2de0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 5 monclient: authenticate success, global_id 84467 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _renew_subs 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e58094aa0 con 0x7f4e580a2de0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e55ffb640 4 set_mon_vals no callback set 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: dump: 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 client.84467.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _renew_subs 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e580944e0 con 0x7f4e580a2de0 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 librados: init done 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 0x7f4e58091700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4e58051e80 con 0x7f4e58091320 2024-09-17T16:44:04.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4e5001b020 con 0x7f4e580a2de0 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4e3803e550 0x7f4e38040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7f4e50050e60 con 0x7f4e580a2de0 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.84467.objecter ms_dispatch 0x7f4e58000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 3 client.84467.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 3 client.84467.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 0x7f4e58091700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient: get_auth_request con 0x7f4e58091320 auth_method 0 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 20 client.84467.objecter dump_active .. 0 homeless 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4e3803e550 0x7f4e38040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 10 monclient: get_auth_request con 0x7f4e3803e550 auth_method 0 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_done global_id 74435 payload 824 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e5809bb60 con 0x7f4e58091320 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.84467.objecter ms_handle_connect 0x7f4e58091320 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.84467.objecter resend_mon_ops 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 0x7f4e58091700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009870 tx=0x7f4e480191e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 client.84467.objecter ms_handle_connect 0x7f4e3803e550 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4e7fc640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4e3803e550 0x7f4e38040a10 secure :-1 s=READY pgs=3646 cs=0 l=1 rev1=1 crypto rx=0x7f4e5809d3a0 tx=0x7f4e40008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e480b9030 con 0x7f4e58091320 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.615+0000 7f4e4ffff640 10 monclient: dump: 2024-09-17T16:44:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.643 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4e480b1040 con 0x7f4e58091320 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 msgr2=0x7f4e58091700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 0x7f4e58091700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009870 tx=0x7f4e480191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 msgr2=0x7f4e58091700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.619+0000 7f4e4ffff640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091320 0x7f4e58091700 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009870 tx=0x7f4e480191e0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e5d53e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 0x7f4e5813e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e5d53e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4e5806a1c0 con 0x7f4e58091c40 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 0x7f4e5813e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient: get_auth_request con 0x7f4e58091c40 auth_method 0 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 10 monclient(hunting): handle_auth_done global_id 74444 payload 824 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 1 -- 172.21.15.105:0/88990342 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e580a8ac0 con 0x7f4e58091c40 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e56ffd640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 0x7f4e5813e8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e4801c270 tx=0x7f4e480a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 10 client.84467.objecter ms_handle_connect 0x7f4e58091c40 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 10 client.84467.objecter resend_mon_ops 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e480b9040 con 0x7f4e58091c40 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.623+0000 7f4e4ffff640 10 monclient: dump: 2024-09-17T16:44:04.647 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.648 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f4e48003830 con 0x7f4e58091c40 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 msgr2=0x7f4e5813e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 0x7f4e5813e8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e4801c270 tx=0x7f4e480a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 msgr2=0x7f4e5813e8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e4ffff640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4e58091c40 0x7f4e5813e8f0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e4801c270 tx=0x7f4e480a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.649 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 284096, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:04.650 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.651 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.652 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.654 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.656 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.657 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.658 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:04.660 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:04.662 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.663 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.664 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.666 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.667 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.668 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.669 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.670 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:04.671 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.672 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:04.673 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 10 librados: watch_flush enter 2024-09-17T16:44:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 10 librados: watch_flush exit 2024-09-17T16:44:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 20 client.84467.objecter shutdown clearing up homeless session... 2024-09-17T16:44:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 10 client.84467.objecter successfully canceled tick 2024-09-17T16:44:04.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4e3803e550 msgr2=0x7f4e38040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4e3803e550 0x7f4e38040a10 secure :-1 s=READY pgs=3646 cs=0 l=1 rev1=1 crypto rx=0x7f4e5809d3a0 tx=0x7f4e40008040 comp rx=0 tx=0).stop 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 10 monclient: shutdown 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 msgr2=0x7f4e58090de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 --2- 172.21.15.105:0/88990342 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4e580a2de0 0x7f4e58090de0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000ef90 tx=0x7f4e5000c490 comp rx=0 tx=0).stop 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 reap_dead start 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e577fe640 1 -- 172.21.15.105:0/88990342 reap_dead start 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 shutdown_connections 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 >> 172.21.15.105:0/88990342 conn(0x7f4e580a83e0 msgr2=0x7f4e5809d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 shutdown_connections 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 -- 172.21.15.105:0/88990342 wait complete. 2024-09-17T16:44:04.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.627+0000 7f4e5d53e640 1 librados: shutdown 2024-09-17T16:44:04.676 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:04.676 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:44:04.676 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: monmap: 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.788572+0000 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.788572+0000 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.793 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 Processor -- start 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 -- start start 2024-09-17T16:44:04.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: init 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient: _add_conns ranks=[8,7,3,6,5,0,4,10,2,1,9] 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-e con 0x7ffb5c0a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-i con 0x7ffb5c0a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb5c1397f0 0x7ffb5c139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-g con 0x7ffb5c1397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:04.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5c0a7c40 con 0x7ffb5c1397f0 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5c0a7890 con 0x7ffb5c0a4ab0 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ffb5c0a4080 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb6449c640 10 monclient(hunting): authenticate will time out at 1928.054239s 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33698/0 (socket says 172.21.15.105:33698) 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 -- 172.21.15.105:0/1399008349 learned_addr learned my addr 172.21.15.105:0/1399008349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): get_auth_request con 0x7ffb5c0a6a30 auth_method 0 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient(hunting): handle_auth_done global_id 54998 payload 274 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62a39640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb5c1397f0 0x7ffb5c139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 -- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 msgr2=0x7ffb5c0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 -- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb5c1397f0 msgr2=0x7ffb5c139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb5c1397f0 0x7ffb5c139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 monclient: found mon.noname-e 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:04.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.787+0000 7ffb62238640 1 -- 172.21.15.105:0/1399008349 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c0a37e0 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb62238640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098a0 tx=0x7ffb5801c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=487f6956a7bd0777 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 1 -- 172.21.15.105:0/1399008349 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb580b1070 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: dump: 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 1 -- 172.21.15.105:0/1399008349 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb580aa7b0 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 1 -- 172.21.15.105:0/1399008349 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb580aae10 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb61236640 10 monclient: dump: 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 5 monclient: authenticate success, global_id 54998 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 4 set_mon_vals no callback set 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: shutdown 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- 172.21.15.105:0/1399008349 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 msgr2=0x7ffb5c0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098a0 tx=0x7ffb5801c410 comp rx=0 tx=0).stop 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- 172.21.15.105:0/1399008349 shutdown_connections 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb5c1397f0 0x7ffb5c139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- 172.21.15.105:0/1399008349 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ffb5c0a6a30 0x7ffb5c0a6e10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- 172.21.15.105:0/1399008349 >> 172.21.15.105:0/1399008349 conn(0x7ffb5c0a8d00 msgr2=0x7ffb5c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- 172.21.15.105:0/1399008349 shutdown_connections 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- 172.21.15.105:0/1399008349 wait complete. 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: monmap: 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.810 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 librados: starting msgr at 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 librados: starting objecter 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 Processor -- start 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- start start 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 librados: setting wanted keys 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 librados: calling monclient init 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: init 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient: _add_conns ranks=[7,5,1,8,2,3,4,0,6] 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-i con 0x7ffb5c0a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb5c0a6a30 0x7ffb5c13d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-h con 0x7ffb5c0a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb5c1397f0 0x7ffb5c13dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): picked mon.noname-f con 0x7ffb5c1397f0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c06e770 con 0x7ffb5c1397f0 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ffb5c13a5a0 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5c143260 con 0x7ffb5c0a4ab0 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb6449c640 10 monclient(hunting): authenticate will time out at 1928.059534s 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb62238640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.791+0000 7ffb62a39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb5c1397f0 0x7ffb5c13dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb5c0a6a30 0x7ffb5c13d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb5c1397f0 0x7ffb5c13dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:36786/0 (socket says 172.21.15.105:36786) 2024-09-17T16:44:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 1 -- 172.21.15.105:0/22439565 learned_addr learned my addr 172.21.15.105:0/22439565 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c138750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:41194/0 (socket says 172.21.15.105:41194) 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): get_auth_request con 0x7ffb5c1397f0 auth_method 0 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): get_auth_request con 0x7ffb5c0a4ab0 auth_method 0 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): get_auth_request con 0x7ffb5c0a6a30 auth_method 0 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient(hunting): handle_auth_done global_id 74453 payload 824 2024-09-17T16:44:04.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 msgr2=0x7ffb5c138750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c0a4ab0 0x7ffb5c138750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb5c1397f0 msgr2=0x7ffb5c13dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb5c1397f0 0x7ffb5c13dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 monclient: found mon.noname-h 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c13faf0 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb61a37640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb5c0a6a30 0x7ffb5c13d860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00b3f0 tx=0x7ffb4c00b8b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 client.?.objecter ms_handle_connect 0x7ffb5c0a6a30 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4c015020 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: dump: 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.816 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 5 monclient: authenticate success, global_id 74453 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 monclient: _renew_subs 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4c011630 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c0aa190 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4c011c50 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb60a35640 4 set_mon_vals no callback set 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 client.74453.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: dump: 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 monclient: _renew_subs 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.160:3302/0 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c123e00 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 1 librados: init done 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 0x7ffb5c02d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5c051e80 con 0x7ffb5c138c90 2024-09-17T16:44:04.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 0x7ffb5c02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient: get_auth_request con 0x7ffb5c138c90 auth_method 0 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.5 v2:172.21.15.160:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ffb4c018020 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb4003e480 0x7ffb40040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 10 monclient(hunting): handle_auth_done global_id 64726 payload 824 2024-09-17T16:44:04.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb5c139c50 con 0x7ffb5c138c90 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.5 v2:172.21.15.160:3302/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7ffb4c04ffc0 con 0x7ffb5c0a6a30 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 client.74453.objecter ms_dispatch 0x7ffb5c000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 3 client.74453.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 3 client.74453.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 20 client.74453.objecter dump_active .. 0 homeless 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 client.74453.objecter ms_handle_connect 0x7ffb5c138c90 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 client.74453.objecter resend_mon_ops 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62238640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 0x7ffb5c02d450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ae000 tx=0x7ffb580191b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb4003e480 0x7ffb40040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb62a39640 10 monclient: get_auth_request con 0x7ffb4003e480 auth_method 0 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb580b1040 con 0x7ffb5c138c90 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:04.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.795+0000 7ffb52ffd640 10 monclient: dump: 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.822 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 10 client.74453.objecter ms_handle_connect 0x7ffb4003e480 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb62a39640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb4003e480 0x7ffb40040940 secure :-1 s=READY pgs=3647 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1396b0 tx=0x7ffb54007650 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ffb58003760 con 0x7ffb5c138c90 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 msgr2=0x7ffb5c02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 0x7ffb5c02d450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ae000 tx=0x7ffb580191b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 msgr2=0x7ffb5c02d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.799+0000 7ffb52ffd640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c138c90 0x7ffb5c02d450 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ae000 tx=0x7ffb580191b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb6449c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb6449c640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 0x7ffb5c02dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb6449c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ffb5c06a200 con 0x7ffb5c02d990 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 0x7ffb5c02dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient: get_auth_request con 0x7ffb5c02d990 auth_method 0 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 10 monclient(hunting): handle_auth_done global_id 64735 payload 824 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 1 -- 172.21.15.105:0/22439565 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffb5c0a1120 con 0x7ffb5c02d990 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb52ffd640 10 client.74453.objecter ms_handle_connect 0x7ffb5c02d990 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb52ffd640 10 client.74453.objecter resend_mon_ops 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.803+0000 7ffb62238640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 0x7ffb5c02dda0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb5801c240 tx=0x7ffb580a6000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb580c3070 con 0x7ffb5c02d990 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 10 monclient: dump: 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.826 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.827 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9455 (secure 0 0 0) 0x7ffb580b6ad0 con 0x7ffb5c02d990 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 msgr2=0x7ffb5c02dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 0x7ffb5c02dda0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb5801c240 tx=0x7ffb580a6000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 msgr2=0x7ffb5c02dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:04.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb52ffd640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ffb5c02d990 0x7ffb5c02dda0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb5801c240 tx=0x7ffb580a6000 comp rx=0 tx=0).stop 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 102, 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 417907, 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:44:04.829 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 5, 2024-09-17T16:44:04.830 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.831 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.832 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:04.833 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:04.834 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:04.835 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.836 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.837 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:04.839 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.840 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:04.841 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.843 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.844 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:04.845 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.846 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.847 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.849 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:04.850 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:04.851 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:04.852 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:04.853 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 10 librados: watch_flush enter 2024-09-17T16:44:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 10 librados: watch_flush exit 2024-09-17T16:44:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 20 client.74453.objecter shutdown clearing up homeless session... 2024-09-17T16:44:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 10 client.74453.objecter successfully canceled tick 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb4003e480 msgr2=0x7ffb40040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ffb4003e480 0x7ffb40040940 secure :-1 s=READY pgs=3647 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c1396b0 tx=0x7ffb54007650 comp rx=0 tx=0).stop 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 10 monclient: shutdown 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb5c0a6a30 msgr2=0x7ffb5c13d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 --2- 172.21.15.105:0/22439565 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ffb5c0a6a30 0x7ffb5c13d860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00b3f0 tx=0x7ffb4c00b8b0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb62a39640 1 -- 172.21.15.105:0/22439565 reap_dead start 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb62a39640 1 -- 172.21.15.105:0/22439565 reap_dead start 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 shutdown_connections 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 >> 172.21.15.105:0/22439565 conn(0x7ffb5c0a8d00 msgr2=0x7ffb5c13fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 shutdown_connections 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 -- 172.21.15.105:0/22439565 wait complete. 2024-09-17T16:44:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.807+0000 7ffb6449c640 1 librados: shutdown 2024-09-17T16:44:04.856 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:44:04.856 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'a', 'f'] 2024-09-17T16:44:04.856 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['i'] 2024-09-17T16:44:04.856 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-17T16:44:04.856 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-09-17T16:44:04.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-09-17T16:44:04.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: get_monmap_and_config 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: build_initial_monmap 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: monmap: 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:04.977427+0000 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:04.977427+0000 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:04.983 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 Processor -- start 2024-09-17T16:44:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 -- start start 2024-09-17T16:44:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: init 2024-09-17T16:44:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient: _add_conns ranks=[2,1,0,4,10,6,3,5,8,7,9] 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 0x7ff1600a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-b con 0x7ff1600a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff1600a4ab0 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff1600a7350 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1600a4080 con 0x7ff1600a7350 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff16013a610 con 0x7ff1600a4ab0 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff1600a7c40 con 0x7ff1600a6a30 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.975+0000 7ff167cfc640 10 monclient(hunting): authenticate will time out at 1928.243957s 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 0x7ff1600a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:04.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff166299640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff1600a7350 0x7ff160139f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42306/0 (socket says 172.21.15.105:42306) 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff166299640 1 -- 172.21.15.105:0/402358362 learned_addr learned my addr 172.21.15.105:0/402358362 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): get_auth_request con 0x7ff1600a6a30 auth_method 0 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 1 -- 172.21.15.105:0/402358362 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157776855 0 0) 0x7ff1600a4080 con 0x7ff1600a7350 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient(hunting): _negotiate 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient(hunting): my global_id is 36648 2024-09-17T16:44:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 1 -- 172.21.15.105:0/402358362 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff14c003530 con 0x7ff1600a7350 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient(hunting): handle_auth_done global_id 84476 payload 274 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 -- 172.21.15.105:0/402358362 >> v1:172.21.15.160:6789/0 conn(0x7ff1600a4ab0 legacy=0x7ff1601397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 -- 172.21.15.105:0/402358362 >> v1:172.21.15.105:6789/0 conn(0x7ff1600a7350 legacy=0x7ff160139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient: _finish_hunting 0 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 monclient: found mon.noname-b 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 -- 172.21.15.105:0/402358362 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1600a37e0 con 0x7ff1600a6a30 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff165a98640 1 --2- 172.21.15.105:0/402358362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 0x7ff1600a6e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097f0 tx=0x7ff15c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a498e54551874af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 1 -- 172.21.15.105:0/402358362 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff15c0b0070 con 0x7ff1600a6a30 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: dump: 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.995 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.996 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: _finish_auth 0 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: _check_auth_tickets 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 1 -- 172.21.15.105:0/402358362 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c0044a0 con 0x7ff1600a6a30 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 5 monclient: authenticate success, global_id 84476 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 1 -- 172.21.15.105:0/402358362 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff15c003a90 con 0x7ff1600a6a30 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 4 set_mon_vals no callback set 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff164a96640 10 monclient: dump: 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 10 monclient: shutdown 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 -- 172.21.15.105:0/402358362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 msgr2=0x7ff1600a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 --2- 172.21.15.105:0/402358362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 0x7ff1600a6e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097f0 tx=0x7ff15c01c370 comp rx=0 tx=0).stop 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 -- 172.21.15.105:0/402358362 shutdown_connections 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 --2- 172.21.15.105:0/402358362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1600a6a30 0x7ff1600a6e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 -- 172.21.15.105:0/402358362 >> 172.21.15.105:0/402358362 conn(0x7ff1600a8d00 msgr2=0x7ff1600a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 -- 172.21.15.105:0/402358362 shutdown_connections 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.979+0000 7ff167cfc640 1 -- 172.21.15.105:0/402358362 wait complete. 2024-09-17T16:44:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: build_initial_monmap 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: monmap: 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 librados: starting msgr at 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 librados: starting objecter 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 Processor -- start 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- start start 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 librados: setting wanted keys 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 librados: calling monclient init 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: init 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _add_conns ranks=[1,8,7,5,3,6,2,4,0] 2024-09-17T16:44:05.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1600a4ab0 0x7ff160140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-f con 0x7ff1600a4ab0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 0x7ff16013dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-e con 0x7ff1600a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff1600a7350 0x7ff16013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): picked mon.noname-i con 0x7ff1600a7350 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff16006e770 con 0x7ff1600a4ab0 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7ff1600a7780 con 0x7ff1600a7350 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7ff160145690 con 0x7ff1600a6a30 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): authenticate will time out at 1928.249230s 2024-09-17T16:44:05.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 0x7ff16013dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 0x7ff16013dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33714/0 (socket says 172.21.15.105:33714) 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 -- 172.21.15.105:0/2675045595 learned_addr learned my addr 172.21.15.105:0/2675045595 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): get_auth_request con 0x7ff1600a6a30 auth_method 0 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1600a4ab0 0x7ff160140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff166299640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff1600a7350 0x7ff16013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): get_auth_request con 0x7ff1600a4ab0 auth_method 0 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff166299640 10 monclient(hunting): get_auth_request con 0x7ff1600a7350 auth_method 0 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff166299640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff166299640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff166299640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient(hunting): handle_auth_done global_id 55007 payload 824 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff1600a7350 msgr2=0x7ff16013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7ff1600a7350 0x7ff16013e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1600a4ab0 msgr2=0x7ff160140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1600a4ab0 0x7ff160140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient: _finish_hunting 0 2024-09-17T16:44:05.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 monclient: found mon.noname-e 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff160122820 con 0x7ff1600a6a30 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165297640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 0x7ff16013dfb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c4b0 tx=0x7ff15000c970 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 client.?.objecter ms_handle_connect 0x7ff1600a6a30 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff150012630 con 0x7ff1600a6a30 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: dump: 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.005 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 5 monclient: authenticate success, global_id 55007 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff150012830 con 0x7ff1600a6a30 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff150012e50 con 0x7ff1600a6a30 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _renew_subs 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1600a1de0 con 0x7ff1600a6a30 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff157fff640 4 set_mon_vals no callback set 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 monclient: dump: 2024-09-17T16:44:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.008 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 client.55007.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _renew_subs 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16013a360 con 0x7ff1600a6a30 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 librados: init done 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 0x7ff16013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff160051e80 con 0x7ff16013ea30 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 0x7ff16013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7ff1500129e0 con 0x7ff1600a6a30 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient: get_auth_request con 0x7ff16013ea30 auth_method 0 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff14403e6f0 0x7ff144040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7ff15004fa40 con 0x7ff1600a6a30 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 10 client.55007.objecter ms_dispatch 0x7ff160000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 3 client.55007.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 3 client.55007.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff1567fc640 20 client.55007.objecter dump_active .. 0 homeless 2024-09-17T16:44:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 10 monclient(hunting): handle_auth_done global_id 74462 payload 824 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.983+0000 7ff165a98640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff16013ee10 con 0x7ff16013ea30 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff166299640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff14403e6f0 0x7ff144040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff166299640 10 monclient: get_auth_request con 0x7ff14403e6f0 auth_method 0 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 client.55007.objecter ms_handle_connect 0x7ff16013ea30 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 client.55007.objecter resend_mon_ops 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff165a98640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 0x7ff16013a800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff16006ebb0 tx=0x7ff15c019110 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff15c0b0040 con 0x7ff16013ea30 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 monclient: dump: 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.011 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 client.55007.objecter ms_handle_connect 0x7ff14403e6f0 2024-09-17T16:44:05.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff166299640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff14403e6f0 0x7ff144040bb0 secure :-1 s=READY pgs=3648 cs=0 l=1 rev1=1 crypto rx=0x7ff158001770 tx=0x7ff158009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff15c003620 con 0x7ff16013ea30 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 msgr2=0x7ff16013a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 0x7ff16013a800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff16006ebb0 tx=0x7ff15c019110 comp rx=0 tx=0).stop 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 msgr2=0x7ff16013a800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.987+0000 7ff1567fc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ea30 0x7ff16013a800 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff16006ebb0 tx=0x7ff15c019110 comp rx=0 tx=0).stop 2024-09-17T16:44:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 0x7ff16013b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7ff1600a6e10 con 0x7ff16013ad40 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 0x7ff16013b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient: get_auth_request con 0x7ff16013ad40 auth_method 0 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 10 monclient(hunting): handle_auth_done global_id 74471 payload 824 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 1 -- 172.21.15.105:0/2675045595 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7ff160140eb0 con 0x7ff16013ad40 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 client.55007.objecter ms_handle_connect 0x7ff16013ad40 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 client.55007.objecter resend_mon_ops 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff165a98640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 0x7ff16013b120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff15c018f80 tx=0x7ff15c0a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff15c0b0070 con 0x7ff16013ad40 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 monclient: dump: 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.016 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7ff15c0b6a20 con 0x7ff16013ad40 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 msgr2=0x7ff16013b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 0x7ff16013b120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff15c018f80 tx=0x7ff15c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 msgr2=0x7ff16013b120 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff1567fc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7ff16013ad40 0x7ff16013b120 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff15c018f80 tx=0x7ff15c0a8000 comp rx=0 tx=0).stop 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:44:05.018 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 librados: watch_flush enter 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 librados: watch_flush exit 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 20 client.55007.objecter shutdown clearing up homeless session... 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 client.55007.objecter successfully canceled tick 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff14403e6f0 msgr2=0x7ff144040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7ff14403e6f0 0x7ff144040bb0 secure :-1 s=READY pgs=3648 cs=0 l=1 rev1=1 crypto rx=0x7ff158001770 tx=0x7ff158009040 comp rx=0 tx=0).stop 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 10 monclient: shutdown 2024-09-17T16:44:05.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 msgr2=0x7ff16013dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 --2- 172.21.15.105:0/2675045595 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7ff1600a6a30 0x7ff16013dfb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c4b0 tx=0x7ff15000c970 comp rx=0 tx=0).stop 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff166299640 1 -- 172.21.15.105:0/2675045595 reap_dead start 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff166299640 1 -- 172.21.15.105:0/2675045595 reap_dead start 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 shutdown_connections 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 >> 172.21.15.105:0/2675045595 conn(0x7ff1600a8d00 msgr2=0x7ff1600a2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 shutdown_connections 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 -- 172.21.15.105:0/2675045595 wait complete. 2024-09-17T16:44:05.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:04.995+0000 7ff167cfc640 1 librados: shutdown 2024-09-17T16:44:05.021 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-17T16:44:05.022 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-17T16:44:05.022 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:44:05.054 INFO:tasks.ceph.mon.h:Stopped 2024-09-17T16:44:05.054 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-17T16:44:05.054 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a store 2024-09-17T16:44:05.055 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a sync_force --yes-i-really-mean-it 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 10 monclient: get_monmap_and_config 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 10 monclient: build_initial_monmap 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 10 monclient: monmap: 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.174 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:05.170363+0000 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:05.170363+0000 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:05.175 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:05.176 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:05.176 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 1 Processor -- start 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 1 -- start start 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.167+0000 7f4d00b90640 10 monclient: init 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient: _add_conns ranks=[3,10,4,0,6,5,1,7,8,2,9] 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0a68e0 0x7f4cfc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-g con 0x7f4cfc0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0a4910 0x7f4cfc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-f con 0x7f4cfc0a4910 addr v2:172.21.15.160:3300/0 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-c con 0x7f4cfc1396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f4cfc0a3fc0 con 0x7f4cfc0a4910 2024-09-17T16:44:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4cfc0a63c0 con 0x7f4cfc0a68e0 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f4cfc0a7740 con 0x7f4cfc1396a0 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient(hunting): authenticate will time out at 1928.436155s 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34200/0 (socket says 172.21.15.105:34200) 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 -- 172.21.15.105:0/135865073 learned_addr learned my addr 172.21.15.105:0/135865073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): get_auth_request con 0x7f4cfc1396a0 auth_method 0 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfa59c640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0a68e0 0x7f4cfc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request con 0x7f4cfc0a68e0 auth_method 0 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfa59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_done global_id 74785 payload 274 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 -- 172.21.15.105:0/135865073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0a68e0 msgr2=0x7f4cfc0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0a68e0 0x7f4cfc0a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 -- 172.21.15.105:0/135865073 >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0a4910 msgr2=0x7f4cfc0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf9d9b640 1 --2- 172.21.15.105:0/135865073 >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0a4910 0x7f4cfc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/135865073 >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0a4910 0x7f4cfc0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 monclient: found mon.noname-c 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 -- 172.21.15.105:0/135865073 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc06ebc0 con 0x7f4cfc1396a0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce401c3f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=76b54b5f15fb2878 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 1 -- 172.21.15.105:0/135865073 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ce40b2070 con 0x7f4cfc1396a0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: dump: 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.189 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: _finish_auth 0 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: _check_auth_tickets 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 5 monclient: authenticate success, global_id 74785 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 1 -- 172.21.15.105:0/135865073 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce40ab7b0 con 0x7f4cfc1396a0 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 4 set_mon_vals no callback set 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 1 -- 172.21.15.105:0/135865073 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ce40abe50 con 0x7f4cfc1396a0 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4cf959a640 10 monclient: dump: 2024-09-17T16:44:05.191 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.192 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 10 monclient: shutdown 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- 172.21.15.105:0/135865073 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 msgr2=0x7f4cfc139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- 172.21.15.105:0/135865073 shutdown_connections 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4cfc1396a0 0x7f4cfc139b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- 172.21.15.105:0/135865073 >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0a4910 0x7f4cfc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 --2- 172.21.15.105:0/135865073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0a68e0 0x7f4cfc0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- 172.21.15.105:0/135865073 >> 172.21.15.105:0/135865073 conn(0x7f4cfc068ba0 msgr2=0x7f4cfc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- 172.21.15.105:0/135865073 shutdown_connections 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.171+0000 7f4d00b90640 1 -- 172.21.15.105:0/135865073 wait complete. 2024-09-17T16:44:05.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: build_initial_monmap 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: monmap: 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:05.194 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 librados: starting msgr at 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 librados: starting objecter 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 Processor -- start 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- start start 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 librados: setting wanted keys 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 librados: calling monclient init 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: init 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _add_conns ranks=[8,0,2,4,6,5,3,1,7] 2024-09-17T16:44:05.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cfc0a4910 0x7f4cfc1d1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-e con 0x7f4cfc0a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc0a68e0 0x7f4cfc1d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-a con 0x7f4cfc0a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 0x7f4cfc13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): picked mon.noname-b con 0x7f4cfc1396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc06e630 con 0x7f4cfc0a68e0 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4cfc0a7af0 con 0x7f4cfc1396a0 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4cfc143320 con 0x7f4cfc0a4910 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): authenticate will time out at 1928.441271s 2024-09-17T16:44:05.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cfc0a4910 0x7f4cfc1d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc0a68e0 0x7f4cfc1d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cfc0a4910 0x7f4cfc1d1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33730/0 (socket says 172.21.15.105:33730) 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 0x7f4cfc13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 -- 172.21.15.105:0/1298493461 learned_addr learned my addr 172.21.15.105:0/1298493461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request con 0x7f4cfc0a4910 auth_method 0 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): get_auth_request con 0x7f4cfc0a68e0 auth_method 0 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): get_auth_request con 0x7f4cfc1396a0 auth_method 0 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient(hunting): handle_auth_done global_id 84485 payload 824 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cfc0a4910 msgr2=0x7f4cfc1d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4cfc0a4910 0x7f4cfc1d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc0a68e0 msgr2=0x7f4cfc1d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc0a68e0 0x7f4cfc1d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 monclient: found mon.noname-b 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc123650 con 0x7f4cfc1396a0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc0a68e0 0x7f4cfc1d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfad9d640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 0x7f4cfc13d4e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40ac000 tx=0x7f4ce4019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 client.?.objecter ms_handle_connect 0x7f4cfc1396a0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ce40bd070 con 0x7f4cfc1396a0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: dump: 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.200 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce40abdd0 con 0x7f4cfc1396a0 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 5 monclient: authenticate success, global_id 84485 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _renew_subs 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc123930 con 0x7f4cfc1396a0 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:05.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ce40b2040 con 0x7f4cfc1396a0 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf8d99640 4 set_mon_vals no callback set 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 client.84485.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 monclient: dump: 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.202 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _renew_subs 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc0a9f50 con 0x7f4cfc1396a0 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 librados: init done 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4ce4004370 con 0x7f4cfc1396a0 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 0x7f4cfc13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc051e80 con 0x7f4cfc13da20 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 0x7f4cfc13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient: get_auth_request con 0x7f4cfc13da20 auth_method 0 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4cdc03e540 0x7f4cdc040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7f4ce40e9a70 con 0x7f4cfc1396a0 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 client.84485.objecter ms_dispatch 0x7f4cfc000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 3 client.84485.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 3 client.84485.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 20 client.84485.objecter dump_active .. 0 homeless 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_done global_id 36666 payload 824 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cfc077ad0 con 0x7f4cfc13da20 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4cdc03e540 0x7f4cdc040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf9d9b640 10 monclient: get_auth_request con 0x7f4cdc03e540 auth_method 0 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 client.84485.objecter ms_handle_connect 0x7f4cfc13da20 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cf2ffd640 10 client.84485.objecter resend_mon_ops 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.175+0000 7f4cfa59c640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 0x7f4cfc13de00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004980 tx=0x7f4cf4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf4007e60 con 0x7f4cfc13da20 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:05.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 monclient: dump: 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.206 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4cf401a070 con 0x7f4cfc13da20 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 msgr2=0x7f4cfc13de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 0x7f4cfc13de00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004980 tx=0x7f4cf4007700 comp rx=0 tx=0).stop 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 msgr2=0x7f4cfc13de00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13da20 0x7f4cfc13de00 secure :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4004980 tx=0x7f4cf4007700 comp rx=0 tx=0).stop 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf2ffd640 10 client.84485.objecter ms_handle_connect 0x7f4cdc03e540 2024-09-17T16:44:05.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.179+0000 7f4cf9d9b640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4cdc03e540 0x7f4cdc040a00 secure :-1 s=READY pgs=3649 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc13ec40 tx=0x7f4cec008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4d00b90640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4d00b90640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 0x7f4cfc0a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4d00b90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc139b50 con 0x7f4cfc13e340 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 0x7f4cfc0a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient: get_auth_request con 0x7f4cfc13e340 auth_method 0 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 10 monclient(hunting): handle_auth_done global_id 36675 payload 824 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.183+0000 7f4cfa59c640 1 -- 172.21.15.105:0/1298493461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f4cfc13f790 con 0x7f4cfc13e340 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cfa59c640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 0x7f4cfc0a0a20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4017040 tx=0x7f4cf4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 client.84485.objecter ms_handle_connect 0x7f4cfc13e340 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 client.84485.objecter resend_mon_ops 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf402c080 con 0x7f4cfc13e340 2024-09-17T16:44:05.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 monclient: dump: 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f4cf40147a0 con 0x7f4cfc13e340 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 msgr2=0x7f4cfc0a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 0x7f4cfc0a0a20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4017040 tx=0x7f4cf4004a50 comp rx=0 tx=0).stop 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 msgr2=0x7f4cfc0a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cf2ffd640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cfc13e340 0x7f4cfc0a0a20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.213 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:05.213 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:44:05.213 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:44:05.213 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 10 librados: watch_flush enter 2024-09-17T16:44:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 10 librados: watch_flush exit 2024-09-17T16:44:05.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 20 client.84485.objecter shutdown clearing up homeless session... 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 10 client.84485.objecter successfully canceled tick 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4cdc03e540 msgr2=0x7f4cdc040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4cdc03e540 0x7f4cdc040a00 secure :-1 s=READY pgs=3649 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc13ec40 tx=0x7f4cec008040 comp rx=0 tx=0).stop 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 10 monclient: shutdown 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 msgr2=0x7f4cfc13d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 0x7f4cfc13d4e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40ac000 tx=0x7f4ce4019190 comp rx=0 tx=0).stop 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4cfad9d640 1 -- 172.21.15.105:0/1298493461 reap_dead start 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 shutdown_connections 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 --2- 172.21.15.105:0/1298493461 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cfc1396a0 0x7f4cfc13d4e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 >> 172.21.15.105:0/1298493461 conn(0x7f4cfc068ba0 msgr2=0x7f4cfc0a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 shutdown_connections 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 -- 172.21.15.105:0/1298493461 wait complete. 2024-09-17T16:44:05.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.187+0000 7f4d00b90640 1 librados: shutdown 2024-09-17T16:44:05.216 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-17T16:44:05.216 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T16:44:05.216 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:44:05.235 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T16:44:05.235 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-17T16:44:05.235 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f store 2024-09-17T16:44:05.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f sync_force --yes-i-really-mean-it 2024-09-17T16:44:05.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.399+0000 7f5946295640 10 monclient: get_monmap_and_config 2024-09-17T16:44:05.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.399+0000 7f5946295640 10 monclient: build_initial_monmap 2024-09-17T16:44:05.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.399+0000 7f5946295640 10 monclient: monmap: 2024-09-17T16:44:05.407 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:05.407 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:05.403285+0000 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:05.403285+0000 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:05.408 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:05.409 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:05.409 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 Processor -- start 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 -- start start 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient: init 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient: _add_conns ranks=[5,0,2,9,7,6,4,8,3,1,10] 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 0x7f5940138c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): picked mon.noname-h con 0x7f5940138870 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f594009ea50 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): picked mon.noname-b con 0x7f59400a4e60 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594009dd90 con 0x7f594009ea50 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f594009e070 con 0x7f59400a4e60 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f594009e350 con 0x7f5940138870 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5946295640 10 monclient(hunting): authenticate will time out at 1928.669550s 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f593f7fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f594009ea50 legacy=0x7f59400a4750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f593ffff640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 msgr2=0x7f5940138c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:05.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f593ffff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 0x7f5940138c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:47272/0 (socket says 172.21.15.105:47272) 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 -- 172.21.15.105:0/3700104469 learned_addr learned my addr 172.21.15.105:0/3700104469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): get_auth_request con 0x7f59400a4e60 auth_method 0 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient(hunting): handle_auth_done global_id 84494 payload 274 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 -- 172.21.15.105:0/3700104469 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 msgr2=0x7f5940138c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 --2- 172.21.15.105:0/3700104469 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 0x7f5940138c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 -- 172.21.15.105:0/3700104469 >> v1:172.21.15.105:6789/0 conn(0x7f594009ea50 legacy=0x7f59400a4750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient: _finish_hunting 0 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 monclient: found mon.noname-b 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:05.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.403+0000 7f5944832640 1 -- 172.21.15.105:0/3700104469 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594006cd40 con 0x7f59400a4e60 2024-09-17T16:44:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 1 --2- 172.21.15.105:0/3700104469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e80 tx=0x7f593801c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da25369799b50031 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 1 -- 172.21.15.105:0/3700104469 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59380090d0 con 0x7f59400a4e60 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: dump: 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.425 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: _finish_auth 0 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 5 monclient: authenticate success, global_id 84494 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:05.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 1 -- 172.21.15.105:0/3700104469 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59380a5990 con 0x7f59400a4e60 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 4 set_mon_vals no callback set 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 1 -- 172.21.15.105:0/3700104469 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59380ad3f0 con 0x7f59400a4e60 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593effd640 10 monclient: dump: 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: shutdown 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3700104469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 msgr2=0x7f59401418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- 172.21.15.105:0/3700104469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e80 tx=0x7f593801c930 comp rx=0 tx=0).stop 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3700104469 shutdown_connections 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- 172.21.15.105:0/3700104469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f59400a4e60 0x7f59401418e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- 172.21.15.105:0/3700104469 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f5940138870 0x7f5940138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3700104469 >> 172.21.15.105:0/3700104469 conn(0x7f59400a5840 msgr2=0x7f59400a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3700104469 shutdown_connections 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3700104469 wait complete. 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: build_initial_monmap 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: monmap: 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:05.430 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 librados: starting msgr at 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 librados: starting objecter 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 Processor -- start 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- start start 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 librados: setting wanted keys 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 librados: calling monclient init 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: init 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:05.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _add_conns ranks=[7,8,1,0,2,4,6,5,3] 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f594009ea50 0x7f59401d1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): picked mon.noname-i con 0x7f594009ea50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 0x7f59401d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): picked mon.noname-e con 0x7f59400a4e60 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5940138870 0x7f59401d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): picked mon.noname-f con 0x7f5940138870 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5940144640 con 0x7f5940138870 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f59401447c0 con 0x7f594009ea50 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f5940144940 con 0x7f59400a4e60 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient(hunting): authenticate will time out at 1928.674590s 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 0x7f59401d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 0x7f59401d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:33740/0 (socket says 172.21.15.105:33740) 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 -- 172.21.15.105:0/3074350585 learned_addr learned my addr 172.21.15.105:0/3074350585 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): get_auth_request con 0x7f59400a4e60 auth_method 0 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5940138870 0x7f59401d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593ffff640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f594009ea50 0x7f59401d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 10 monclient(hunting): get_auth_request con 0x7f5940138870 auth_method 0 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593ffff640 10 monclient(hunting): get_auth_request con 0x7f594009ea50 auth_method 0 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient(hunting): handle_auth_done global_id 55025 payload 824 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f594009ea50 msgr2=0x7f59401d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f594009ea50 0x7f59401d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5940138870 msgr2=0x7f59401d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5940138870 0x7f59401d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 monclient: found mon.noname-e 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5940121770 con 0x7f59400a4e60 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5944832640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5940138870 0x7f59401d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593f7fe640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 0x7f59401d1d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f593400b4a0 tx=0x7f593400b960 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 client.?.objecter ms_handle_connect 0x7f59400a4e60 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5934003ba0 con 0x7f59400a4e60 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: dump: 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: _finish_auth 0 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5934015070 con 0x7f59400a4e60 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 5 monclient: authenticate success, global_id 55025 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _renew_subs 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5940141170 con 0x7f59400a4e60 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5934003d80 con 0x7f59400a4e60 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593cff9640 10 monclient: dump: 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.438 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 client.55025.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _renew_subs 2024-09-17T16:44:05.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f593e7fc640 4 set_mon_vals no callback set 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f5940139e10 con 0x7f59400a4e60 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.407+0000 7f5946295640 1 librados: init done 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5946295640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5946295640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 0x7f594011f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f59401393a0 con 0x7f594011f390 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f5934005070 con 0x7f59400a4e60 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 0x7f594011f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient: get_auth_request con 0x7f594011f390 auth_method 0 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f592c03e550 0x7f592c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(644..644 src has 1..644) ==== 6761+0+0 (secure 0 0 0) 0x7f593400d040 con 0x7f59400a4e60 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 client.55025.objecter ms_dispatch 0x7f5940000ba0 osd_map(644..644 src has 1..644) 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 3 client.55025.objecter handle_osd_map got epochs [644,644] > 0 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 3 client.55025.objecter handle_osd_map decoding full epoch 644 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 20 client.55025.objecter dump_active .. 0 homeless 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5944832640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f592c03e550 0x7f592c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5944832640 10 monclient: get_auth_request con 0x7f592c03e550 auth_method 0 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 10 monclient(hunting): handle_auth_done global_id 84457 payload 824 2024-09-17T16:44:05.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59400a24f0 con 0x7f594011f390 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 client.55025.objecter ms_handle_connect 0x7f594011f390 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 client.55025.objecter resend_mon_ops 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593ffff640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 0x7f594011f770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f593000d700 tx=0x7f593000dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 client.55025.objecter ms_handle_connect 0x7f592c03e550 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f5944832640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f592c03e550 0x7f592c040a10 secure :-1 s=READY pgs=3650 cs=0 l=1 rev1=1 crypto rx=0x7f59380ae000 tx=0x7f59380a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5930003e30 con 0x7f594011f390 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 monclient: dump: 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5930017070 con 0x7f594011f390 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:05.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 msgr2=0x7f594011f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 0x7f594011f770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f593000d700 tx=0x7f593000dbc0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 msgr2=0x7f594011f770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.411+0000 7f593cff9640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011f390 0x7f594011f770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f5946295640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f5946295640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 0x7f594013c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f5946295640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5940051e80 con 0x7f594011fcb0 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 0x7f594013c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient: get_auth_request con 0x7f594011fcb0 auth_method 0 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:05.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 10 monclient(hunting): handle_auth_done global_id 84466 payload 824 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 1 -- 172.21.15.105:0/3074350585 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f5940141a20 con 0x7f594011fcb0 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593ffff640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 0x7f594013c6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f593000d700 tx=0x7f59300059e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 10 client.55025.objecter ms_handle_connect 0x7f594011fcb0 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 10 client.55025.objecter resend_mon_ops 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f593002a080 con 0x7f594011fcb0 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.419+0000 7f593cff9640 10 monclient: dump: 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:05.447 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f593000b730 con 0x7f594011fcb0 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:05.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 msgr2=0x7f594013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 0x7f594013c6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f593000d700 tx=0x7f59300059e0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 msgr2=0x7f594013c6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f593cff9640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f594011fcb0 0x7f594013c6a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stdout: "ret": 0, 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-09-17T16:44:05.449 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 10 librados: watch_flush enter 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 10 librados: watch_flush exit 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 20 client.55025.objecter shutdown clearing up homeless session... 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 10 client.55025.objecter successfully canceled tick 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f592c03e550 msgr2=0x7f592c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f592c03e550 0x7f592c040a10 secure :-1 s=READY pgs=3650 cs=0 l=1 rev1=1 crypto rx=0x7f59380ae000 tx=0x7f59380a7070 comp rx=0 tx=0).stop 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 10 monclient: shutdown 2024-09-17T16:44:05.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 msgr2=0x7f59401d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 --2- 172.21.15.105:0/3074350585 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f59400a4e60 0x7f59401d1d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f593400b4a0 tx=0x7f593400b960 comp rx=0 tx=0).stop 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5944832640 1 -- 172.21.15.105:0/3074350585 reap_dead start 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5944832640 1 -- 172.21.15.105:0/3074350585 reap_dead start 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 shutdown_connections 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 >> 172.21.15.105:0/3074350585 conn(0x7f59400a5840 msgr2=0x7f59400a6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 shutdown_connections 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 -- 172.21.15.105:0/3074350585 wait complete. 2024-09-17T16:44:05.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:05.423+0000 7f5946295640 1 librados: shutdown 2024-09-17T16:44:05.452 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-17T16:44:05.453 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-17T16:44:05.453 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:44:05.476 INFO:tasks.ceph.mon.f:Stopped 2024-09-17T16:44:05.476 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-17T16:44:05.476 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-17T16:44:05.476 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:44:20.477 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-17T16:44:20.478 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-17T16:44:20.478 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-17T16:44:20.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: get_monmap_and_config 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: build_initial_monmap 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: monmap: 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:20.605 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:20.601235+0000 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:20.601235+0000 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:20.606 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 Processor -- start 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 -- start start 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: init 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient: _add_conns ranks=[6,1,7,10,9,4,0,8,5,3,2] 2024-09-17T16:44:20.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-d con 0x7f242c0a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f242c0a4ab0 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-i con 0x7f242c0a7350 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0a4080 con 0x7f242c0a4ab0 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f242c0a7c40 con 0x7f242c0a6a30 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f242c0a37e0 con 0x7f242c0a7350 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f24319ff640 10 monclient(hunting): authenticate will time out at 1943.867004s 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:20.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242a7fc640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f242c0a4ab0 legacy=0x7f242c1397f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:41522/0 (socket says 172.21.15.105:41522) 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 learned_addr learned my addr 172.21.15.105:0/1688420608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f242c0a6a30 auth_method 0 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f242c0a7350 auth_method 0 2024-09-17T16:44:20.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:20.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.599+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f242c0a7350 auth_method 0 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:20.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.160:6789/0 conn(0x7f242c0a4ab0 legacy=0x7f242c1397f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f242c0a6a30 auth_method 0 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:20.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:20.803+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f242c0a6a30 auth_method 0 2024-09-17T16:44:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:21.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.160:6789/0 conn(0x7f242c0a4ab0 legacy=0x7f242c1397f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f242c0a7350 auth_method 0 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:21.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:44:21.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:21.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:21.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:21.207+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.160:6789/0 conn(0x7f242c0a4ab0 legacy=0x7f242c1397f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f242c0a6a30 auth_method 0 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:22.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f242c0a7350 auth_method 0 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:22.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:22.011+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): tick 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 msgr2=0x7f242c139f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:23.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f242c0a7350 0x7f242c139f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 msgr2=0x7f242c0a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f242c0a6a30 0x7f242c0a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.160:6789/0 conn(0x7f242c0a4ab0 legacy=0x7f242c1397f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient: _add_conns ranks=[9,5,10,2,8,0,4,3] 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f2418003810 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f2418006240 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f2418008d90 addr v2:172.21.15.160:3300/0 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f242c0a7c40 con 0x7f2418003810 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f242c0a37e0 con 0x7f2418008d90 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f241800b950 con 0x7f2418006240 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 msgr2=0x7f2418005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f2428ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 msgr2=0x7f241800b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:23.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.599+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 msgr2=0x7f2418005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 msgr2=0x7f241800b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:23.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:23.803+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 msgr2=0x7f241800b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 msgr2=0x7f2418005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:24.207+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 msgr2=0x7f2418005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 msgr2=0x7f241800b250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:25.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:25.011+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): tick 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2418006240 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:26.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 msgr2=0x7f241800b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.160:3300/0 conn(0x7f2418008d90 0x7f241800b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 msgr2=0x7f2418005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> v2:172.21.15.105:3300/0 conn(0x7f2418003810 0x7f2418005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 reap_dead start 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient: _add_conns ranks=[2,0,8,4,3] 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 reap_dead start 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f2418003c30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2418009780 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f241800b280 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.599+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0a3ac0 con 0x7f2418009780 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0a7c40 con 0x7f2418003c30 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f242c0a37e0 con 0x7f241800b280 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f2428ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.105:6789/0 conn(0x7f2418009780 legacy=0x7f2418009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f2418003c30 auth_method 0 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f241800b280 auth_method 0 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:26.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:26.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.603+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f2418003c30 auth_method 0 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f241800b280 auth_method 0 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:26.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.105:6789/0 conn(0x7f2418009780 legacy=0x7f2418009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:26.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:26.807+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.105:6789/0 conn(0x7f2418009780 legacy=0x7f2418009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:27.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f2418003c30 auth_method 0 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f241800b280 auth_method 0 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:27.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:27.211+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:28.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.105:6789/0 conn(0x7f2418009780 legacy=0x7f2418009b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f2418003c30 auth_method 0 2024-09-17T16:44:28.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 10 monclient(hunting): get_auth_request con 0x7f241800b280 auth_method 0 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:28.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:28.015+0000 7f242b7fe640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): tick 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 monclient(hunting): continuing hunt 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 msgr2=0x7f24180086d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f241800b280 0x7f24180086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 msgr2=0x7f2418009240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2418003c30 0x7f2418009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:29.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 >> v1:172.21.15.105:6789/0 conn(0x7f2418009780 legacy=0x7f2418009b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient: _add_conns ranks=[4,3] 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f241800e030 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f241800ea20 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0a7c40 con 0x7f241800ea20 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f242c0a37e0 con 0x7f241800e030 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f2428ff9640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:29.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f241800e030 auth_method 0 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 10 monclient(hunting): get_auth_request con 0x7f241800ea20 auth_method 0 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:29.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:29.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:29.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.603+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:29.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:29.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f241800e030 auth_method 0 2024-09-17T16:44:29.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:29.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 10 monclient(hunting): get_auth_request con 0x7f241800ea20 auth_method 0 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:29.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:29.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:29.807+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f241800e030 auth_method 0 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:30.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 10 monclient(hunting): get_auth_request con 0x7f241800ea20 auth_method 0 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:44:30.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:30.211+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): get_auth_request con 0x7f241800ea20 auth_method 0 2024-09-17T16:44:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f241800e030 auth_method 0 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient(hunting): handle_auth_done global_id 54957 payload 274 2024-09-17T16:44:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 msgr2=0x7f241800e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f241800e030 0x7f241800e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 1 monclient: found mon.noname-g 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:44:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.015+0000 7f242a7fc640 1 -- 172.21.15.105:0/1688420608 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0a4080 con 0x7f241800ea20 2024-09-17T16:44:31.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.019+0000 7f242a7fc640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f241c009d30 tx=0x7f241c01d9b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b19288db5066a549 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.019+0000 7f2429ffb640 1 -- 172.21.15.105:0/1688420608 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f241c0ac070 con 0x7f241800ea20 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.019+0000 7f2429ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: dump: 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.028 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 1 -- 172.21.15.105:0/1688420608 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c0a6b50 con 0x7f241800ea20 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 1 -- 172.21.15.105:0/1688420608 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f241c009440 con 0x7f241800ea20 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 5 monclient: authenticate success, global_id 54957 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 4 set_mon_vals no callback set 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f2429ffb640 10 monclient: dump: 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.030 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.031 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: shutdown 2024-09-17T16:44:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 msgr2=0x7f24180132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 --2- 172.21.15.105:0/1688420608 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f241800ea20 0x7f24180132a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f241c009d30 tx=0x7f241c01d9b0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f242b7fe640 1 -- 172.21.15.105:0/1688420608 reap_dead start 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- 172.21.15.105:0/1688420608 shutdown_connections 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- 172.21.15.105:0/1688420608 >> 172.21.15.105:0/1688420608 conn(0x7f242c0a8d00 msgr2=0x7f242c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- 172.21.15.105:0/1688420608 shutdown_connections 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- 172.21.15.105:0/1688420608 wait complete. 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: monmap: 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 librados: starting msgr at 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 librados: starting objecter 2024-09-17T16:44:31.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 Processor -- start 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- start start 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 librados: setting wanted keys 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 librados: calling monclient init 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: init 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient: _add_conns ranks=[7,2,0,4,5,3,1,6,8] 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f241800ea20 0x7f242c138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-i con 0x7f241800ea20 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-b con 0x7f242c138970 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:31.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 0x7f242c122bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): picked mon.noname-a con 0x7f242c138d90 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f242c06e860 con 0x7f242c138d90 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f242c06e6e0 con 0x7f242c138970 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f242c13a000 con 0x7f241800ea20 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f24319ff640 10 monclient(hunting): authenticate will time out at 1954.291879s 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f242b7fe640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 msgr2=0x7f242c122bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.023+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 0x7f242c122bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38042/0 (socket says 172.21.15.105:38042) 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 -- 172.21.15.105:0/4193006221 learned_addr learned my addr 172.21.15.105:0/4193006221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): get_auth_request con 0x7f242c138970 auth_method 0 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f241800ea20 0x7f242c138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 10 monclient(hunting): get_auth_request con 0x7f241800ea20 auth_method 0 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient(hunting): handle_auth_done global_id 84503 payload 824 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 -- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f241800ea20 msgr2=0x7f242c138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f241800ea20 0x7f242c138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 -- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 msgr2=0x7f242c122bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 0x7f242c122bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 monclient: found mon.noname-b 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 -- 172.21.15.105:0/4193006221 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c06def0 con 0x7f242c138970 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242a7fc640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f241c01e250 tx=0x7f241c01a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 client.?.objecter ms_handle_connect 0x7f242c138970 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f241c0bf030 con 0x7f242c138970 2024-09-17T16:44:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: dump: 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c0a6920 con 0x7f242c138970 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f241c0ac040 con 0x7f242c138970 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 5 monclient: authenticate success, global_id 84503 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _renew_subs 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c1395a0 con 0x7f242c138970 2024-09-17T16:44:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24297fa640 4 set_mon_vals no callback set 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 monclient: dump: 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.044 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 client.84503.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _renew_subs 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1248a0 con 0x7f242c138970 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 1 librados: init done 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c124a70 con 0x7f242c138970 2024-09-17T16:44:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f241c0c2070 con 0x7f242c138970 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 0x7f24180427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(646..646 src has 1..646) ==== 6774+0+0 (secure 0 0 0) 0x7f241c0c0070 con 0x7f242c138970 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 10 client.84503.objecter ms_dispatch 0x7f242c000ba0 osd_map(646..646 src has 1..646) 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 3 client.84503.objecter handle_osd_map got epochs [646,646] > 0 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 3 client.84503.objecter handle_osd_map decoding full epoch 646 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f2413fff640 20 client.84503.objecter dump_active .. 0 homeless 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 0x7f24180427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.027+0000 7f242affd640 10 monclient: get_auth_request con 0x7f24180402f0 auth_method 0 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.031+0000 7f2413fff640 10 client.84503.objecter ms_handle_connect 0x7f24180402f0 2024-09-17T16:44:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.031+0000 7f242affd640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 0x7f24180427b0 secure :-1 s=READY pgs=3652 cs=0 l=1 rev1=1 crypto rx=0x7f2420010230 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.031+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f242c124a70 con 0x7f242c138970 2024-09-17T16:44:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.031+0000 7f2413fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.031+0000 7f2413fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:31.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f24319ff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:44:31.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f24319ff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:44:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f24319ff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f242c13a390 con 0x7f242c138970 2024-09-17T16:44:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f2413fff640 1 -- 172.21.15.105:0/4193006221 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3195 (secure 0 0 0) 0x7f242c06def0 con 0x7f242c138970 2024-09-17T16:44:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f2413fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:44:31.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.267+0000 7f2413fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:31.277 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":110,"quorum":[2,3,4,6,7,8],"quorum_names":["b","g","c","d","i","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:44:31.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 10 librados: watch_flush enter 2024-09-17T16:44:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 10 librados: watch_flush exit 2024-09-17T16:44:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 20 client.84503.objecter shutdown clearing up homeless session... 2024-09-17T16:44:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 10 client.84503.objecter successfully canceled tick 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 msgr2=0x7f24180427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 0x7f24180427b0 secure :-1 s=READY pgs=3652 cs=0 l=1 rev1=1 crypto rx=0x7f2420010230 tx=0x7f2420008040 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 10 monclient: shutdown 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 msgr2=0x7f242c122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f241c01e250 tx=0x7f241c01a750 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 shutdown_connections 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f24180402f0 0x7f24180427b0 unknown :-1 s=CLOSED pgs=3652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c138d90 0x7f242c122bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c138970 0x7f242c122680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 --2- 172.21.15.105:0/4193006221 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f241800ea20 0x7f242c138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 >> 172.21.15.105:0/4193006221 conn(0x7f242c0a8d00 msgr2=0x7f242c1c1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 shutdown_connections 2024-09-17T16:44:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 -- 172.21.15.105:0/4193006221 wait complete. 2024-09-17T16:44:31.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.271+0000 7f24319ff640 1 librados: shutdown 2024-09-17T16:44:31.295 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-17T16:44:31.295 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-17T16:44:31.295 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:44:31.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: get_monmap_and_config 2024-09-17T16:44:31.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: monmap: 2024-09-17T16:44:31.412 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:31.408007+0000 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:31.408007+0000 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 Processor -- start 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 -- start start 2024-09-17T16:44:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: init 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient: _add_conns ranks=[0,2,1,9,4,10,6,5,3,8,7] 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9ee00a6a30 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-b con 0x7f9ee00a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9ee00a7520 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00a4080 con 0x7f9ee00a6a30 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00a37e0 con 0x7f9ee00a7520 2024-09-17T16:44:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee00a7c40 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee76c2640 10 monclient(hunting): authenticate will time out at 1954.673648s 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee545e640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f9ee00a6a30 legacy=0x7f9ee00a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee5c5f640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f9ee00a7520 legacy=0x7f9ee0140f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38050/0 (socket says 172.21.15.105:38050) 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 -- 172.21.15.105:0/3922400435 learned_addr learned my addr 172.21.15.105:0/3922400435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): get_auth_request con 0x7f9ee00a4ab0 auth_method 0 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_done global_id 84512 payload 274 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 -- 172.21.15.105:0/3922400435 >> v1:172.21.15.160:6789/0 conn(0x7f9ee00a7520 legacy=0x7f9ee0140f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 -- 172.21.15.105:0/3922400435 >> v1:172.21.15.105:6789/0 conn(0x7f9ee00a6a30 legacy=0x7f9ee00a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 monclient: found mon.noname-b 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.407+0000 7f9ee4c5d640 1 -- 172.21.15.105:0/3922400435 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00a3ac0 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 1 --2- 172.21.15.105:0/3922400435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0098a0 tx=0x7f9edc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e3392087eb6e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 1 -- 172.21.15.105:0/3922400435 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9edc0b2070 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: dump: 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.427 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 1 -- 172.21.15.105:0/3922400435 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc0ab7f0 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 5 monclient: authenticate success, global_id 84512 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 1 -- 172.21.15.105:0/3922400435 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9edc0abe50 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 4 set_mon_vals no callback set 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed7fff640 10 monclient: dump: 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.429 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: shutdown 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/3922400435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 msgr2=0x7f9ee013ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/3922400435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0098a0 tx=0x7f9edc01c400 comp rx=0 tx=0).stop 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/3922400435 shutdown_connections 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/3922400435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee00a4ab0 0x7f9ee013ea30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/3922400435 >> 172.21.15.105:0/3922400435 conn(0x7f9ee00a8d00 msgr2=0x7f9ee00a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/3922400435 shutdown_connections 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/3922400435 wait complete. 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: monmap: 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.431 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 librados: starting msgr at 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 librados: starting objecter 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 Processor -- start 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- start start 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 librados: setting wanted keys 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 librados: calling monclient init 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: init 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _add_conns ranks=[8,4,6,3,7,2,5,1,0] 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 0x7f9ee01d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-e con 0x7f9ee00a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee00a6a30 0x7f9ee01d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-c con 0x7f9ee00a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee00a7520 0x7f9ee01d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): picked mon.noname-d con 0x7f9ee00a7520 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9ee0145040 con 0x7f9ee00a6a30 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f9ee0144ec0 con 0x7f9ee00a7520 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f9ee01451c0 con 0x7f9ee00a4ab0 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient(hunting): authenticate will time out at 1954.678581s 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 0x7f9ee01d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 0x7f9ee01d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:37046/0 (socket says 172.21.15.105:37046) 2024-09-17T16:44:31.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 1 -- 172.21.15.105:0/388563799 learned_addr learned my addr 172.21.15.105:0/388563799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee00a6a30 0x7f9ee01d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): get_auth_request con 0x7f9ee00a4ab0 auth_method 0 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee00a7520 0x7f9ee01d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): get_auth_request con 0x7f9ee00a6a30 auth_method 0 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): get_auth_request con 0x7f9ee00a7520 auth_method 0 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient(hunting): handle_auth_done global_id 94272 payload 824 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 msgr2=0x7f9ee01d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 0x7f9ee01d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee00a6a30 msgr2=0x7f9ee01d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee00a6a30 0x7f9ee01d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 monclient: found mon.noname-d 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee0129cf0 con 0x7f9ee00a7520 2024-09-17T16:44:31.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee545e640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f9ee00a4ab0 0x7f9ee01d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee4c5d640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f9ee00a6a30 0x7f9ee01d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee5c5f640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee00a7520 0x7f9ee01d1b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800ba10 tx=0x7f9ed800bed0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 client.?.objecter ms_handle_connect 0x7f9ee00a7520 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed800cd90 con 0x7f9ee00a7520 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: dump: 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.437 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 5 monclient: authenticate success, global_id 94272 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _renew_subs 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed8010070 con 0x7f9ee00a7520 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee0129fd0 con 0x7f9ee00a7520 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed80158e0 con 0x7f9ee00a7520 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed77fe640 4 set_mon_vals no callback set 2024-09-17T16:44:31.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 client.94272.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ed5ffb640 10 monclient: dump: 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.440 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _renew_subs 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee006e0c0 con 0x7f9ee00a7520 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.411+0000 7f9ee76c2640 1 librados: init done 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee76c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 0x7f9ee01266e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee0051e80 con 0x7f9ee0126300 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 0x7f9ee01266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient: get_auth_request con 0x7f9ee0126300 auth_method 0 2024-09-17T16:44:31.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f9ed8015470 con 0x7f9ee00a7520 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 10 monclient(hunting): handle_auth_done global_id 84521 payload 824 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ee00a6e10 con 0x7f9ee0126300 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ec403e550 0x7f9ec4040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(646..646 src has 1..646) ==== 6774+0+0 (secure 0 0 0) 0x7f9ed8050120 con 0x7f9ee00a7520 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 client.94272.objecter ms_dispatch 0x7f9ee0000ba0 osd_map(646..646 src has 1..646) 2024-09-17T16:44:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 3 client.94272.objecter handle_osd_map got epochs [646,646] > 0 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 3 client.94272.objecter handle_osd_map decoding full epoch 646 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee545e640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 0x7f9ee01266e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed0005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 20 client.94272.objecter dump_active .. 0 homeless 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 client.94272.objecter ms_handle_connect 0x7f9ee0126300 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 client.94272.objecter resend_mon_ops 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed0016070 con 0x7f9ee0126300 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 monclient: dump: 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ed000eaf0 con 0x7f9ee0126300 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 msgr2=0x7f9ee01266e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 0x7f9ee01266e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed0005800 comp rx=0 tx=0).stop 2024-09-17T16:44:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee4c5d640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ec403e550 0x7f9ec4040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 msgr2=0x7f9ee01266e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126300 0x7f9ee01266e0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed0005800 comp rx=0 tx=0).stop 2024-09-17T16:44:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee4c5d640 10 monclient: get_auth_request con 0x7f9ec403e550 auth_method 0 2024-09-17T16:44:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ed5ffb640 10 client.94272.objecter ms_handle_connect 0x7f9ec403e550 2024-09-17T16:44:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.415+0000 7f9ee4c5d640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ec403e550 0x7f9ec4040a10 secure :-1 s=READY pgs=3653 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0af000 tx=0x7f9edc009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee76c2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 0x7f9ee0122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee76c2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee0129160 con 0x7f9ee0126c20 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 0x7f9ee0122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient: get_auth_request con 0x7f9ee0126c20 auth_method 0 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 10 monclient(hunting): handle_auth_done global_id 84530 payload 824 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 1 -- 172.21.15.105:0/388563799 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ee00008e0 con 0x7f9ee0126c20 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 client.94272.objecter ms_handle_connect 0x7f9ee0126c20 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 client.94272.objecter resend_mon_ops 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ee545e640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 0x7f9ee0122680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed00059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ed0016040 con 0x7f9ee0126c20 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 monclient: dump: 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7f9ed001b450 con 0x7f9ee0126c20 2024-09-17T16:44:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 msgr2=0x7f9ee0122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 0x7f9ee0122680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed00059e0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 msgr2=0x7f9ee0122680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.419+0000 7f9ed5ffb640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9ee0126c20 0x7f9ee0122680 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005340 tx=0x7f9ed00059e0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:31.450 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 178654, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 0, 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.451 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:31.452 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:31.454 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.455 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.456 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.457 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:31.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.461 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.462 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.463 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:31.464 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.465 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:31.466 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.468 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.470 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:31.471 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:31.472 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 10 librados: watch_flush enter 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 10 librados: watch_flush exit 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 20 client.94272.objecter shutdown clearing up homeless session... 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 10 client.94272.objecter successfully canceled tick 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ec403e550 msgr2=0x7f9ec4040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f9ec403e550 0x7f9ec4040a10 secure :-1 s=READY pgs=3653 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0af000 tx=0x7f9edc009040 comp rx=0 tx=0).stop 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 10 monclient: shutdown 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee00a7520 msgr2=0x7f9ee01d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 --2- 172.21.15.105:0/388563799 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f9ee00a7520 0x7f9ee01d1b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800ba10 tx=0x7f9ed800bed0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee5c5f640 1 -- 172.21.15.105:0/388563799 reap_dead start 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee5c5f640 1 -- 172.21.15.105:0/388563799 reap_dead start 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 shutdown_connections 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 >> 172.21.15.105:0/388563799 conn(0x7f9ee00a8d00 msgr2=0x7f9ee009fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 shutdown_connections 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 -- 172.21.15.105:0/388563799 wait complete. 2024-09-17T16:44:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.423+0000 7f9ee76c2640 1 librados: shutdown 2024-09-17T16:44:31.475 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:31.475 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-17T16:44:31.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.579+0000 7fe76b723640 10 monclient: get_monmap_and_config 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.579+0000 7fe76b723640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.579+0000 7fe76b723640 10 monclient: monmap: 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:31.582974+0000 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:31.582974+0000 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:31.587 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:31.588 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.579+0000 7fe76b723640 1 Processor -- start 2024-09-17T16:44:31.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 -- start start 2024-09-17T16:44:31.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient: init 2024-09-17T16:44:31.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient: _add_conns ranks=[8,2,4,3,6,10,0,7,9,5,1] 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 0x7fe7640a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-e con 0x7fe7640a6a30 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7640a4ab0 0x7fe7640a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-b con 0x7fe7640a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-c con 0x7fe7641397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe7640a7c40 con 0x7fe7640a4ab0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe7640a7890 con 0x7fe7641397f0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe7640a4080 con 0x7fe7640a6a30 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient(hunting): authenticate will time out at 1954.848568s 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7640a4ab0 0x7fe7640a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 0x7fe7640a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52572/0 (socket says 172.21.15.105:52572) 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 -- 172.21.15.105:0/1663247206 learned_addr learned my addr 172.21.15.105:0/1663247206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): get_auth_request con 0x7fe7641397f0 auth_method 0 2024-09-17T16:44:31.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): get_auth_request con 0x7fe7640a6a30 auth_method 0 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): get_auth_request con 0x7fe7640a4ab0 auth_method 0 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe768cbe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient(hunting): handle_auth_done global_id 74812 payload 274 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 -- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 msgr2=0x7fe7640a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 0x7fe7640a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 -- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7640a4ab0 msgr2=0x7fe7640a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7640a4ab0 0x7fe7640a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 monclient: found mon.noname-c 2024-09-17T16:44:31.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 -- 172.21.15.105:0/1663247206 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640a37e0 con 0x7fe7641397f0 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 0x7fe7640a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009eb0 tx=0x7fe75c01c8c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c5423352e569682c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 1 -- 172.21.15.105:0/1663247206 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe75c0090d0 con 0x7fe7641397f0 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: dump: 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.597 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 1 -- 172.21.15.105:0/1663247206 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c0a5af0 con 0x7fe7641397f0 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 1 -- 172.21.15.105:0/1663247206 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe75c0a5ce0 con 0x7fe7641397f0 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe75bfff640 10 monclient: dump: 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.599 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 5 monclient: authenticate success, global_id 74812 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 4 set_mon_vals no callback set 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 10 monclient: shutdown 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 -- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 msgr2=0x7fe764139c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.583+0000 7fe76b723640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009eb0 tx=0x7fe75c01c8c0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1663247206 shutdown_connections 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7640a4ab0 0x7fe7640a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7641397f0 0x7fe764139c50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- 172.21.15.105:0/1663247206 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7640a6a30 0x7fe7640a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1663247206 >> 172.21.15.105:0/1663247206 conn(0x7fe7640a8d00 msgr2=0x7fe7640a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1663247206 shutdown_connections 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1663247206 wait complete. 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: monmap: 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:31.601 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 librados: starting msgr at 2024-09-17T16:44:31.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 librados: starting objecter 2024-09-17T16:44:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 Processor -- start 2024-09-17T16:44:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- start start 2024-09-17T16:44:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 librados: setting wanted keys 2024-09-17T16:44:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 librados: calling monclient init 2024-09-17T16:44:31.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: init 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _add_conns ranks=[3,0,6,2,8,4,5,1,7] 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7640a4ab0 0x7fe7641381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-g con 0x7fe7640a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7640a6a30 0x7fe7641386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-a con 0x7fe7640a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 0x7fe76413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): picked mon.noname-d con 0x7fe7641397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406e770 con 0x7fe7640a6a30 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe76413a5a0 con 0x7fe7640a4ab0 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe764143260 con 0x7fe7641397f0 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): authenticate will time out at 1954.853605s 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe768cbe640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7640a6a30 msgr2=0x7fe7641386e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe768cbe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7640a6a30 0x7fe7641386e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 0x7fe76413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 0x7fe76413d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:58118/0 (socket says 172.21.15.105:58118) 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 learned_addr learned my addr 172.21.15.105:0/1482583096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): get_auth_request con 0x7fe7641397f0 auth_method 0 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7640a4ab0 0x7fe7641381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient(hunting): handle_auth_done global_id 94281 payload 824 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7640a4ab0 msgr2=0x7fe7641381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7640a4ab0 0x7fe7641381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7640a6a30 msgr2=0x7fe7641386e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7640a6a30 0x7fe7641386e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 monclient: found mon.noname-d 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76413f850 con 0x7fe7641397f0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient: get_auth_request con 0x7fe7640a4ab0 auth_method 0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7640a4ab0 0x7fe7641381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe769cc0640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 0x7fe76413d640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0af040 tx=0x7fe75c019660 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 client.?.objecter ms_handle_connect 0x7fe7641397f0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe75c0bc020 con 0x7fe7641397f0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: dump: 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.613 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c003e10 con 0x7fe7641397f0 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 5 monclient: authenticate success, global_id 94281 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe75c0044b0 con 0x7fe7641397f0 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 10 monclient: dump: 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.615 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe75b7fe640 4 set_mon_vals no callback set 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _renew_subs 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe764123fa0 con 0x7fe7641397f0 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 client.94281.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _renew_subs 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640aba90 con 0x7fe7641397f0 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 librados: init done 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 0x7fe76402d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe764123940 con 0x7fe76413e340 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 0x7fe76402d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient: get_auth_request con 0x7fe76413e340 auth_method 0 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe75c0bf020 con 0x7fe7641397f0 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.587+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe7694bf640 10 monclient(hunting): handle_auth_done global_id 74821 payload 824 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe7694bf640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe764139c50 con 0x7fe76413e340 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe74803e580 0x7fe748040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(646..646 src has 1..646) ==== 6774+0+0 (secure 0 0 0) 0x7fe75c0b3040 con 0x7fe7641397f0 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 client.94281.objecter ms_dispatch 0x7fe764000ba0 osd_map(646..646 src has 1..646) 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 3 client.94281.objecter handle_osd_map got epochs [646,646] > 0 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 3 client.94281.objecter handle_osd_map decoding full epoch 646 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 0x7fe76402d450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7600107c0 tx=0x7fe760010c80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 20 client.94281.objecter dump_active .. 0 homeless 2024-09-17T16:44:31.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 client.94281.objecter ms_handle_connect 0x7fe76413e340 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 client.94281.objecter resend_mon_ops 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe76000f040 con 0x7fe76413e340 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 monclient: dump: 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.619 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe768cbe640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe74803e580 0x7fe748040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe768cbe640 10 monclient: get_auth_request con 0x7fe74803e580 auth_method 0 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe76001c400 con 0x7fe76413e340 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:31.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 msgr2=0x7fe76402d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 0x7fe76402d450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7600107c0 tx=0x7fe760010c80 comp rx=0 tx=0).stop 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 msgr2=0x7fe76402d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e340 0x7fe76402d450 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7600107c0 tx=0x7fe760010c80 comp rx=0 tx=0).stop 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe759ffb640 10 client.94281.objecter ms_handle_connect 0x7fe74803e580 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.591+0000 7fe768cbe640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe74803e580 0x7fe748040a40 secure :-1 s=READY pgs=3654 cs=0 l=1 rev1=1 crypto rx=0x7fe7640a2400 tx=0x7fe754009190 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 0x7fe76402d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe76413efc0 con 0x7fe76413e720 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 0x7fe76402d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient: get_auth_request con 0x7fe76413e720 auth_method 0 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 10 monclient(hunting): handle_auth_done global_id 74830 payload 824 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 1 -- 172.21.15.105:0/1482583096 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe7641229c0 con 0x7fe76413e720 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 client.94281.objecter ms_handle_connect 0x7fe76413e720 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 client.94281.objecter resend_mon_ops 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe7694bf640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 0x7fe76402d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe76001b000 tx=0x7fe7600046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe76001cda0 con 0x7fe76413e720 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 monclient: dump: 2024-09-17T16:44:31.622 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.623 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9589 (secure 0 0 0) 0x7fe760025070 con 0x7fe76413e720 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 msgr2=0x7fe76402d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 0x7fe76402d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe76001b000 tx=0x7fe7600046d0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 msgr2=0x7fe76402d990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe759ffb640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe76413e720 0x7fe76402d990 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe76001b000 tx=0x7fe7600046d0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 311061, 2024-09-17T16:44:31.625 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.626 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.627 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:31.628 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.629 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.630 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:31.631 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:31.632 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.634 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.635 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.636 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:31.637 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.638 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.640 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.642 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.643 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:31.644 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:31.645 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.646 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.647 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:31.648 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 librados: watch_flush enter 2024-09-17T16:44:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 librados: watch_flush exit 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 20 client.94281.objecter shutdown clearing up homeless session... 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 client.94281.objecter successfully canceled tick 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe74803e580 msgr2=0x7fe748040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe74803e580 0x7fe748040a40 secure :-1 s=READY pgs=3654 cs=0 l=1 rev1=1 crypto rx=0x7fe7640a2400 tx=0x7fe754009190 comp rx=0 tx=0).stop 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 10 monclient: shutdown 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 msgr2=0x7fe76413d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 --2- 172.21.15.105:0/1482583096 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe7641397f0 0x7fe76413d640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0af040 tx=0x7fe75c019660 comp rx=0 tx=0).stop 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 reap_dead start 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe769cc0640 1 -- 172.21.15.105:0/1482583096 reap_dead start 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 shutdown_connections 2024-09-17T16:44:31.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 >> 172.21.15.105:0/1482583096 conn(0x7fe7640a8d00 msgr2=0x7fe76413fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 shutdown_connections 2024-09-17T16:44:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.599+0000 7fe76b723640 1 -- 172.21.15.105:0/1482583096 wait complete. 2024-09-17T16:44:31.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.603+0000 7fe76b723640 1 librados: shutdown 2024-09-17T16:44:31.651 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:31.651 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-17T16:44:31.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:44:31.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: get_monmap_and_config 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: monmap: 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:31.764619+0000 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:31.764619+0000 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:31.769 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:31.770 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 Processor -- start 2024-09-17T16:44:31.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 -- start start 2024-09-17T16:44:31.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: init 2024-09-17T16:44:31.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient: _add_conns ranks=[9,1,2,3,7,8,10,6,5,0,4] 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 0x7fa4500a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-a con 0x7fa4500a3720 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa4500a3b00 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-b con 0x7fa4500a3f00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45006db50 con 0x7fa4500a3b00 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa4500a0e80 con 0x7fa4500a3720 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa4500a1490 con 0x7fa4500a3f00 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa455e34640 10 monclient(hunting): authenticate will time out at 1955.030641s 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44f7fe640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 msgr2=0x7fa4500a1e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 0x7fa4500a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa447fff640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7fa4500a3b00 legacy=0x7fa4500a56c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38090/0 (socket says 172.21.15.105:38090) 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 -- 172.21.15.105:0/1542122695 learned_addr learned my addr 172.21.15.105:0/1542122695 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): get_auth_request con 0x7fa4500a3f00 auth_method 0 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient(hunting): handle_auth_done global_id 84548 payload 274 2024-09-17T16:44:31.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 -- 172.21.15.105:0/1542122695 >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 msgr2=0x7fa4500a1e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 --2- 172.21.15.105:0/1542122695 >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 0x7fa4500a1e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 -- 172.21.15.105:0/1542122695 >> v1:172.21.15.160:6789/0 conn(0x7fa4500a3b00 legacy=0x7fa4500a56c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 monclient: found mon.noname-b 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:44:31.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.763+0000 7fa44ffff640 1 -- 172.21.15.105:0/1542122695 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4500a7240 con 0x7fa4500a3f00 2024-09-17T16:44:31.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44ffff640 1 --2- 172.21.15.105:0/1542122695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa448009e50 tx=0x7fa44801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7955b7a7f05d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 1 -- 172.21.15.105:0/1542122695 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4480090d0 con 0x7fa4500a3f00 2024-09-17T16:44:31.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: dump: 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.782 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 1 -- 172.21.15.105:0/1542122695 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4480a5b60 con 0x7fa4500a3f00 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 5 monclient: authenticate success, global_id 84548 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 4 set_mon_vals no callback set 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 1 -- 172.21.15.105:0/1542122695 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4480a5d50 con 0x7fa4500a3f00 2024-09-17T16:44:31.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa44effd640 10 monclient: dump: 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.784 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: shutdown 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- 172.21.15.105:0/1542122695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 msgr2=0x7fa4500a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- 172.21.15.105:0/1542122695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa448009e50 tx=0x7fa44801c9a0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- 172.21.15.105:0/1542122695 shutdown_connections 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- 172.21.15.105:0/1542122695 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4500a3f00 0x7fa4500a5dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- 172.21.15.105:0/1542122695 >> v2:172.21.15.105:3300/0 conn(0x7fa4500a3720 0x7fa4500a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- 172.21.15.105:0/1542122695 >> 172.21.15.105:0/1542122695 conn(0x7fa4500aa390 msgr2=0x7fa4500a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- 172.21.15.105:0/1542122695 shutdown_connections 2024-09-17T16:44:31.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- 172.21.15.105:0/1542122695 wait complete. 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: monmap: 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.786 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 librados: starting msgr at 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 librados: starting objecter 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 Processor -- start 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- start start 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 librados: setting wanted keys 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 librados: calling monclient init 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: init 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient: _add_conns ranks=[5,3,1,2,4,8,0,6,7] 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4500a3720 0x7fa4501d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-h con 0x7fa4500a3720 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 0x7fa4501d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-g con 0x7fa4500a3b00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4500a3f00 0x7fa45013e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient(hunting): picked mon.noname-f con 0x7fa4500a3f00 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501442e0 con 0x7fa4500a3f00 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.767+0000 7fa455e34640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fa4500a6520 con 0x7fa4500a3b00 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fa450144460 con 0x7fa4500a3720 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient(hunting): authenticate will time out at 1955.036017s 2024-09-17T16:44:31.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44ffff640 1 -- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4500a3f00 msgr2=0x7fa45013e3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4500a3720 msgr2=0x7fa4501d1630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4500a3f00 0x7fa45013e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4500a3720 0x7fa4501d1630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 0x7fa4501d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 0x7fa4501d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:47264/0 (socket says 172.21.15.105:47264) 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 -- 172.21.15.105:0/1139263064 learned_addr learned my addr 172.21.15.105:0/1139263064 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): get_auth_request con 0x7fa4500a3b00 auth_method 0 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient(hunting): handle_auth_done global_id 54966 payload 824 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4500a3720 msgr2=0x7fa4501d1630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fa4500a3720 0x7fa4501d1630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4500a3f00 msgr2=0x7fa45013e3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4500a3f00 0x7fa45013e3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 monclient: found mon.noname-g 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa450125d80 con 0x7fa4500a3b00 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa447fff640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 0x7fa4501d1b70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b510 tx=0x7fa43800b9d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 client.?.objecter ms_handle_connect 0x7fa4500a3b00 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:31.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4380092f0 con 0x7fa4500a3b00 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: dump: 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.791 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438015070 con 0x7fa4500a3b00 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 5 monclient: authenticate success, global_id 54966 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient: _renew_subs 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4500a7b60 con 0x7fa4500a3b00 2024-09-17T16:44:31.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa438005540 con 0x7fa4500a3b00 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: dump: 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.793 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 client.54966.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient: _renew_subs 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45013fde0 con 0x7fa4500a3b00 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 librados: init done 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44e7fc640 4 set_mon_vals no callback set 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 0x7fa45013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa450051e80 con 0x7fa45013e920 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 0x7fa45013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient: get_auth_request con 0x7fa45013e920 auth_method 0 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_done global_id 94290 payload 824 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4500a1e70 con 0x7fa45013e920 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 client.54966.objecter ms_handle_connect 0x7fa45013e920 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 client.54966.objecter resend_mon_ops 2024-09-17T16:44:31.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44f7fe640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 0x7fa45013ed00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa44000b910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fa4380050b0 con 0x7fa4500a3b00 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa43003e580 0x7fa430040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa440003bd0 con 0x7fa45013e920 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: dump: 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.796 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(647..647 src has 1..647) ==== 6769+0+0 (secure 0 0 0) 0x7fa43804ffa0 con 0x7fa4500a3b00 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 client.54966.objecter ms_dispatch 0x7fa450000ba0 osd_map(647..647 src has 1..647) 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 3 client.54966.objecter handle_osd_map got epochs [647,647] > 0 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 3 client.54966.objecter handle_osd_map decoding full epoch 647 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 20 client.54966.objecter dump_active .. 0 homeless 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa440015070 con 0x7fa45013e920 2024-09-17T16:44:31.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 msgr2=0x7fa45013ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 0x7fa45013ed00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa44000b910 comp rx=0 tx=0).stop 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 msgr2=0x7fa45013ed00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44cff9640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013e920 0x7fa45013ed00 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa44000b910 comp rx=0 tx=0).stop 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44ffff640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa43003e580 0x7fa430040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.771+0000 7fa44ffff640 10 monclient: get_auth_request con 0x7fa43003e580 auth_method 0 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.775+0000 7fa44ffff640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa43003e580 0x7fa430040a40 secure :-1 s=READY pgs=3655 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ae000 tx=0x7fa4480a7070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.775+0000 7fa44cff9640 10 client.54966.objecter ms_handle_connect 0x7fa43003e580 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 0x7fa450071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fa45006c860 con 0x7fa45013f240 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 0x7fa450071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient: get_auth_request con 0x7fa45013f240 auth_method 0 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 10 monclient(hunting): handle_auth_done global_id 94299 payload 824 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 1 -- 172.21.15.105:0/1139263064 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa4501d20b0 con 0x7fa45013f240 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 client.54966.objecter ms_handle_connect 0x7fa45013f240 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44f7fe640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 0x7fa450071090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa4400059e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 client.54966.objecter resend_mon_ops 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4400057b0 con 0x7fa45013f240 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 monclient: dump: 2024-09-17T16:44:31.801 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.802 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9254 (secure 0 0 0) 0x7fa440005340 con 0x7fa45013f240 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 msgr2=0x7fa450071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 0x7fa450071090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa4400059e0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 msgr2=0x7fa450071090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44cff9640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fa45013f240 0x7fa450071090 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b450 tx=0x7fa4400059e0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:31.803 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 47623, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:31.804 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:31.805 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:31.806 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:31.807 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:31.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.809 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.810 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.811 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.812 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.813 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:31.815 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.816 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:31.817 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.818 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.819 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.820 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.821 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.822 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.823 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:31.824 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:31.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 librados: watch_flush enter 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 librados: watch_flush exit 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 20 client.54966.objecter shutdown clearing up homeless session... 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 client.54966.objecter successfully canceled tick 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa43003e580 msgr2=0x7fa430040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fa43003e580 0x7fa430040a40 secure :-1 s=READY pgs=3655 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ae000 tx=0x7fa4480a7070 comp rx=0 tx=0).stop 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 10 monclient: shutdown 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 msgr2=0x7fa4501d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 --2- 172.21.15.105:0/1139263064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4500a3b00 0x7fa4501d1b70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b510 tx=0x7fa43800b9d0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44ffff640 1 -- 172.21.15.105:0/1139263064 reap_dead start 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa44ffff640 1 -- 172.21.15.105:0/1139263064 reap_dead start 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 shutdown_connections 2024-09-17T16:44:31.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 >> 172.21.15.105:0/1139263064 conn(0x7fa4500aa390 msgr2=0x7fa4500a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 shutdown_connections 2024-09-17T16:44:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 -- 172.21.15.105:0/1139263064 wait complete. 2024-09-17T16:44:31.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.779+0000 7fa455e34640 1 librados: shutdown 2024-09-17T16:44:31.828 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:31.828 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-17T16:44:31.828 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:44:31.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: get_monmap_and_config 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: monmap: 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:31.936656+0000 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:31.936656+0000 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:31.942 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 Processor -- start 2024-09-17T16:44:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 -- start start 2024-09-17T16:44:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: init 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient: _add_conns ranks=[5,3,7,6,1,4,9,0,10,2,8] 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 0x7f0c240a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-h con 0x7f0c240a4750 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c240a5070 0x7f0c24137510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-g con 0x7f0c240a5070 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-i con 0x7f0c24137a50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2406cf10 con 0x7f0c240a5070 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f0c2406c590 con 0x7f0c240a4750 2024-09-17T16:44:31.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f0c240a62d0 con 0x7f0c24137a50 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c2a66e640 10 monclient(hunting): authenticate will time out at 1955.202463s 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c23fff640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 msgr2=0x7f0c240a4b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 0x7f0c240a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:52940/0 (socket says 172.21.15.105:52940) 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2054429590 learned_addr learned my addr 172.21.15.105:0/2054429590 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c240a5070 0x7f0c24137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): get_auth_request con 0x7f0c24137a50 auth_method 0 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): get_auth_request con 0x7f0c240a5070 auth_method 0 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient(hunting): handle_auth_done global_id 64771 payload 274 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 msgr2=0x7f0c240a4b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 0x7f0c240a4b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c240a5070 msgr2=0x7f0c24137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c240a5070 0x7f0c24137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 monclient: found mon.noname-i 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:44:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.935+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2054429590 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c240a09d0 con 0x7f0c24137a50 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009eb0 tx=0x7f0c1c01c8b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=847ebff228dea567 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 1 -- 172.21.15.105:0/2054429590 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c1c0090d0 con 0x7f0c24137a50 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: dump: 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 1 -- 172.21.15.105:0/2054429590 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0a6970 con 0x7f0c24137a50 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 5 monclient: authenticate success, global_id 64771 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 1 -- 172.21.15.105:0/2054429590 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c1c0af480 con 0x7f0c24137a50 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 4 set_mon_vals no callback set 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c22ffd640 10 monclient: dump: 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 10 monclient: shutdown 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 msgr2=0x7f0c24137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009eb0 tx=0x7f0c1c01c8b0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2054429590 shutdown_connections 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f0c24137a50 0x7f0c24137e90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c240a5070 0x7f0c24137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2054429590 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f0c240a4750 0x7f0c240a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2054429590 >> 172.21.15.105:0/2054429590 conn(0x7f0c240a94d0 msgr2=0x7f0c240a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2054429590 shutdown_connections 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2054429590 wait complete. 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 10 monclient: build_initial_monmap 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 10 monclient: monmap: 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 librados: starting msgr at 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 librados: starting objecter 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 Processor -- start 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 -- start start 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 librados: setting wanted keys 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 1 librados: calling monclient init 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.939+0000 7f0c2a66e640 10 monclient: init 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _add_conns ranks=[8,0,3,4,6,7,2,5,1] 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 0x7f0c2413c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-e con 0x7f0c240a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c240a5070 0x7f0c241cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-a con 0x7f0c240a5070 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c24137a50 0x7f0c241cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): picked mon.noname-g con 0x7f0c24137a50 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2406c810 con 0x7f0c240a5070 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f0c240a05e0 con 0x7f0c24137a50 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0c2406c990 con 0x7f0c240a4750 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): authenticate will time out at 1955.208228s 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c240a5070 msgr2=0x7f0c241cf890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c240a5070 0x7f0c241cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 0x7f0c2413c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 0x7f0c2413c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:37056/0 (socket says 172.21.15.105:37056) 2024-09-17T16:44:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 -- 172.21.15.105:0/2077264092 learned_addr learned my addr 172.21.15.105:0/2077264092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): get_auth_request con 0x7f0c240a4750 auth_method 0 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c24137a50 0x7f0c241cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 10 monclient(hunting): get_auth_request con 0x7f0c24137a50 auth_method 0 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient(hunting): handle_auth_done global_id 55052 payload 824 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c24137a50 msgr2=0x7f0c241cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0c24137a50 0x7f0c241cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c240a5070 msgr2=0x7f0c241cf890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c240a5070 0x7f0c241cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient: _finish_hunting 0 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 monclient: found mon.noname-e 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c241218c0 con 0x7f0c240a4750 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c23fff640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 0x7f0c2413c320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b560 tx=0x7f0c1800ba20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.?.objecter ms_handle_connect 0x7f0c240a4750 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c18013020 con 0x7f0c240a4750 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: dump: 2024-09-17T16:44:31.964 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.965 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: _finish_auth 0 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 5 monclient: authenticate success, global_id 55052 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _renew_subs 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c24121ba0 con 0x7f0c240a4750 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c18003bd0 con 0x7f0c240a4750 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c18003d80 con 0x7f0c240a4750 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:31.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: dump: 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.967 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 client.55052.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _renew_subs 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c240a6e20 con 0x7f0c240a4750 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 librados: init done 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c227fc640 4 set_mon_vals no callback set 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 0x7f0c2411f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0c241c62b0 con 0x7f0c2411f390 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 0x7f0c2411f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient: get_auth_request con 0x7f0c2411f390 auth_method 0 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f0c18016020 con 0x7f0c240a4750 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 10 monclient(hunting): handle_auth_done global_id 55061 payload 824 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c24137510 con 0x7f0c2411f390 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c1003e580 0x7f0c10040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(647..647 src has 1..647) ==== 6769+0+0 (secure 0 0 0) 0x7f0c1804fb20 con 0x7f0c240a4750 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.55052.objecter ms_dispatch 0x7f0c24000ba0 osd_map(647..647 src has 1..647) 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 3 client.55052.objecter handle_osd_map got epochs [647,647] > 0 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 3 client.55052.objecter handle_osd_map decoding full epoch 647 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c237fe640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 0x7f0c2411f770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c2413ca40 tx=0x7f0c1400ebe0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 20 client.55052.objecter dump_active .. 0 homeless 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.55052.objecter ms_handle_connect 0x7f0c2411f390 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.55052.objecter resend_mon_ops 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c14017070 con 0x7f0c2411f390 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: dump: 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.970 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.971 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c1003e580 0x7f0c10040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 10 monclient: get_auth_request con 0x7f0c1003e580 auth_method 0 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0c1400ee30 con 0x7f0c2411f390 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 msgr2=0x7f0c2411f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 0x7f0c2411f770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c2413ca40 tx=0x7f0c1400ebe0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 msgr2=0x7f0c2411f770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411f390 0x7f0c2411f770 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c2413ca40 tx=0x7f0c1400ebe0 comp rx=0 tx=0).stop 2024-09-17T16:44:31.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c20ff9640 10 client.55052.objecter ms_handle_connect 0x7f0c1003e580 2024-09-17T16:44:31.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.943+0000 7f0c28c0b640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c1003e580 0x7f0c10040a40 secure :-1 s=READY pgs=3656 cs=0 l=1 rev1=1 crypto rx=0x7f0c2409ffb0 tx=0x7f0c1c0a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 0x7f0c24138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f0c24051e80 con 0x7f0c2411fcb0 2024-09-17T16:44:31.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 0x7f0c24138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient: get_auth_request con 0x7f0c2411fcb0 auth_method 0 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 10 monclient(hunting): handle_auth_done global_id 55070 payload 824 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 1 -- 172.21.15.105:0/2077264092 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0c240008e0 con 0x7f0c2411fcb0 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c237fe640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 0x7f0c24138730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c14018000 tx=0x7f0c14004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 client.55052.objecter ms_handle_connect 0x7f0c2411fcb0 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 client.55052.objecter resend_mon_ops 2024-09-17T16:44:31.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c14017030 con 0x7f0c2411fcb0 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 monclient: dump: 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:31.977 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9424 (secure 0 0 0) 0x7f0c14005c70 con 0x7f0c2411fcb0 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 msgr2=0x7f0c24138730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 0x7f0c24138730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c14018000 tx=0x7f0c14004740 comp rx=0 tx=0).stop 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 msgr2=0x7f0c24138730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:31.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c20ff9640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c2411fcb0 0x7f0c24138730 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c14018000 tx=0x7f0c14004740 comp rx=0 tx=0).stop 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 578948, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:31.979 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 1, 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.980 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.981 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.982 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:31.983 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.986 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.987 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.990 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.991 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:31.992 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.993 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.994 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.995 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.996 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:31.997 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.998 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:31.999 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:32.000 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:32.001 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 librados: watch_flush enter 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 librados: watch_flush exit 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 20 client.55052.objecter shutdown clearing up homeless session... 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 client.55052.objecter successfully canceled tick 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c1003e580 msgr2=0x7f0c10040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f0c1003e580 0x7f0c10040a40 secure :-1 s=READY pgs=3656 cs=0 l=1 rev1=1 crypto rx=0x7f0c2409ffb0 tx=0x7f0c1c0a7000 comp rx=0 tx=0).stop 2024-09-17T16:44:32.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 10 monclient: shutdown 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 msgr2=0x7f0c2413c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 --2- 172.21.15.105:0/2077264092 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f0c240a4750 0x7f0c2413c320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b560 tx=0x7f0c1800ba20 comp rx=0 tx=0).stop 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2077264092 reap_dead start 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c28c0b640 1 -- 172.21.15.105:0/2077264092 reap_dead start 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 shutdown_connections 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 >> 172.21.15.105:0/2077264092 conn(0x7f0c240a94d0 msgr2=0x7f0c240a1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 shutdown_connections 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 -- 172.21.15.105:0/2077264092 wait complete. 2024-09-17T16:44:32.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:31.951+0000 7f0c2a66e640 1 librados: shutdown 2024-09-17T16:44:32.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:32.005 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-09-17T16:44:32.005 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: get_monmap_and_config 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: build_initial_monmap 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: monmap: 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:32.115996+0000 2024-09-17T16:44:32.120 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:32.115996+0000 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:32.121 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 Processor -- start 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 -- start start 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: init 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient: _add_conns ranks=[5,10,9,7,2,8,0,3,6,4,1] 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-h con 0x7efdb80a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-f con 0x7efdb80a4ab0 addr v2:172.21.15.160:3300/0 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-a con 0x7efdb81397f0 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7efdb80a7c40 con 0x7efdb81397f0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7efdb80a7890 con 0x7efdb80a4ab0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7efdb80a4080 con 0x7efdb80a6a30 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdbdb6a640 10 monclient(hunting): authenticate will time out at 1955.381672s 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb7fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 msgr2=0x7efdb8139ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb6ffd640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 msgr2=0x7efdb80a7350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb77fe640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 msgr2=0x7efdb80a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.115+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb7fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 msgr2=0x7efdb8139ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb6ffd640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 msgr2=0x7efdb80a7350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb77fe640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 msgr2=0x7efdb80a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:32.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.319+0000 7efdb6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb7fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 msgr2=0x7efdb8139ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb77fe640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 msgr2=0x7efdb80a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb6ffd640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 msgr2=0x7efdb80a7350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:32.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:32.723+0000 7efdb6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:44:33.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb7fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 msgr2=0x7efdb8139ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:33.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb6ffd640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 msgr2=0x7efdb80a7350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3300/0 2024-09-17T16:44:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb77fe640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 msgr2=0x7efdb80a6e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.160:3302/0 2024-09-17T16:44:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:33.527+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): tick 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 monclient(hunting): continuing hunt 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 msgr2=0x7efdb80a6e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7efdb80a6a30 0x7efdb80a6e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 msgr2=0x7efdb80a7350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efdb80a4ab0 0x7efdb80a7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 msgr2=0x7efdb8139ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efdb81397f0 0x7efdb8139ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient: _add_conns ranks=[4,1,0,8,2,6,7,3] 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): picked mon.noname-c con 0x7efda4003820 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efda4006270 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efda4009fa0 addr v1:172.21.15.105:6789/0 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb80a37e0 con 0x7efda4009fa0 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda400ed50 con 0x7efda4006270 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7efdb80a7c40 con 0x7efda4003820 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb57fa640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb7fff640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7efda4009fa0 legacy=0x7efda400c460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb6ffd640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7efda4006270 legacy=0x7efda4008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52604/0 (socket says 172.21.15.105:52604) 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 -- 172.21.15.105:0/109426790 learned_addr learned my addr 172.21.15.105:0/109426790 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): get_auth_request con 0x7efda4003820 auth_method 0 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient(hunting): handle_auth_done global_id 74839 payload 274 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 -- 172.21.15.105:0/109426790 >> v1:172.21.15.160:6789/0 conn(0x7efda4006270 legacy=0x7efda4008700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 -- 172.21.15.105:0/109426790 >> v1:172.21.15.105:6789/0 conn(0x7efda4009fa0 legacy=0x7efda400c460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 monclient: found mon.noname-c 2024-09-17T16:44:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb7fff640 1 -- 172.21.15.105:0/109426790 reap_dead start 2024-09-17T16:44:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.115+0000 7efdb77fe640 1 -- 172.21.15.105:0/109426790 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40100a0 con 0x7efda4003820 2024-09-17T16:44:35.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb77fe640 1 --2- 172.21.15.105:0/109426790 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efdac009df0 tx=0x7efdac01c800 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6c637a253c43f51e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 1 -- 172.21.15.105:0/109426790 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdac0a55c0 con 0x7efda4003820 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: dump: 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.129 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:35.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 1 -- 172.21.15.105:0/109426790 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac0a9030 con 0x7efda4003820 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 1 -- 172.21.15.105:0/109426790 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdac0a5c10 con 0x7efda4003820 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 5 monclient: authenticate success, global_id 74839 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 4 set_mon_vals no callback set 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdb67fc640 10 monclient: dump: 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.131 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.132 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 10 monclient: shutdown 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 -- 172.21.15.105:0/109426790 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 msgr2=0x7efda4005c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/109426790 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efdac009df0 tx=0x7efdac01c800 comp rx=0 tx=0).stop 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 -- 172.21.15.105:0/109426790 shutdown_connections 2024-09-17T16:44:35.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/109426790 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efda4005c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 -- 172.21.15.105:0/109426790 >> 172.21.15.105:0/109426790 conn(0x7efdb80a8d00 msgr2=0x7efdb80a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 -- 172.21.15.105:0/109426790 shutdown_connections 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.123+0000 7efdbdb6a640 1 -- 172.21.15.105:0/109426790 wait complete. 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: build_initial_monmap 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: monmap: 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:35.139 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 librados: starting msgr at 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 librados: starting objecter 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 Processor -- start 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 -- start start 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 librados: setting wanted keys 2024-09-17T16:44:35.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 librados: calling monclient init 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: init 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: _add_conns ranks=[4,0,6,7,1,8,3,5,2] 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efdb8138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-c con 0x7efda4003820 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8138c60 0x7efdb8122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-a con 0x7efdb8138c60 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdb8122bc0 0x7efdb8123020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): picked mon.noname-d con 0x7efdb8122bc0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efdb806e890 con 0x7efdb8138c60 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7efdb806e710 con 0x7efda4003820 2024-09-17T16:44:35.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb6ffd640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8138c60 msgr2=0x7efdb8122680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8138c60 0x7efdb8122680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efdb8139ca0 con 0x7efdb8122bc0 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient(hunting): authenticate will time out at 1958.394763s 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efdb8138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efdb8138720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:52612/0 (socket says 172.21.15.105:52612) 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 -- 172.21.15.105:0/341692391 learned_addr learned my addr 172.21.15.105:0/341692391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): get_auth_request con 0x7efda4003820 auth_method 0 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdb8122bc0 0x7efdb8123020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): get_auth_request con 0x7efdb8122bc0 auth_method 0 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient(hunting): handle_auth_done global_id 74848 payload 824 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdb8122bc0 msgr2=0x7efdb8123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efdb8122bc0 0x7efdb8123020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8138c60 msgr2=0x7efdb8122680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdb8138c60 0x7efdb8122680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 monclient: found mon.noname-c 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb8125e00 con 0x7efda4003820 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb77fe640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efdb8138720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdac01cce0 tx=0x7efdac0195a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 client.?.objecter ms_handle_connect 0x7efda4003820 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdac0a5bf0 con 0x7efda4003820 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: dump: 2024-09-17T16:44:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: _finish_auth 0 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 5 monclient: authenticate success, global_id 74848 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: _renew_subs 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac0a5d90 con 0x7efda4003820 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb80a2290 con 0x7efda4003820 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdac005550 con 0x7efda4003820 2024-09-17T16:44:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdbdb6a640 10 client.74848.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efdb5ffb640 4 set_mon_vals no callback set 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.127+0000 7efd9ffff640 10 monclient: dump: 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.147 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 10 monclient: _renew_subs 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb8139080 con 0x7efda4003820 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 1 librados: init done 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 0x7efdb809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efdb8123e10 con 0x7efdb8123560 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efdac0056f0 con 0x7efda4003820 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efda403fa50 0x7efda4041f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(650..650 src has 1..650) ==== 6795+0+0 (secure 0 0 0) 0x7efdac0b5030 con 0x7efda4003820 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 client.74848.objecter ms_dispatch 0x7efdb8000ba0 osd_map(650..650 src has 1..650) 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 3 client.74848.objecter handle_osd_map got epochs [650,650] > 0 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 3 client.74848.objecter handle_osd_map decoding full epoch 650 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 0x7efdb809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 20 client.74848.objecter dump_active .. 0 homeless 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient: get_auth_request con 0x7efdb8123560 auth_method 0 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb7fff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efda403fa50 0x7efda4041f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb7fff640 10 monclient: get_auth_request con 0x7efda403fa50 auth_method 0 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_done global_id 54993 payload 824 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdb8124ad0 con 0x7efdb8123560 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb6ffd640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 0x7efdb809da00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdb80a7e50 tx=0x7efda800b870 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 client.74848.objecter ms_handle_connect 0x7efdb8123560 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 client.74848.objecter resend_mon_ops 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 client.74848.objecter ms_handle_connect 0x7efda403fa50 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efdb7fff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efda403fa50 0x7efda4041f10 secure :-1 s=READY pgs=3657 cs=0 l=1 rev1=1 crypto rx=0x7efdb80a82d0 tx=0x7efdb0007290 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda8016070 con 0x7efdb8123560 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 monclient: dump: 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.151 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.152 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.152 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efda8003d90 con 0x7efdb8123560 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 msgr2=0x7efdb809da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 0x7efdb809da00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdb80a7e50 tx=0x7efda800b870 comp rx=0 tx=0).stop 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 msgr2=0x7efdb809da00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.131+0000 7efd9ffff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb8123560 0x7efdb809da00 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efdb80a7e50 tx=0x7efda800b870 comp rx=0 tx=0).stop 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdbdb6a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 0x7efdb809e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdbdb6a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efdb8051e80 con 0x7efdb809df40 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 0x7efdb809e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient: get_auth_request con 0x7efdb809df40 auth_method 0 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 10 monclient(hunting): handle_auth_done global_id 55002 payload 824 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 1 -- 172.21.15.105:0/341692391 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efdb8077310 con 0x7efdb809df40 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efdb6ffd640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 0x7efdb809e350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda8000eb0 tx=0x7efda8000bd0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 10 client.74848.objecter ms_handle_connect 0x7efdb809df40 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 10 client.74848.objecter resend_mon_ops 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efda8016070 con 0x7efdb809df40 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.139+0000 7efd9ffff640 10 monclient: dump: 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.156 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7efda8017040 con 0x7efdb809df40 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 msgr2=0x7efdb809e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 0x7efdb809e350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda8000eb0 tx=0x7efda8000bd0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 msgr2=0x7efdb809e350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efd9ffff640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdb809df40 0x7efdb809e350 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda8000eb0 tx=0x7efda8000bd0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.157 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 582145, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.158 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.159 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.160 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.162 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.164 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:35.166 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.167 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.168 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.169 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:35.170 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.171 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:35.173 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.174 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.176 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.177 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:35.178 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:35.179 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:35.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 10 librados: watch_flush enter 2024-09-17T16:44:35.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 10 librados: watch_flush exit 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 20 client.74848.objecter shutdown clearing up homeless session... 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 10 client.74848.objecter successfully canceled tick 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efda403fa50 msgr2=0x7efda4041f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efda403fa50 0x7efda4041f10 secure :-1 s=READY pgs=3657 cs=0 l=1 rev1=1 crypto rx=0x7efdb80a82d0 tx=0x7efdb0007290 comp rx=0 tx=0).stop 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 10 monclient: shutdown 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 msgr2=0x7efdb8138720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 --2- 172.21.15.105:0/341692391 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efda4003820 0x7efdb8138720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdac01cce0 tx=0x7efdac0195a0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdb7fff640 1 -- 172.21.15.105:0/341692391 reap_dead start 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdb7fff640 1 -- 172.21.15.105:0/341692391 reap_dead start 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 shutdown_connections 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 >> 172.21.15.105:0/341692391 conn(0x7efdb80a8d00 msgr2=0x7efdb80a24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 shutdown_connections 2024-09-17T16:44:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 -- 172.21.15.105:0/341692391 wait complete. 2024-09-17T16:44:35.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.143+0000 7efdbdb6a640 1 librados: shutdown 2024-09-17T16:44:35.183 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:35.183 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-17T16:44:35.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:44:35.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: get_monmap_and_config 2024-09-17T16:44:35.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: build_initial_monmap 2024-09-17T16:44:35.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: monmap: 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:44:35.300439+0000 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:44:35.300439+0000 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:35.305 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:35.306 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:44:35.306 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:44:35.306 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 Processor -- start 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 -- start start 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: init 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient: _add_conns ranks=[8,9,1,5,4,0,10,2,3,7,6] 2024-09-17T16:44:35.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-e con 0x7f4b4c0a1b60 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 0x7f4b4c0a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-a con 0x7f4b4c09f670 addr v2:172.21.15.105:3300/0 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4b4c0a30a0 addr v1:172.21.15.160:6789/0 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c09fef0 con 0x7f4b4c0a30a0 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c0a01d0 con 0x7f4b4c09f670 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4b4c0a4070 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b51bd3640 10 monclient(hunting): authenticate will time out at 1958.566168s 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4affd640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 msgr2=0x7f4b4c0a2b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 0x7f4b4c0a2b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4bfff640 1 -- >> v1:172.21.15.160:6789/0 conn(0x7f4b4c0a30a0 legacy=0x7f4b4c13c420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.160:6789/0 2024-09-17T16:44:35.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:37090/0 (socket says 172.21.15.105:37090) 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/536203377 learned_addr learned my addr 172.21.15.105:0/536203377 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): get_auth_request con 0x7f4b4c0a1b60 auth_method 0 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_done global_id 55079 payload 274 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/536203377 >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 msgr2=0x7f4b4c0a2b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/536203377 >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 0x7f4b4c0a2b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/536203377 >> v1:172.21.15.160:6789/0 conn(0x7f4b4c0a30a0 legacy=0x7f4b4c13c420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:44:35.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:44:35.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 monclient: found mon.noname-e 2024-09-17T16:44:35.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:35.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:35.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.299+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/536203377 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c0a12f0 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/536203377 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e80 tx=0x7f4b4401c8f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a69e2237953b02a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 1 -- 172.21.15.105:0/536203377 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b440090d0 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: dump: 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 1 -- 172.21.15.105:0/536203377 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a6930 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 5 monclient: authenticate success, global_id 55079 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: get_monmap_and_config success 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 1 -- 172.21.15.105:0/536203377 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b440af480 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 4 set_mon_vals no callback set 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b4a7fc640 10 monclient: dump: 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: shutdown 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- 172.21.15.105:0/536203377 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 msgr2=0x7f4b4c0a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/536203377 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e80 tx=0x7f4b4401c8f0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- 172.21.15.105:0/536203377 shutdown_connections 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/536203377 >> v2:172.21.15.105:3300/0 conn(0x7f4b4c09f670 0x7f4b4c0a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/536203377 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c0a2620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- 172.21.15.105:0/536203377 >> 172.21.15.105:0/536203377 conn(0x7f4b4c068ba0 msgr2=0x7f4b4c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- 172.21.15.105:0/536203377 shutdown_connections 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- 172.21.15.105:0/536203377 wait complete. 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: build_initial_monmap 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: monmap: 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:44:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 librados: starting msgr at 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 librados: starting objecter 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 Processor -- start 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- start start 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 librados: setting wanted keys 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 librados: calling monclient init 2024-09-17T16:44:35.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: init 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: _reopen_session rank -1 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient: _add_conns ranks=[3,8,2,5,0,7,4,6,1] 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b4c09f670 0x7f4b4c126240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-g con 0x7f4b4c09f670 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c1cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-e con 0x7f4b4c0a1b60 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b4c0a30a0 0x7f4b4c13d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): picked mon.noname-b con 0x7f4b4c0a30a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c13ded0 con 0x7f4b4c0a30a0 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c13dd50 con 0x7f4b4c09f670 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f4b4c13e050 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): _renew_subs 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.303+0000 7f4b51bd3640 10 monclient(hunting): authenticate will time out at 1958.571938s 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c1cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c1cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:37098/0 (socket says 172.21.15.105:37098) 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 -- 172.21.15.105:0/1325600169 learned_addr learned my addr 172.21.15.105:0/1325600169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): get_auth_request con 0x7f4b4c0a1b60 auth_method 0 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b4c0a30a0 0x7f4b4c13d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): get_auth_request con 0x7f4b4c0a30a0 auth_method 0 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient(hunting): handle_auth_done global_id 55088 payload 824 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b4c09f670 msgr2=0x7f4b4c126240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b4c09f670 0x7f4b4c126240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b4c09f670 0x7f4b4c126240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b4c0a30a0 msgr2=0x7f4b4c13d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b4c0a30a0 0x7f4b4c13d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient: _finish_hunting 0 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 monclient: found mon.noname-e 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c1250f0 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b4c0a30a0 0x7f4b4c13d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4affd640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c1cbf00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400b450 tx=0x7f4b3400b910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.?.objecter ms_handle_connect 0x7f4b4c0a1b60 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b34016070 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:44:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: dump: 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: _finish_auth 0 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: _check_auth_tickets 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 5 monclient: authenticate success, global_id 55088 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34003ba0 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 monclient: _renew_subs 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c075160 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b34003d50 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:44:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b49ffb640 4 set_mon_vals no callback set 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 client.55088.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: dump: 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 monclient: _renew_subs 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1256e0 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 1 librados: init done 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 0x7f4b4c1cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f4b4c051e80 con 0x7f4b4c13d5d0 2024-09-17T16:44:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f4b34005070 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4b2c03e580 0x7f4b2c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 0x7f4b4c1cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(651..651 src has 1..651) ==== 6795+0+0 (secure 0 0 0) 0x7f4b3404f200 con 0x7f4b4c0a1b60 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.55088.objecter ms_dispatch 0x7f4b4c000ba0 osd_map(651..651 src has 1..651) 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 3 client.55088.objecter handle_osd_map got epochs [651,651] > 0 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 3 client.55088.objecter handle_osd_map decoding full epoch 651 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient: get_auth_request con 0x7f4b4c13d5d0 auth_method 0 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 20 client.55088.objecter dump_active .. 0 homeless 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4b2c03e580 0x7f4b2c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 10 monclient: get_auth_request con 0x7f4b2c03e580 auth_method 0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_done global_id 64780 payload 824 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b4c13e3d0 con 0x7f4b4c13d5d0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.55088.objecter ms_handle_connect 0x7f4b2c03e580 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4bfff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4b2c03e580 0x7f4b2c040a40 secure :-1 s=READY pgs=3658 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c003940 tx=0x7f4b3c006f90 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.55088.objecter ms_handle_connect 0x7f4b4c13d5d0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 client.55088.objecter resend_mon_ops 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 0x7f4b4c1cef60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b44019690 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b44009220 con 0x7f4b4c13d5d0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.307+0000 7f4b43fff640 10 monclient: dump: 2024-09-17T16:44:35.334 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.335 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b44009220 con 0x7f4b4c13d5d0 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 msgr2=0x7f4b4c1cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 0x7f4b4c1cef60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b44019690 comp rx=0 tx=0).stop 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 msgr2=0x7f4b4c1cef60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.311+0000 7f4b43fff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c13d5d0 0x7f4b4c1cef60 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b44019690 comp rx=0 tx=0).stop 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b51bd3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 0x7f4b4c1cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b51bd3640 10 monclient(hunting): start opening mon connection 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f4b4c1cfdf0 con 0x7f4b4c1cf4a0 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 0x7f4b4c1cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient: get_auth_request con 0x7f4b4c1cf4a0 auth_method 0 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:44:35.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 10 monclient(hunting): handle_auth_done global_id 64789 payload 824 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 1 -- 172.21.15.105:0/1325600169 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b4c127560 con 0x7f4b4c1cf4a0 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b4b7fe640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 0x7f4b4c1cf8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e80 tx=0x7f4b440a9000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b43fff640 10 client.55088.objecter ms_handle_connect 0x7f4b4c1cf4a0 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.315+0000 7f4b43fff640 10 client.55088.objecter resend_mon_ops 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b440c02a0 con 0x7f4b4c1cf4a0 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 10 monclient: dump: 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.339 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7f4b440c0040 con 0x7f4b4c1cf4a0 2024-09-17T16:44:35.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 msgr2=0x7f4b4c1cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 0x7f4b4c1cf8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e80 tx=0x7f4b440a9000 comp rx=0 tx=0).stop 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 msgr2=0x7f4b4c1cf8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b43fff640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f4b4c1cf4a0 0x7f4b4c1cf8b0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e80 tx=0x7f4b440a9000 comp rx=0 tx=0).stop 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 110, 2024-09-17T16:44:35.341 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 448418, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:44:35.342 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.343 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:44:35.344 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:44:35.345 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.346 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:44:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.349 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:44:35.351 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.354 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.355 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.358 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:44:35.359 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:44:35.360 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:44:35.361 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:44:35.362 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 10 librados: watch_flush enter 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 10 librados: watch_flush exit 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 20 client.55088.objecter shutdown clearing up homeless session... 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 10 client.55088.objecter successfully canceled tick 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4b2c03e580 msgr2=0x7f4b2c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f4b2c03e580 0x7f4b2c040a40 secure :-1 s=READY pgs=3658 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c003940 tx=0x7f4b3c006f90 comp rx=0 tx=0).stop 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 10 monclient: shutdown 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 msgr2=0x7f4b4c1cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 --2- 172.21.15.105:0/1325600169 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f4b4c0a1b60 0x7f4b4c1cbf00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400b450 tx=0x7f4b3400b910 comp rx=0 tx=0).stop 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b4bfff640 1 -- 172.21.15.105:0/1325600169 reap_dead start 2024-09-17T16:44:35.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b4bfff640 1 -- 172.21.15.105:0/1325600169 reap_dead start 2024-09-17T16:44:35.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 shutdown_connections 2024-09-17T16:44:35.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 >> 172.21.15.105:0/1325600169 conn(0x7f4b4c068ba0 msgr2=0x7f4b4c0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:44:35.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 shutdown_connections 2024-09-17T16:44:35.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 -- 172.21.15.105:0/1325600169 wait complete. 2024-09-17T16:44:35.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:44:35.319+0000 7f4b51bd3640 1 librados: shutdown 2024-09-17T16:44:35.365 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-17T16:44:35.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-17T16:46:00.259 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7872 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12187 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (7795 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13032 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7975 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (7984 ms) 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-17T16:46:00.260 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: ok, hit_set contains 332:602f83fe:::foo:head 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (20003 ms) 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,0 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first is 1726591476 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,0 2024-09-17T16:46:00.261 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,1726591484,1726591485,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,1726591484,1726591485,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591476,1726591478,1726591479,1726591481,1726591482,1726591484,1726591485,0 2024-09-17T16:46:00.262 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: got ls 1726591479,1726591481,1726591482,1726591484,1726591485,1726591487,1726591488,0 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: first now 1726591479, trimmed 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (46091 ms) 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: foo0 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-17T16:46:00.263 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (13930 ms) 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18136 ms) 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22028 ms) 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-17T16:46:00.264 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (4866 ms) 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3029 ms) 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3127 ms) 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3055 ms) 2024-09-17T16:46:00.265 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (285420 ms total) 2024-09-17T16:46:00.266 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: 2024-09-17T16:46:00.266 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-17T16:46:00.266 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (1019499 ms total) 2024-09-17T16:46:00.266 INFO:tasks.workunit.client.0.smithi160.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-17T16:46:00.268 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29375 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29375 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29557 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29557 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29926 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29926 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29745 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29745 2024-09-17T16:46:00.269 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29618 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29618 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29430 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29430 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29679 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29679 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30055 2024-09-17T16:46:00.270 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30055 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30131 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30131 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29459 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29459 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29941 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29941 2024-09-17T16:46:00.277 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29359 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29359 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29391 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29391 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29879 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29879 2024-09-17T16:46:00.278 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29351 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29351 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29367 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29367 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30261 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30261 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29591 2024-09-17T16:46:00.279 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29591 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30311 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30311 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29658 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29658 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30008 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30008 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30354 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30354 2024-09-17T16:46:00.280 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29607 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29607 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29917 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29917 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30234 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30234 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29992 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29992 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.281 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30278 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30278 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29859 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29859 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29515 2024-09-17T16:46:00.282 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29515 2024-09-17T16:46:00.283 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30337 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30337 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29778 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29778 2024-09-17T16:46:00.284 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29343 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29343 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29383 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29383 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29697 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29697 2024-09-17T16:46:00.285 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29407 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29407 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29799 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29799 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=29849 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 29849 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.286 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30161 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30161 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ for t in "${!pids[@]}" 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ pid=30370 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ wait 30370 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ exit 0 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ cleanup 2024-09-17T16:46:00.287 INFO:tasks.workunit.client.0.smithi160.stderr:+ pkill -P 29337 2024-09-17T16:46:00.288 INFO:tasks.workunit.client.0.smithi160.stderr:+ true 2024-09-17T16:46:00.288 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-17T16:46:00.289 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-17T16:46:00.303 INFO:tasks.workunit:Stopping ['rados/test.sh unit_test_scan'] on client.0... 2024-09-17T16:46:00.303 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-17T16:46:00.651 DEBUG:teuthology.parallel:result is None 2024-09-17T16:46:00.651 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T16:46:00.663 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T16:46:00.663 DEBUG:teuthology.orchestra.run.smithi160:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-17T16:46:00.717 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-17T16:46:00.717 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-17T16:46:00.726 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-17T16:46:05.366 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-17T16:46:05.366 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-17T16:46:05.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-17T16:46:05.369 INFO:tasks.ceph.mon.h:Started 2024-09-17T16:46:05.370 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-17T16:46:05.370 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T16:46:05.370 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T16:46:05.373 INFO:tasks.ceph.mon.a:Started 2024-09-17T16:46:05.373 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-17T16:46:05.373 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-17T16:46:05.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-17T16:46:05.376 INFO:tasks.ceph.mon.f:Started 2024-09-17T16:46:05.376 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-09-17T16:46:05.376 INFO:tasks.ceph.mon.i:Sent signal 19 2024-09-17T16:46:05.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-17T16:46:05.482 INFO:tasks.ceph.mon.h.smithi160.stderr:2024-09-17T16:46:05.480+0000 7f99eefa0e00 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 663 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:46:05.493 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-09-17T16:46:05.486+0000 7ff61f90fe00 -1 mon.a@-1(???).log v0 log_external_backlog rewinding external_log_to from 663 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:46:05.493 INFO:tasks.ceph.mon.f.smithi160.stderr:2024-09-17T16:46:05.492+0000 7f423ca2de00 -1 mon.f@-1(???).log v0 log_external_backlog rewinding external_log_to from 663 -> 0 (sync_force? mon rebuild?) 2024-09-17T16:46:20.377 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-09-17T16:46:20.378 INFO:tasks.ceph.mon.i:Sent signal 18 2024-09-17T16:46:20.378 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:46:20.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:46:20.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: get_monmap_and_config 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: build_initial_monmap 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: monmap: 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:20.504955+0000 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:20.504955+0000 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:20.509 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:20.510 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 Processor -- start 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 -- start start 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: init 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient: _add_conns ranks=[8,4,6,7,10,1,2,9,0,5,3] 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-e con 0x7f1b980a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:20.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-c con 0x7f1b980a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-d con 0x7f1b981396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1b980a3fc0 con 0x7f1b980a4910 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f1b980a63c0 con 0x7f1b981396a0 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1b980a7740 con 0x7f1b980a68e0 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9fbc2640 10 monclient(hunting): authenticate will time out at 2063.770970s 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:38474/0 (socket says 172.21.15.105:38474) 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 learned_addr learned my addr 172.21.15.105:0/172649779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b980a4910 auth_method 0 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b981396a0 auth_method 0 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b980a68e0 auth_method 0 2024-09-17T16:46:20.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.502+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b980a68e0 auth_method 0 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b981396a0 auth_method 0 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b980a4910 auth_method 0 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:20.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:20.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b981396a0 auth_method 0 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b980a68e0 auth_method 0 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b980a4910 auth_method 0 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:21.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b980a68e0 auth_method 0 2024-09-17T16:46:21.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b981396a0 auth_method 0 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b980a4910 auth_method 0 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:21.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:21.906+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): tick 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 msgr2=0x7f1b980a6cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1b980a68e0 0x7f1b980a6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 msgr2=0x7f1b98139ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1b981396a0 0x7f1b98139ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 msgr2=0x7f1b980a7200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1b980a4910 0x7f1b980a7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient: _add_conns ranks=[10,7,5,2,9,3,0,1] 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:23.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-f con 0x7f1b84003840 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-i con 0x7f1b84006270 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-h con 0x7f1b84008d40 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f1b980a3fc0 con 0x7f1b84003840 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1b980a7740 con 0x7f1b84008d40 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f1b8400b930 con 0x7f1b84006270 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b8f7fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b84008d40 auth_method 0 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9e15f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b84006270 auth_method 0 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d15d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b84003840 auth_method 0 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.502+0000 7f1b9d95e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:23.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.506+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:23.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b84006270 auth_method 0 2024-09-17T16:46:23.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b84008d40 auth_method 0 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b84003840 auth_method 0 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:23.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:23.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:23.706+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-17T16:46:24.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b84006270 auth_method 0 2024-09-17T16:46:24.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b84008d40 auth_method 0 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b84003840 auth_method 0 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:24.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:24.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.106+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-17T16:46:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b84006270 auth_method 0 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b84003840 auth_method 0 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b84008d40 auth_method 0 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:24.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:24.910+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): tick 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 monclient(hunting): continuing hunt 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 msgr2=0x7f1b840086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1b84006270 0x7f1b840086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 msgr2=0x7f1b8400b200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84008d40 0x7f1b8400b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 msgr2=0x7f1b84005c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.160:3300/0 conn(0x7f1b84003840 0x7f1b84005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 reap_dead start 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient: _add_conns ranks=[1,2,9,0,3] 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 reap_dead start 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1b840091f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 0x7f1b84005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f1b84007df0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f1b840081d0 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9806ec10 con 0x7f1b840091f0 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:26.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1b980a3fc0 con 0x7f1b840081d0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 1 -- 172.21.15.105:0/172649779 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1b980a7740 con 0x7f1b84007df0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b8f7fe640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.502+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 0x7f1b84005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b84007df0 auth_method 0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b840081d0 auth_method 0 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:26.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 msgr2=0x7f1b840085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 msgr2=0x7f1b840085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_done global_id 84566 payload 274 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 msgr2=0x7f1b840085d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 >> v1:172.21.15.160:6789/0 conn(0x7f1b840091f0 legacy=0x7f1b840066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 1 monclient: found mon.noname-b 2024-09-17T16:46:26.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:26.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:26.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.506+0000 7f1b9d15d640 1 -- 172.21.15.105:0/172649779 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8400f090 con 0x7f1b84007df0 2024-09-17T16:46:26.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 0x7f1b84005c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b98074500 tx=0x7f1b8801dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1567580c1226182 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 1 -- 172.21.15.105:0/172649779 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b880b3070 con 0x7f1b84007df0 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: dump: 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:26.519 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: _finish_auth 0 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: _check_auth_tickets 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 1 -- 172.21.15.105:0/172649779 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b880173e0 con 0x7f1b84007df0 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9fbc2640 5 monclient: authenticate success, global_id 84566 2024-09-17T16:46:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9fbc2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9fbc2640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9fbc2640 4 set_mon_vals no callback set 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 1 -- 172.21.15.105:0/172649779 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b8800f040 con 0x7f1b84007df0 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.510+0000 7f1b9c95c640 10 monclient: dump: 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:26.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:26.523 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: shutdown 2024-09-17T16:46:26.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 msgr2=0x7f1b84005c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 0x7f1b84005c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b98074500 tx=0x7f1b8801dde0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/172649779 shutdown_connections 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/172649779 >> v2:172.21.15.105:3300/0 conn(0x7f1b840081d0 0x7f1b840085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/172649779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b84007df0 0x7f1b84005c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/172649779 >> 172.21.15.105:0/172649779 conn(0x7f1b98068ba0 msgr2=0x7f1b98068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/172649779 shutdown_connections 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/172649779 wait complete. 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: build_initial_monmap 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: monmap: 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:26.529 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 librados: starting msgr at 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 librados: starting objecter 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 Processor -- start 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- start start 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 librados: setting wanted keys 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 librados: calling monclient init 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: init 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient: _add_conns ranks=[5,3,0,2,6,8,1,7,4] 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-h con 0x7f1b84007df0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 0x7f1b98138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-g con 0x7f1b840081d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): picked mon.noname-a con 0x7f1b840091f0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98139c00 con 0x7f1b840091f0 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1b98139a80 con 0x7f1b840081d0 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1b98139d80 con 0x7f1b84007df0 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9fbc2640 10 monclient(hunting): authenticate will time out at 2069.783080s 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54984/0 (socket says 172.21.15.105:54984) 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 -- 172.21.15.105:0/4019396749 learned_addr learned my addr 172.21.15.105:0/4019396749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request con 0x7f1b840091f0 auth_method 0 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 msgr2=0x7f1b98122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 msgr2=0x7f1b98122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:26.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 0x7f1b98138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9e15f640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request con 0x7f1b840081d0 auth_method 0 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request con 0x7f1b84007df0 auth_method 0 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 msgr2=0x7f1b98138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 msgr2=0x7f1b98138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.514+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 10 monclient(hunting): handle_auth_done global_id 55011 payload 824 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 msgr2=0x7f1b98138120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 msgr2=0x7f1b98122530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 monclient: found mon.noname-g 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 -- 172.21.15.105:0/4019396749 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b98069210 con 0x7f1b840081d0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d15d640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 0x7f1b98138660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400f060 tx=0x7f1b8801ba00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 client.?.objecter ms_handle_connect 0x7f1b840081d0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b880b3050 con 0x7f1b840081d0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: dump: 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:26.537 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b880b4040 con 0x7f1b840081d0 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 5 monclient: authenticate success, global_id 55011 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _renew_subs 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b981258d0 con 0x7f1b840081d0 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b8800f040 con 0x7f1b840081d0 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 monclient: dump: 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:26.540 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 client.55011.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _renew_subs 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8f7fe640 4 set_mon_vals no callback set 2024-09-17T16:46:26.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b981255f0 con 0x7f1b840081d0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 1 librados: init done 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9813a0b0 con 0x7f1b840081d0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1b880c1030 con 0x7f1b840081d0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 0x7f1b84042e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 5 ==== osd_map(707..707 src has 501..707) ==== 5529+0+0 (secure 0 0 0) 0x7f1b880d6030 con 0x7f1b840081d0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 10 client.55011.objecter ms_dispatch 0x7f1b98000ba0 osd_map(707..707 src has 501..707) 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 3 client.55011.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 3 client.55011.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:26.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b8e7fc640 20 client.55011.objecter dump_active .. 0 homeless 2024-09-17T16:46:26.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 0x7f1b84042e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:26.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.518+0000 7f1b9d95e640 10 monclient: get_auth_request con 0x7f1b84040990 auth_method 0 2024-09-17T16:46:26.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.522+0000 7f1b8e7fc640 10 client.55011.objecter ms_handle_connect 0x7f1b84040990 2024-09-17T16:46:26.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.522+0000 7f1b9d95e640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 0x7f1b84042e50 secure :-1 s=READY pgs=3662 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c10 tx=0x7f1b94008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:26.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.534+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1b880c8020 con 0x7f1b840081d0 2024-09-17T16:46:26.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.534+0000 7f1b8e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:26.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.534+0000 7f1b8e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b9fbc2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:46:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b9fbc2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b9fbc2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.160:3301/0 2024-09-17T16:46:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1b980746d0 con 0x7f1b840081d0 2024-09-17T16:46:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b8e7fc640 1 -- 172.21.15.105:0/4019396749 <== mon.3 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3201 (secure 0 0 0) 0x7f1b880098e0 con 0x7f1b840081d0 2024-09-17T16:46:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b8e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:26.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.766+0000 7f1b8e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:26.774 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":116,"quorum":[1,2,3,4,6,7,8],"quorum_names":["f","b","g","c","d","i","e"],"quorum_leader_name":"e","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:46:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 10 librados: watch_flush enter 2024-09-17T16:46:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 10 librados: watch_flush exit 2024-09-17T16:46:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 20 client.55011.objecter shutdown clearing up homeless session... 2024-09-17T16:46:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 10 client.55011.objecter successfully canceled tick 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 msgr2=0x7f1b84042e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 0x7f1b84042e50 secure :-1 s=READY pgs=3662 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c10 tx=0x7f1b94008040 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 10 monclient: shutdown 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 msgr2=0x7f1b98138660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 0x7f1b98138660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400f060 tx=0x7f1b8801ba00 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 shutdown_connections 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1b84040990 0x7f1b84042e50 unknown :-1 s=CLOSED pgs=3662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b840091f0 0x7f1b98122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b840081d0 0x7f1b98138660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 --2- 172.21.15.105:0/4019396749 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1b84007df0 0x7f1b98138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 >> 172.21.15.105:0/4019396749 conn(0x7f1b98068ba0 msgr2=0x7f1b980a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:26.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 shutdown_connections 2024-09-17T16:46:26.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 -- 172.21.15.105:0/4019396749 wait complete. 2024-09-17T16:46:26.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:26.770+0000 7f1b9fbc2640 1 librados: shutdown 2024-09-17T16:46:29.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: get_monmap_and_config 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: build_initial_monmap 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: monmap: 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:29.925 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:29.921596+0000 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:29.921596+0000 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:29.926 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:29.927 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:29.927 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 Processor -- start 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 -- start start 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: init 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient: _add_conns ranks=[8,1,6,5,7,10,2,0,3,4,9] 2024-09-17T16:46:29.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-e con 0x7efde00a3720 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efde00a3b00 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde00a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-d con 0x7efde00a3f00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde006db50 con 0x7efde00a3b00 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efde00a0e80 con 0x7efde00a3f00 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7efde00a1490 con 0x7efde00a3720 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efde5e55640 10 monclient(hunting): authenticate will time out at 2073.187514s 2024-09-17T16:46:29.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde00a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51160/0 (socket says 172.21.15.105:51160) 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 1 -- 172.21.15.105:0/598949239 learned_addr learned my addr 172.21.15.105:0/598949239 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): get_auth_request con 0x7efde00a3f00 auth_method 0 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): get_auth_request con 0x7efde00a3720 auth_method 0 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:29.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.918+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 10 monclient(hunting): handle_auth_done global_id 55097 payload 274 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 -- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 msgr2=0x7efde00a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde00a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 -- 172.21.15.105:0/598949239 >> v1:172.21.15.160:6789/0 conn(0x7efde00a3b00 legacy=0x7efde00a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 monclient: found mon.noname-e 2024-09-17T16:46:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 -- 172.21.15.105:0/598949239 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde00a7240 con 0x7efde00a3720 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddffff640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde00a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efddf7fe640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efdd40098a0 tx=0x7efdd401c560 comp rx=0 tx=0).ready entity=mon.8 client_cookie=fbffc50bb9c7ee18 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 1 -- 172.21.15.105:0/598949239 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd40b2070 con 0x7efde00a3720 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: dump: 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 1 -- 172.21.15.105:0/598949239 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd40ab7b0 con 0x7efde00a3720 2024-09-17T16:46:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 1 -- 172.21.15.105:0/598949239 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd40abe50 con 0x7efde00a3720 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efdde7fc640 10 monclient: dump: 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 5 monclient: authenticate success, global_id 55097 2024-09-17T16:46:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 4 set_mon_vals no callback set 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 10 monclient: shutdown 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 -- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 msgr2=0x7efde00a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efdd40098a0 tx=0x7efdd401c560 comp rx=0 tx=0).stop 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 -- 172.21.15.105:0/598949239 shutdown_connections 2024-09-17T16:46:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde00a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 --2- 172.21.15.105:0/598949239 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7efde00a3720 0x7efde00a1e70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 -- 172.21.15.105:0/598949239 >> 172.21.15.105:0/598949239 conn(0x7efde00aa390 msgr2=0x7efde00a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 -- 172.21.15.105:0/598949239 shutdown_connections 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 1 -- 172.21.15.105:0/598949239 wait complete. 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 10 monclient: build_initial_monmap 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.922+0000 7efde5e55640 10 monclient: monmap: 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 librados: starting msgr at 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 librados: starting objecter 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 Processor -- start 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- start start 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 librados: setting wanted keys 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 librados: calling monclient init 2024-09-17T16:46:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: init 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _add_conns ranks=[4,3,6,0,8,2,7,5,1] 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efde00a3720 0x7efde01d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-c con 0x7efde00a3720 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efde00a3b00 0x7efde01d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-g con 0x7efde00a3b00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): picked mon.noname-d con 0x7efde00a3f00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7efde01442e0 con 0x7efde00a3b00 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:29.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7efde00a6520 con 0x7efde00a3720 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7efde0144460 con 0x7efde00a3f00 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient(hunting): authenticate will time out at 2073.192953s 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efde00a3720 0x7efde01d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54760/0 (socket says 172.21.15.105:54760) 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 -- 172.21.15.105:0/1959830229 learned_addr learned my addr 172.21.15.105:0/1959830229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): get_auth_request con 0x7efde00a3f00 auth_method 0 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): get_auth_request con 0x7efde00a3720 auth_method 0 2024-09-17T16:46:29.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddeffd640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efde00a3b00 0x7efde01d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddeffd640 10 monclient(hunting): get_auth_request con 0x7efde00a3b00 auth_method 0 2024-09-17T16:46:29.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddeffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient(hunting): handle_auth_done global_id 94326 payload 824 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 -- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efde00a3720 msgr2=0x7efde01d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efde00a3720 0x7efde01d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:29.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 -- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efde00a3b00 msgr2=0x7efde01d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efde00a3b00 0x7efde01d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient: _finish_hunting 0 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 monclient: found mon.noname-d 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:29.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 -- 172.21.15.105:0/1959830229 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde0125d80 con 0x7efde00a3f00 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efddffff640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efdd800efc0 tx=0x7efdd800c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 client.?.objecter ms_handle_connect 0x7efde00a3f00 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd8018020 con 0x7efde00a3f00 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:29.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: dump: 2024-09-17T16:46:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.953 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:29.954 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: _finish_auth 0 2024-09-17T16:46:29.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd800cd30 con 0x7efde00a3f00 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 5 monclient: authenticate success, global_id 94326 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd8014d60 con 0x7efde00a3f00 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:29.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdddffb640 4 set_mon_vals no callback set 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 monclient: dump: 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:29.957 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:29.958 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:29.959 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:29.959 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:29.959 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:29.959 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:29.959 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:29.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _renew_subs 2024-09-17T16:46:29.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:29.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde00a7cb0 con 0x7efde00a3f00 2024-09-17T16:46:29.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 client.94326.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:29.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _renew_subs 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01257c0 con 0x7efde00a3f00 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 librados: init done 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:29.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00a1e70 con 0x7efde00a3f00 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7efdd801b020 con 0x7efde00a3f00 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 0x7efdcc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7efdd80505e0 con 0x7efde00a3f00 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 10 client.94326.objecter ms_dispatch 0x7efde0000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 3 client.94326.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 3 client.94326.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.926+0000 7efdc7fff640 20 client.94326.objecter dump_active .. 0 homeless 2024-09-17T16:46:29.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.930+0000 7efddf7fe640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 0x7efdcc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.930+0000 7efddf7fe640 10 monclient: get_auth_request con 0x7efdcc03e550 auth_method 0 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.930+0000 7efdc7fff640 10 client.94326.objecter ms_handle_connect 0x7efdcc03e550 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.930+0000 7efddf7fe640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 0x7efdcc040a10 secure :-1 s=READY pgs=3663 cs=0 l=1 rev1=1 crypto rx=0x7efdd40af000 tx=0x7efdd4009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.934+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efdd8022070 con 0x7efde00a3f00 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.934+0000 7efdc7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:29.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:29.934+0000 7efdc7fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:30.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efde5e55640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:46:30.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efde5e55640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:30.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efde5e55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:30.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efde0124850 con 0x7efde00a3f00 2024-09-17T16:46:30.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efdc7fff640 1 -- 172.21.15.105:0/1959830229 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3201 (secure 0 0 0) 0x7efdd8010040 con 0x7efde00a3f00 2024-09-17T16:46:30.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efdc7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:30.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.162+0000 7efdc7fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:30.170 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":116,"quorum":[1,2,3,4,6,7,8],"quorum_names":["f","b","g","c","d","i","e"],"quorum_leader_name":"e","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:46:30.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 10 librados: watch_flush enter 2024-09-17T16:46:30.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 10 librados: watch_flush exit 2024-09-17T16:46:30.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 20 client.94326.objecter shutdown clearing up homeless session... 2024-09-17T16:46:30.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 10 client.94326.objecter successfully canceled tick 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 msgr2=0x7efdcc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 0x7efdcc040a10 secure :-1 s=READY pgs=3663 cs=0 l=1 rev1=1 crypto rx=0x7efdd40af000 tx=0x7efdd4009040 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 10 monclient: shutdown 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 msgr2=0x7efde013e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efdd800efc0 tx=0x7efdd800c490 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 shutdown_connections 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7efdcc03e550 0x7efdcc040a10 unknown :-1 s=CLOSED pgs=3663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7efde00a3f00 0x7efde013e470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efde00a3b00 0x7efde01d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 --2- 172.21.15.105:0/1959830229 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7efde00a3720 0x7efde01d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 >> 172.21.15.105:0/1959830229 conn(0x7efde00aa390 msgr2=0x7efde00a01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 shutdown_connections 2024-09-17T16:46:30.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 -- 172.21.15.105:0/1959830229 wait complete. 2024-09-17T16:46:30.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:30.166+0000 7efde5e55640 1 librados: shutdown 2024-09-17T16:46:33.189 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:46:33.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: get_monmap_and_config 2024-09-17T16:46:33.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: monmap: 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:33.337010+0000 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:33.337010+0000 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.341 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.342 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.342 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.342 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:33.342 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:33.342 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 Processor -- start 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 -- start start 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: init 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient: _add_conns ranks=[4,3,7,2,9,6,10,8,0,1,5] 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-c con 0x7f2e6c0a6a30 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-g con 0x7f2e6c0a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-i con 0x7f2e6c1397f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c0a7c40 con 0x7f2e6c0a4ab0 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2e6c0a7890 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f2e6c0a4080 con 0x7f2e6c1397f0 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e72444640 10 monclient(hunting): authenticate will time out at 2076.602423s 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34416/0 (socket says 172.21.15.105:34416) 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 -- 172.21.15.105:0/850679877 learned_addr learned my addr 172.21.15.105:0/850679877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a6a30 auth_method 0 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e63fff640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e709e1640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e709e1640 10 monclient(hunting): get_auth_request con 0x7f2e6c1397f0 auth_method 0 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e709e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e709e1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e709e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e63fff640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a4ab0 auth_method 0 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e63fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_done global_id 74866 payload 274 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 -- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c1397f0 msgr2=0x7f2e6c139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 -- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4ab0 msgr2=0x7f2e6c0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c0a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 monclient: found mon.noname-c 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.334+0000 7f2e6bfff640 1 -- 172.21.15.105:0/850679877 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c0a37e0 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0098d0 tx=0x7f2e5c01c410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c2077079e4b80722 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 1 -- 172.21.15.105:0/850679877 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e5c0b1070 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: dump: 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.355 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 5 monclient: authenticate success, global_id 74866 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 1 -- 172.21.15.105:0/850679877 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0aa7b0 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 1 -- 172.21.15.105:0/850679877 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e5c0aae50 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 4 set_mon_vals no callback set 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6b7fe640 10 monclient: dump: 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.357 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: shutdown 2024-09-17T16:46:33.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- 172.21.15.105:0/850679877 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 msgr2=0x7f2e6c0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0098d0 tx=0x7f2e5c01c410 comp rx=0 tx=0).stop 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- 172.21.15.105:0/850679877 shutdown_connections 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- 172.21.15.105:0/850679877 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2e6c0a6a30 0x7f2e6c0a6e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- 172.21.15.105:0/850679877 >> 172.21.15.105:0/850679877 conn(0x7f2e6c0a8d00 msgr2=0x7f2e6c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- 172.21.15.105:0/850679877 shutdown_connections 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- 172.21.15.105:0/850679877 wait complete. 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: monmap: 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.359 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.360 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 librados: starting msgr at 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 librados: starting objecter 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 Processor -- start 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- start start 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 librados: setting wanted keys 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 librados: calling monclient init 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: init 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient: _add_conns ranks=[2,1,6,5,3,8,7,0,4] 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c1362b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-b con 0x7f2e6c0a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 0x7f2e6c1367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-f con 0x7f2e6c0a6a30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): picked mon.noname-d con 0x7f2e6c1397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c06e770 con 0x7f2e6c0a6a30 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c13a5a0 con 0x7f2e6c0a4ab0 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2e6c143260 con 0x7f2e6c1397f0 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e72444640 10 monclient(hunting): authenticate will time out at 2076.607352s 2024-09-17T16:46:33.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54776/0 (socket says 172.21.15.105:54776) 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 -- 172.21.15.105:0/844202033 learned_addr learned my addr 172.21.15.105:0/844202033 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): get_auth_request con 0x7f2e6c1397f0 auth_method 0 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c1362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a4ab0 auth_method 0 2024-09-17T16:46:33.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 0x7f2e6c1367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 10 monclient(hunting): get_auth_request con 0x7f2e6c0a6a30 auth_method 0 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient(hunting): handle_auth_done global_id 94335 payload 824 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 -- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e6c0a4ab0 msgr2=0x7f2e6c1362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c1362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 -- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 msgr2=0x7f2e6c1367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 0x7f2e6c1367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 monclient: found mon.noname-d 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e709e1640 1 -- 172.21.15.105:0/844202033 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c13faf0 con 0x7f2e6c1397f0 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.338+0000 7f2e63fff640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 0x7f2e6c1367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e709e1640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e64007ca0 tx=0x7f2e6400e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 client.?.objecter ms_handle_connect 0x7f2e6c1397f0 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e6400edd0 con 0x7f2e6c1397f0 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: dump: 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.366 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 5 monclient: authenticate success, global_id 94335 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e64019070 con 0x7f2e6c1397f0 2024-09-17T16:46:33.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e64014d00 con 0x7f2e6c1397f0 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e6affd640 4 set_mon_vals no callback set 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 monclient: dump: 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _renew_subs 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c0aa070 con 0x7f2e6c1397f0 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 client.94335.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _renew_subs 2024-09-17T16:46:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c13f500 con 0x7f2e6c1397f0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 1 librados: init done 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c139c50 con 0x7f2e6c1397f0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2e64014420 con 0x7f2e6c1397f0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 0x7f2e50040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f2e64050dd0 con 0x7f2e6c1397f0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 client.94335.objecter ms_dispatch 0x7f2e6c000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 3 client.94335.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 3 client.94335.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 20 client.94335.objecter dump_active .. 0 homeless 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 0x7f2e50040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e6bfff640 10 monclient: get_auth_request con 0x7f2e5003e580 auth_method 0 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e6bfff640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 0x7f2e50040a40 secure :-1 s=READY pgs=3664 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ae000 tx=0x7f2e5c0a6070 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.342+0000 7f2e697fa640 10 client.94335.objecter ms_handle_connect 0x7f2e5003e580 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.346+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2e6c0aa070 con 0x7f2e6c1397f0 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.346+0000 7f2e697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.346+0000 7f2e697fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e72444640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:46:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e72444640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e72444640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2e6c13da40 con 0x7f2e6c1397f0 2024-09-17T16:46:33.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e697fa640 1 -- 172.21.15.105:0/844202033 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7f2e6c13faf0 con 0x7f2e6c1397f0 2024-09-17T16:46:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:33.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.586+0000 7f2e697fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:33.594 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":118,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 10 librados: watch_flush enter 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 10 librados: watch_flush exit 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 20 client.94335.objecter shutdown clearing up homeless session... 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 10 client.94335.objecter successfully canceled tick 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 msgr2=0x7f2e50040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 0x7f2e50040a40 secure :-1 s=READY pgs=3664 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ae000 tx=0x7f2e5c0a6070 comp rx=0 tx=0).stop 2024-09-17T16:46:33.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 10 monclient: shutdown 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 msgr2=0x7f2e6c13b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2e64007ca0 tx=0x7f2e6400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 shutdown_connections 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2e5003e580 0x7f2e50040a40 unknown :-1 s=CLOSED pgs=3664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e6c0a6a30 0x7f2e6c1367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2e6c1397f0 0x7f2e6c13b8c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 --2- 172.21.15.105:0/844202033 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e6c0a4ab0 0x7f2e6c1362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 >> 172.21.15.105:0/844202033 conn(0x7f2e6c0a8d00 msgr2=0x7f2e6c13fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 shutdown_connections 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 -- 172.21.15.105:0/844202033 wait complete. 2024-09-17T16:46:33.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.590+0000 7f2e72444640 1 librados: shutdown 2024-09-17T16:46:33.613 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:46:33.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-17T16:46:33.613 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.726+0000 7f8bae72a640 10 monclient: get_monmap_and_config 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient: monmap: 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:33.731822+0000 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:33.731822+0000 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:33.736 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 Processor -- start 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 -- start start 2024-09-17T16:46:33.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient: init 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient: _add_conns ranks=[10,6,7,5,3,2,1,4,8,9,0] 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8ba80a6a30 0x7f8ba80a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-f con 0x7f8ba80a6a30 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-d con 0x7f8ba80a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 0x7f8ba8139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-i con 0x7f8ba81397f0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba80a7c40 con 0x7f8ba80a6a30 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f8ba80a7890 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8ba80a4080 con 0x7f8ba81397f0 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8bae72a640 10 monclient(hunting): authenticate will time out at 2076.997524s 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54780/0 (socket says 172.21.15.105:54780) 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 -- 172.21.15.105:0/2490681635 learned_addr learned my addr 172.21.15.105:0/2490681635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): get_auth_request con 0x7f8ba80a4ab0 auth_method 0 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba7fff640 1 --2- 172.21.15.105:0/2490681635 >> v2:172.21.15.160:3300/0 conn(0x7f8ba80a6a30 0x7f8ba80a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8baccc7640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 0x7f8ba8139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba7fff640 10 monclient(hunting): get_auth_request con 0x7f8ba80a6a30 auth_method 0 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_done global_id 94344 payload 274 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 -- 172.21.15.105:0/2490681635 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 msgr2=0x7f8ba8139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 0x7f8ba8139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 -- 172.21.15.105:0/2490681635 >> v2:172.21.15.160:3300/0 conn(0x7f8ba80a6a30 msgr2=0x7f8ba80a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/2490681635 >> v2:172.21.15.160:3300/0 conn(0x7f8ba80a6a30 0x7f8ba80a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 monclient: found mon.noname-d 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8ba77fe640 1 -- 172.21.15.105:0/2490681635 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba80a37e0 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8baccc7640 10 monclient: get_auth_request con 0x7f8ba81397f0 auth_method 0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.730+0000 7f8baccc7640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 0x7f8ba8139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b980098a0 tx=0x7f8b9801c3d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=da032fe32670b261 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/2490681635 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b980b0070 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: dump: 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/2490681635 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980aa810 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 5 monclient: authenticate success, global_id 94344 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/2490681635 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b980a9300 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 4 set_mon_vals no callback set 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba6ffd640 10 monclient: dump: 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: shutdown 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- 172.21.15.105:0/2490681635 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 msgr2=0x7f8ba80a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b980098a0 tx=0x7f8b9801c3d0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- 172.21.15.105:0/2490681635 shutdown_connections 2024-09-17T16:46:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8ba81397f0 0x7f8ba8139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- 172.21.15.105:0/2490681635 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f8ba80a4ab0 0x7f8ba80a7350 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- 172.21.15.105:0/2490681635 >> v2:172.21.15.160:3300/0 conn(0x7f8ba80a6a30 0x7f8ba80a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- 172.21.15.105:0/2490681635 >> 172.21.15.105:0/2490681635 conn(0x7f8ba80a8d00 msgr2=0x7f8ba80a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- 172.21.15.105:0/2490681635 shutdown_connections 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- 172.21.15.105:0/2490681635 wait complete. 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: monmap: 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.750 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 librados: starting msgr at 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 librados: starting objecter 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 Processor -- start 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- start start 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 librados: setting wanted keys 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 librados: calling monclient init 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: init 2024-09-17T16:46:33.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient: _add_conns ranks=[2,5,3,0,6,7,1,8,4] 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 0x7f8ba81d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-b con 0x7f8ba80a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8ba80a6a30 0x7f8ba81d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-h con 0x7f8ba80a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba81397f0 0x7f8ba813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): picked mon.noname-g con 0x7f8ba81397f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba806e770 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80a2fc0 con 0x7f8ba81397f0 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8ba8143260 con 0x7f8ba80a6a30 2024-09-17T16:46:33.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8bae72a640 10 monclient(hunting): authenticate will time out at 2077.003342s 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 0x7f8ba81d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 0x7f8ba81d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49438/0 (socket says 172.21.15.105:49438) 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 -- 172.21.15.105:0/4268700581 learned_addr learned my addr 172.21.15.105:0/4268700581 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): get_auth_request con 0x7f8ba80a4ab0 auth_method 0 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8baccc7640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba81397f0 0x7f8ba813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8ba80a6a30 0x7f8ba81d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8baccc7640 10 monclient(hunting): get_auth_request con 0x7f8ba81397f0 auth_method 0 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8baccc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8baccc7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8baccc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient(hunting): handle_auth_done global_id 84584 payload 824 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8ba80a6a30 msgr2=0x7f8ba81d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8ba80a6a30 0x7f8ba81d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba81397f0 msgr2=0x7f8ba813d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba81397f0 0x7f8ba813d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 monclient: found mon.noname-b 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:33.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba7fff640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba813fb80 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba77fe640 10 monclient: get_auth_request con 0x7f8ba80a6a30 auth_method 0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.734+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8ba80a6a30 0x7f8ba81d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba7fff640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 0x7f8ba81d1440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00d800 tx=0x7f8b9c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.?.objecter ms_handle_connect 0x7f8ba80a4ab0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b9c00bb30 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: dump: 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.762 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 5 monclient: authenticate success, global_id 84584 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c019070 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 monclient: _renew_subs 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba80aa100 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b9c00bcf0 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: dump: 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.765 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba5ffb640 4 set_mon_vals no callback set 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 client.84584.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 monclient: _renew_subs 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba8123b60 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 1 librados: init done 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 0x7f8ba813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8051e80 con 0x7f8ba8124260 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8b9c00b6c0 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 0x7f8ba813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient: get_auth_request con 0x7f8ba8124260 auth_method 0 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b9403e480 0x7f8b94040940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f8b9c00f040 con 0x7f8ba80a4ab0 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.84584.objecter ms_dispatch 0x7f8ba8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 3 client.84584.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 3 client.84584.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 20 client.84584.objecter dump_active .. 0 homeless 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_done global_id 104103 payload 824 2024-09-17T16:46:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ba8139ca0 con 0x7f8ba8124260 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.84584.objecter ms_handle_connect 0x7f8ba8124260 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.84584.objecter resend_mon_ops 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8baccc7640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b9403e480 0x7f8b94040940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 0x7f8ba813da10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80a6570 tx=0x7f8b98019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8baccc7640 10 monclient: get_auth_request con 0x7f8b9403e480 auth_method 0 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b980b0040 con 0x7f8ba8124260 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: dump: 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8b98003620 con 0x7f8ba8124260 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 msgr2=0x7f8ba813da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 0x7f8ba813da10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80a6570 tx=0x7f8b98019170 comp rx=0 tx=0).stop 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 msgr2=0x7f8ba813da10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba8124260 0x7f8ba813da10 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80a6570 tx=0x7f8b98019170 comp rx=0 tx=0).stop 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8ba4ff9640 10 client.84584.objecter ms_handle_connect 0x7f8b9403e480 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.738+0000 7f8baccc7640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b9403e480 0x7f8b94040940 secure :-1 s=READY pgs=3665 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00074d0 tx=0x7f8ba0007460 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 0x7f8ba813e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba806a200 con 0x7f8ba813df50 2024-09-17T16:46:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 0x7f8ba813e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient: get_auth_request con 0x7f8ba813df50 auth_method 0 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 10 monclient(hunting): handle_auth_done global_id 104112 payload 824 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 1 -- 172.21.15.105:0/4268700581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8ba80abcd0 con 0x7f8ba813df50 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 client.84584.objecter ms_handle_connect 0x7f8ba813df50 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 client.84584.objecter resend_mon_ops 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba77fe640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 0x7f8ba813e360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801cc70 tx=0x7f8b980a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b980b0040 con 0x7f8ba813df50 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 monclient: dump: 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.772 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 <== mon.0 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f8b980bd8c0 con 0x7f8ba813df50 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 msgr2=0x7f8ba813e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 0x7f8ba813e360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801cc70 tx=0x7f8b980a8000 comp rx=0 tx=0).stop 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 msgr2=0x7f8ba813e360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:33.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8ba4ff9640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ba813df50 0x7f8ba813e360 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801cc70 tx=0x7f8b980a8000 comp rx=0 tx=0).stop 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 28296, 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 2, 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.775 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:33.776 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.777 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:33.778 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.779 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.782 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.783 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:33.784 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.785 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.788 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:33.789 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.791 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.792 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.793 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.794 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.795 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:33.796 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:33.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 librados: watch_flush enter 2024-09-17T16:46:33.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 librados: watch_flush exit 2024-09-17T16:46:33.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 20 client.84584.objecter shutdown clearing up homeless session... 2024-09-17T16:46:33.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 client.84584.objecter successfully canceled tick 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b9403e480 msgr2=0x7f8b94040940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8b9403e480 0x7f8b94040940 secure :-1 s=READY pgs=3665 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00074d0 tx=0x7f8ba0007460 comp rx=0 tx=0).stop 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 10 monclient: shutdown 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 msgr2=0x7f8ba81d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 --2- 172.21.15.105:0/4268700581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8ba80a4ab0 0x7f8ba81d1440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00d800 tx=0x7f8b9c00dcc0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8baccc7640 1 -- 172.21.15.105:0/4268700581 reap_dead start 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8baccc7640 1 -- 172.21.15.105:0/4268700581 reap_dead start 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 shutdown_connections 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 >> 172.21.15.105:0/4268700581 conn(0x7f8ba80a8d00 msgr2=0x7f8ba80a4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 shutdown_connections 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 -- 172.21.15.105:0/4268700581 wait complete. 2024-09-17T16:46:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.746+0000 7f8bae72a640 1 librados: shutdown 2024-09-17T16:46:33.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:33.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-17T16:46:33.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: get_monmap_and_config 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: monmap: 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:33.912506+0000 2024-09-17T16:46:33.916 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:33.912506+0000 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:33.917 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:33.918 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 Processor -- start 2024-09-17T16:46:33.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 -- start start 2024-09-17T16:46:33.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: init 2024-09-17T16:46:33.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient: _add_conns ranks=[9,0,8,4,1,5,3,7,2,10,6] 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96440a3720 0x7f96440a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-a con 0x7f96440a3720 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f96440a3b00 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f96440a3f00 0x7f96440a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-e con 0x7f96440a3f00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f964406db50 con 0x7f96440a3b00 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f96440a0e80 con 0x7f96440a3720 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f96440a1490 con 0x7f96440a3f00 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964c1f7640 10 monclient(hunting): authenticate will time out at 2077.178300s 2024-09-17T16:46:33.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f96440a3720 0x7f96440a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649792640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f96440a3b00 0x7f96440a56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54696/0 (socket says 172.21.15.105:54696) 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649792640 1 -- 172.21.15.105:0/803889407 learned_addr learned my addr 172.21.15.105:0/803889407 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): get_auth_request con 0x7f96440a3720 auth_method 0 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f96440a3f00 0x7f96440a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): get_auth_request con 0x7f96440a3f00 auth_method 0 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient(hunting): handle_auth_done global_id 104121 payload 274 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 -- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f96440a3f00 msgr2=0x7f96440a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f96440a3f00 0x7f96440a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 -- 172.21.15.105:0/803889407 >> v1:172.21.15.105:6789/0 conn(0x7f96440a3b00 legacy=0x7f96440a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 monclient: found mon.noname-a 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:33.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.910+0000 7f9649f93640 1 -- 172.21.15.105:0/803889407 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440a7240 con 0x7f96440a3720 2024-09-17T16:46:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96440a1e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96400097c0 tx=0x7f964001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a614f1053965f1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 1 -- 172.21.15.105:0/803889407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96400b0070 con 0x7f96440a3720 2024-09-17T16:46:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: dump: 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 5 monclient: authenticate success, global_id 104121 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 1 -- 172.21.15.105:0/803889407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96400aa810 con 0x7f96440a3720 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:33.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 4 set_mon_vals no callback set 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 1 -- 172.21.15.105:0/803889407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96400a9300 con 0x7f96440a3720 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9648f91640 10 monclient: dump: 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: shutdown 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 msgr2=0x7f96440a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96440a1e70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96400097c0 tx=0x7f964001c3f0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- 172.21.15.105:0/803889407 shutdown_connections 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f96440a3f00 0x7f96440a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- 172.21.15.105:0/803889407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96440a1e70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- 172.21.15.105:0/803889407 >> 172.21.15.105:0/803889407 conn(0x7f96440aa390 msgr2=0x7f96440a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- 172.21.15.105:0/803889407 shutdown_connections 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- 172.21.15.105:0/803889407 wait complete. 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: build_initial_monmap 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: monmap: 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:33.934 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 librados: starting msgr at 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 librados: starting objecter 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 Processor -- start 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- start start 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 librados: setting wanted keys 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 librados: calling monclient init 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: init 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient: _add_conns ranks=[0,2,4,3,1,7,8,5,6] 2024-09-17T16:46:33.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96441d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-a con 0x7f96440a3720 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96440a3b00 0x7f96441d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-b con 0x7f96440a3b00 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 0x7f964413e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): picked mon.noname-c con 0x7f96440a3f00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96441442e0 con 0x7f96440a3720 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f96440a6520 con 0x7f96440a3b00 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f9644144460 con 0x7f96440a3f00 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:33.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964c1f7640 10 monclient(hunting): authenticate will time out at 2077.183500s 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96441d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96440a3b00 0x7f96441d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96441d1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51392/0 (socket says 172.21.15.105:51392) 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 1 -- 172.21.15.105:0/1208045485 learned_addr learned my addr 172.21.15.105:0/1208045485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): get_auth_request con 0x7f96440a3720 auth_method 0 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 0x7f964413e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): get_auth_request con 0x7f96440a3f00 auth_method 0 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): get_auth_request con 0x7f96440a3b00 auth_method 0 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649792640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.914+0000 7f964a794640 10 monclient(hunting): handle_auth_done global_id 74875 payload 824 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96440a3b00 msgr2=0x7f96441d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f96440a3b00 0x7f96441d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 msgr2=0x7f96441d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96441d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 10 monclient: _finish_hunting 0 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 monclient: found mon.noname-c 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440a7d40 con 0x7f96440a3f00 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96440a3720 0x7f96441d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964a794640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 0x7f964413e750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f963c00eb70 tx=0x7f963c00c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.?.objecter ms_handle_connect 0x7f96440a3f00 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f963c018020 con 0x7f96440a3f00 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: dump: 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c014480 con 0x7f96440a3f00 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 5 monclient: authenticate success, global_id 74875 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 monclient: _renew_subs 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9644125b80 con 0x7f96440a3f00 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:33.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f963c014ae0 con 0x7f96440a3f00 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: dump: 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963bfff640 4 set_mon_vals no callback set 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 client.74875.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 monclient: _renew_subs 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f964413c8b0 con 0x7f96440a3f00 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 1 librados: init done 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 0x7f964413f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9644051e80 con 0x7f964413f450 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 0x7f964413f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient: get_auth_request con 0x7f964413f450 auth_method 0 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f963c01b020 con 0x7f96440a3f00 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 10 monclient(hunting): handle_auth_done global_id 84602 payload 824 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96440a5fa0 con 0x7f964413f450 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f962803e550 0x7f9628040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f963c050920 con 0x7f96440a3f00 2024-09-17T16:46:33.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.74875.objecter ms_dispatch 0x7f9644000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 3 client.74875.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 3 client.74875.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649f93640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 0x7f964413f830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9640009710 tx=0x7f9640019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 20 client.74875.objecter dump_active .. 0 homeless 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.74875.objecter ms_handle_connect 0x7f964413f450 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.74875.objecter resend_mon_ops 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96400b0040 con 0x7f964413f450 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649792640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f962803e550 0x7f9628040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: dump: 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.945 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649792640 10 monclient: get_auth_request con 0x7f962803e550 auth_method 0 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f96400aa750 con 0x7f964413f450 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 msgr2=0x7f964413f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 0x7f964413f830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9640009710 tx=0x7f9640019190 comp rx=0 tx=0).stop 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 msgr2=0x7f964413f830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964413f450 0x7f964413f830 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9640009710 tx=0x7f9640019190 comp rx=0 tx=0).stop 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f963a7fc640 10 client.74875.objecter ms_handle_connect 0x7f962803e550 2024-09-17T16:46:33.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.918+0000 7f9649792640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f962803e550 0x7f9628040a10 secure :-1 s=READY pgs=3666 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f70 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 0x7f964402d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f96440727a0 con 0x7f964402d450 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 0x7f964402d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient: get_auth_request con 0x7f964402d450 auth_method 0 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:33.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 10 monclient(hunting): handle_auth_done global_id 84611 payload 824 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 1 -- 172.21.15.105:0/1208045485 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f96440a1e70 con 0x7f964402d450 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f9649f93640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 0x7f964402d860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f964001aaa0 tx=0x7f96400a6000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 client.74875.objecter ms_handle_connect 0x7f964402d450 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 client.74875.objecter resend_mon_ops 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9640003e20 con 0x7f964402d450 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 monclient: dump: 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.950 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f9640003e20 con 0x7f964402d450 2024-09-17T16:46:33.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 msgr2=0x7f964402d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 0x7f964402d860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f964001aaa0 tx=0x7f96400a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 msgr2=0x7f964402d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f963a7fc640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f964402d450 0x7f964402d860 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f964001aaa0 tx=0x7f96400a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:33.952 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 301162, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.953 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.954 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.955 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:33.956 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:33.957 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.958 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.959 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.960 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.961 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.962 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.963 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:33.964 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:33.965 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.967 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.968 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.970 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:33.971 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:33.972 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:33.973 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:33.974 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:33.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 librados: watch_flush enter 2024-09-17T16:46:33.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 librados: watch_flush exit 2024-09-17T16:46:33.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 20 client.74875.objecter shutdown clearing up homeless session... 2024-09-17T16:46:33.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 client.74875.objecter successfully canceled tick 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f962803e550 msgr2=0x7f9628040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f962803e550 0x7f9628040a10 secure :-1 s=READY pgs=3666 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f70 tx=0x7f9634009040 comp rx=0 tx=0).stop 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 10 monclient: shutdown 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 msgr2=0x7f964413e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 0x7f964413e750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f963c00eb70 tx=0x7f963c00c490 comp rx=0 tx=0).stop 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964a794640 1 -- 172.21.15.105:0/1208045485 reap_dead start 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 shutdown_connections 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 --2- 172.21.15.105:0/1208045485 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f96440a3f00 0x7f964413e750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 >> 172.21.15.105:0/1208045485 conn(0x7f96440aa390 msgr2=0x7f96440a01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 shutdown_connections 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 -- 172.21.15.105:0/1208045485 wait complete. 2024-09-17T16:46:33.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:33.926+0000 7f964c1f7640 1 librados: shutdown 2024-09-17T16:46:33.977 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:33.977 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-17T16:46:33.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: get_monmap_and_config 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: monmap: 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:34.088542+0000 2024-09-17T16:46:34.093 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:34.088542+0000 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:34.094 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 Processor -- start 2024-09-17T16:46:34.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 -- start start 2024-09-17T16:46:34.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: init 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient: _add_conns ranks=[3,9,6,8,4,0,2,5,7,1,10] 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a68e0 0x7f69940a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): picked mon.noname-g con 0x7f69940a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69940a4910 0x7f69940a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): picked mon.noname-a con 0x7f69940a4910 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): picked mon.noname-d con 0x7f69941396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f69940a3fc0 con 0x7f69940a4910 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f69940a63c0 con 0x7f69940a68e0 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f69940a7740 con 0x7f69941396a0 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699c016640 10 monclient(hunting): authenticate will time out at 2077.354686s 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69940a4910 0x7f69940a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54790/0 (socket says 172.21.15.105:54790) 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 1 -- 172.21.15.105:0/2092330645 learned_addr learned my addr 172.21.15.105:0/2092330645 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): get_auth_request con 0x7f69941396a0 auth_method 0 2024-09-17T16:46:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): get_auth_request con 0x7f69940a4910 auth_method 0 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f6999db2640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a68e0 0x7f69940a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f6999db2640 10 monclient(hunting): get_auth_request con 0x7f69940a68e0 auth_method 0 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f6999db2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f6999db2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f6999db2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient(hunting): handle_auth_done global_id 104148 payload 274 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 -- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 msgr2=0x7f6994139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 -- 172.21.15.105:0/2092330645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a68e0 msgr2=0x7f69940a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a68e0 0x7f69940a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 monclient: found mon.noname-a 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f69995b1640 1 -- 172.21.15.105:0/2092330645 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699406ebc0 con 0x7f69940a4910 2024-09-17T16:46:34.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.086+0000 7f699a5b3640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f69995b1640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69940a4910 0x7f69940a7200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69880097c0 tx=0x7f698801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f947fda7e5b1ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 1 -- 172.21.15.105:0/2092330645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69880b0070 con 0x7f69940a4910 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: dump: 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 1 -- 172.21.15.105:0/2092330645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880aa810 con 0x7f69940a4910 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 5 monclient: authenticate success, global_id 104148 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 1 -- 172.21.15.105:0/2092330645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69880a9300 con 0x7f69940a4910 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 4 set_mon_vals no callback set 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f6998db0640 10 monclient: dump: 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 10 monclient: shutdown 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69940a4910 msgr2=0x7f69940a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69940a4910 0x7f69940a7200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f69880097c0 tx=0x7f698801c330 comp rx=0 tx=0).stop 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- 172.21.15.105:0/2092330645 shutdown_connections 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f69941396a0 0x7f6994139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69940a4910 0x7f69940a7200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 --2- 172.21.15.105:0/2092330645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a68e0 0x7f69940a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- 172.21.15.105:0/2092330645 >> 172.21.15.105:0/2092330645 conn(0x7f6994068ba0 msgr2=0x7f6994068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- 172.21.15.105:0/2092330645 shutdown_connections 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- 172.21.15.105:0/2092330645 wait complete. 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 10 monclient: monmap: 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:34.114 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 librados: starting msgr at 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 librados: starting objecter 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 Processor -- start 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 -- start start 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 librados: setting wanted keys 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 1 librados: calling monclient init 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.090+0000 7f699c016640 10 monclient: init 2024-09-17T16:46:34.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _add_conns ranks=[3,8,4,2,5,6,1,7,0] 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a4910 0x7f69941d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): picked mon.noname-g con 0x7f69940a4910 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): picked mon.noname-e con 0x7f69940a68e0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f69941396a0 0x7f699413d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): picked mon.noname-c con 0x7f69941396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f699406e630 con 0x7f69940a4910 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f69940a7af0 con 0x7f69941396a0 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f6994143320 con 0x7f69940a68e0 2024-09-17T16:46:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): authenticate will time out at 2077.360442s 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f69941396a0 0x7f699413d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51176/0 (socket says 172.21.15.105:51176) 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 -- 172.21.15.105:0/4004046478 learned_addr learned my addr 172.21.15.105:0/4004046478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): get_auth_request con 0x7f69940a68e0 auth_method 0 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): get_auth_request con 0x7f69941396a0 auth_method 0 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a4910 0x7f69941d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient(hunting): handle_auth_done global_id 55115 payload 824 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f69941396a0 msgr2=0x7f699413d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f69941396a0 0x7f699413d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a4910 msgr2=0x7f69941d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a4910 0x7f69941d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 monclient: found mon.noname-e 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69940aa1a0 con 0x7f69940a68e0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient: get_auth_request con 0x7f69940a4910 auth_method 0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69940a4910 0x7f69941d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f69941396a0 0x7f699413d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69995b1640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f698801cbd0 tx=0x7f69880190d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 client.?.objecter ms_handle_connect 0x7f69940a68e0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69880b0040 con 0x7f69940a68e0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: dump: 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.119 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 5 monclient: authenticate success, global_id 55115 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880b9070 con 0x7f69940a68e0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _renew_subs 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6994123710 con 0x7f69940a68e0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69880044b0 con 0x7f69940a68e0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 client.55115.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6993fff640 4 set_mon_vals no callback set 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: dump: 2024-09-17T16:46:34.121 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.122 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _renew_subs 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f69940a9de0 con 0x7f69940a68e0 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 librados: init done 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 0x7f699413e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6994051e80 con 0x7f699413dc80 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 0x7f699413e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient: get_auth_request con 0x7f699413dc80 auth_method 0 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6988004040 con 0x7f69940a68e0 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 10 monclient(hunting): handle_auth_done global_id 74893 payload 824 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6994139b50 con 0x7f699413dc80 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f698003e580 0x7f6980040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 0x7f699413e060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6984007db0 tx=0x7f698400f520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 client.55115.objecter ms_handle_connect 0x7f699413dc80 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 client.55115.objecter resend_mon_ops 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f69880e98c0 con 0x7f69940a68e0 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 client.55115.objecter ms_dispatch 0x7f6994000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 3 client.55115.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 3 client.55115.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 20 client.55115.objecter dump_active .. 0 homeless 2024-09-17T16:46:34.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6984003e10 con 0x7f699413dc80 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: dump: 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.125 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f698003e580 0x7f6980040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f699a5b3640 10 monclient: get_auth_request con 0x7f698003e580 auth_method 0 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6984019070 con 0x7f699413dc80 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 msgr2=0x7f699413e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 0x7f699413e060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6984007db0 tx=0x7f698400f520 comp rx=0 tx=0).stop 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 msgr2=0x7f699413e060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.094+0000 7f69927fc640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413dc80 0x7f699413e060 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6984007db0 tx=0x7f698400f520 comp rx=0 tx=0).stop 2024-09-17T16:46:34.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.098+0000 7f69927fc640 10 client.55115.objecter ms_handle_connect 0x7f698003e580 2024-09-17T16:46:34.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.098+0000 7f699a5b3640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f698003e580 0x7f6980040a40 secure :-1 s=READY pgs=3667 cs=0 l=1 rev1=1 crypto rx=0x7f698c0096b0 tx=0x7f698c009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 0x7f69940a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f699413efd0 con 0x7f699413e5a0 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 0x7f69940a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient: get_auth_request con 0x7f699413e5a0 auth_method 0 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 10 monclient(hunting): handle_auth_done global_id 74902 payload 824 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 1 -- 172.21.15.105:0/4004046478 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f69940746d0 con 0x7f699413e5a0 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 client.55115.objecter ms_handle_connect 0x7f699413e5a0 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f6999db2640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 0x7f69940a0a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6984018000 tx=0x7f6984004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 client.55115.objecter resend_mon_ops 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69840044a0 con 0x7f699413e5a0 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 monclient: dump: 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.130 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 <== mon.4 v2:172.21.15.105:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9622 (secure 0 0 0) 0x7f6984004030 con 0x7f699413e5a0 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:34.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 msgr2=0x7f69940a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 0x7f69940a0a20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6984018000 tx=0x7f6984004a50 comp rx=0 tx=0).stop 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 msgr2=0x7f69940a0a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f69927fc640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f699413e5a0 0x7f69940a0a20 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6984018000 tx=0x7f6984004a50 comp rx=0 tx=0).stop 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 433565, 2024-09-17T16:46:34.132 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:34.133 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:34.134 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:34.135 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:34.136 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:34.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:34.138 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.139 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.140 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.142 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.143 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.145 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.147 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.148 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.150 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:34.152 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 2 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.153 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": [ 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.154 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:34.155 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 librados: watch_flush enter 2024-09-17T16:46:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 librados: watch_flush exit 2024-09-17T16:46:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 20 client.55115.objecter shutdown clearing up homeless session... 2024-09-17T16:46:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 client.55115.objecter successfully canceled tick 2024-09-17T16:46:34.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f698003e580 msgr2=0x7f6980040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f698003e580 0x7f6980040a40 secure :-1 s=READY pgs=3667 cs=0 l=1 rev1=1 crypto rx=0x7f698c0096b0 tx=0x7f698c009240 comp rx=0 tx=0).stop 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 10 monclient: shutdown 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 msgr2=0x7f69941d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f698801cbd0 tx=0x7f69880190d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699a5b3640 1 -- 172.21.15.105:0/4004046478 reap_dead start 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 shutdown_connections 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 --2- 172.21.15.105:0/4004046478 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f69940a68e0 0x7f69941d19e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 >> 172.21.15.105:0/4004046478 conn(0x7f6994068ba0 msgr2=0x7f69940a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 shutdown_connections 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 -- 172.21.15.105:0/4004046478 wait complete. 2024-09-17T16:46:34.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.102+0000 7f699c016640 1 librados: shutdown 2024-09-17T16:46:34.158 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:34.158 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-17T16:46:34.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-17T16:46:34.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: get_monmap_and_config 2024-09-17T16:46:34.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: monmap: 2024-09-17T16:46:34.275 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:34.271940+0000 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:34.271940+0000 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.276 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.277 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:34.277 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:34.277 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 Processor -- start 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 -- start start 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: init 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient: _add_conns ranks=[0,10,9,3,1,5,6,4,8,7,2] 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdaa80a7060 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 0x7fdaa813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-f con 0x7fdaa80a5f70 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdaa80a63a0 0x7fdaa8141190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-a con 0x7fdaa80a63a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa806d260 con 0x7fdaa80a7060 2024-09-17T16:46:34.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa80a0e20 con 0x7fdaa80a63a0 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa80a1f10 con 0x7fdaa80a5f70 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaacb64640 10 monclient(hunting): authenticate will time out at 2077.537696s 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdaa80a7060 0x7fdaa80a5860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54710/0 (socket says 172.21.15.105:54710) 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa659c640 1 -- 172.21.15.105:0/2741879099 learned_addr learned my addr 172.21.15.105:0/2741879099 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/2741879099 >> v2:172.21.15.105:3300/0 conn(0x7fdaa80a63a0 0x7fdaa8141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): get_auth_request con 0x7fdaa80a63a0 auth_method 0 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/2741879099 >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 0x7fdaa813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 1 -- 172.21.15.105:0/2741879099 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927439138 0 0) 0x7fdaa806d260 con 0x7fdaa80a7060 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 10 monclient(hunting): _negotiate 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 10 monclient(hunting): my global_id is 104166 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa559a640 1 -- 172.21.15.105:0/2741879099 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda94003530 con 0x7fdaa80a7060 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_done global_id 104157 payload 274 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 -- 172.21.15.105:0/2741879099 >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 msgr2=0x7fdaa813ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/2741879099 >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 0x7fdaa813ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 -- 172.21.15.105:0/2741879099 >> v1:172.21.15.105:6789/0 conn(0x7fdaa80a7060 legacy=0x7fdaa80a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 monclient: found mon.noname-a 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa6d9d640 1 -- 172.21.15.105:0/2741879099 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa806d8b0 con 0x7fdaa80a63a0 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa5d9b640 10 monclient: get_auth_request con 0x7fdaa80a5f70 auth_method 0 2024-09-17T16:46:34.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.270+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/2741879099 >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 0x7fdaa813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/2741879099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa80a63a0 0x7fdaa8141190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80a1880 tx=0x7fda9801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad94a9e94043bf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 1 -- 172.21.15.105:0/2741879099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda980b0070 con 0x7fdaa80a63a0 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: dump: 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.289 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 1 -- 172.21.15.105:0/2741879099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda980aa810 con 0x7fdaa80a63a0 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 5 monclient: authenticate success, global_id 104157 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 1 -- 172.21.15.105:0/2741879099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda980a9300 con 0x7fdaa80a63a0 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 4 set_mon_vals no callback set 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa559a640 10 monclient: dump: 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.291 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: shutdown 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/2741879099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa80a63a0 msgr2=0x7fdaa8141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- 172.21.15.105:0/2741879099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa80a63a0 0x7fdaa8141190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80a1880 tx=0x7fda9801c340 comp rx=0 tx=0).stop 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/2741879099 shutdown_connections 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- 172.21.15.105:0/2741879099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaa80a63a0 0x7fdaa8141190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- 172.21.15.105:0/2741879099 >> v2:172.21.15.160:3300/0 conn(0x7fdaa80a5f70 0x7fdaa813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/2741879099 >> 172.21.15.105:0/2741879099 conn(0x7fdaa80aa450 msgr2=0x7fdaa80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/2741879099 shutdown_connections 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/2741879099 wait complete. 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: monmap: 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.293 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 librados: starting msgr at 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 librados: starting objecter 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 Processor -- start 2024-09-17T16:46:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- start start 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 librados: setting wanted keys 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 librados: calling monclient init 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: init 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _add_conns ranks=[4,8,6,0,3,1,2,5,7] 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 0x7fdaa812a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-c con 0x7fdaa80a5f70 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdaa80a63a0 0x7fdaa81d1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-e con 0x7fdaa80a63a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 0x7fdaa81d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): picked mon.noname-d con 0x7fdaa80a7060 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fdaa8144070 con 0x7fdaa80a5f70 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdaa8143ef0 con 0x7fdaa80a7060 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fdaa81441f0 con 0x7fdaa80a63a0 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient(hunting): authenticate will time out at 2077.542678s 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 0x7fdaa81d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 0x7fdaa812a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdaa80a63a0 0x7fdaa81d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 0x7fdaa81d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54802/0 (socket says 172.21.15.105:54802) 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 1 -- 172.21.15.105:0/1568704925 learned_addr learned my addr 172.21.15.105:0/1568704925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): get_auth_request con 0x7fdaa80a7060 auth_method 0 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): get_auth_request con 0x7fdaa80a5f70 auth_method 0 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): get_auth_request con 0x7fdaa80a63a0 auth_method 0 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient(hunting): handle_auth_done global_id 74911 payload 824 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdaa80a63a0 msgr2=0x7fdaa81d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdaa80a63a0 0x7fdaa81d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 msgr2=0x7fdaa81d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 0x7fdaa81d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 monclient: found mon.noname-c 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa8129450 con 0x7fdaa80a5f70 2024-09-17T16:46:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa80a7060 0x7fdaa81d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fdaa80a63a0 0x7fdaa81d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa659c640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 0x7fdaa812a6e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000cb90 tx=0x7fdaa000a430 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 client.?.objecter ms_handle_connect 0x7fdaa80a5f70 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa0012400 con 0x7fdaa80a5f70 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: dump: 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.299 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa00125c0 con 0x7fdaa80a5f70 2024-09-17T16:46:34.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa0012be0 con 0x7fdaa80a5f70 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fda9effd640 10 monclient: dump: 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.301 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaa4d99640 4 set_mon_vals no callback set 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 5 monclient: authenticate success, global_id 74911 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _renew_subs 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa8127f60 con 0x7fdaa80a5f70 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 client.74911.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _renew_subs 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:34.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa8128d80 con 0x7fdaa80a5f70 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.274+0000 7fdaacb64640 1 librados: init done 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaacb64640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaacb64640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 0x7fdaa8126dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdaa8051e80 con 0x7fdaa81d1bf0 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 0x7fdaa8126dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient: get_auth_request con 0x7fdaa81d1bf0 auth_method 0 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdaa0012770 con 0x7fdaa80a5f70 2024-09-17T16:46:34.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_done global_id 94371 payload 824 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaa80a5280 con 0x7fdaa81d1bf0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fda8803e5b0 0x7fda88040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fdaa0050100 con 0x7fdaa80a5f70 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 client.74911.objecter ms_dispatch 0x7fdaa8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 3 client.74911.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 3 client.74911.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 0x7fdaa8126dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaa81285d0 tx=0x7fda9000d590 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 20 client.74911.objecter dump_active .. 0 homeless 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 client.74911.objecter ms_handle_connect 0x7fdaa81d1bf0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 client.74911.objecter resend_mon_ops 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda9000fc80 con 0x7fdaa81d1bf0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 monclient: dump: 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.305 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fda8803e5b0 0x7fda88040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa6d9d640 10 monclient: get_auth_request con 0x7fda8803e5b0 auth_method 0 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fda9000fc80 con 0x7fdaa81d1bf0 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:34.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 msgr2=0x7fdaa8126dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 0x7fdaa8126dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaa81285d0 tx=0x7fda9000d590 comp rx=0 tx=0).stop 2024-09-17T16:46:34.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 msgr2=0x7fdaa8126dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa81d1bf0 0x7fdaa8126dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fdaa6d9d640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fda8803e5b0 0x7fda88040a70 secure :-1 s=READY pgs=3668 cs=0 l=1 rev1=1 crypto rx=0x7fda9801c380 tx=0x7fda980a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.278+0000 7fda9effd640 10 client.74911.objecter ms_handle_connect 0x7fda8803e5b0 2024-09-17T16:46:34.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaacb64640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaacb64640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 0x7fdaa802d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaacb64640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdaa81d1fd0 con 0x7fdaa802d450 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 0x7fdaa802d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient: get_auth_request con 0x7fdaa802d450 auth_method 0 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.282+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaa5d9b640 10 monclient(hunting): handle_auth_done global_id 94380 payload 824 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaa5d9b640 1 -- 172.21.15.105:0/1568704925 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdaa8140a60 con 0x7fdaa802d450 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaa5d9b640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 0x7fdaa802d880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda9001a000 tx=0x7fda900046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 client.74911.objecter ms_handle_connect 0x7fdaa802d450 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 client.74911.objecter resend_mon_ops 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda9000fe30 con 0x7fdaa802d450 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 monclient: dump: 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.311 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 <== mon.6 v2:172.21.15.105:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7fda9000fe30 con 0x7fdaa802d450 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 msgr2=0x7fdaa802d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 0x7fdaa802d880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda9001a000 tx=0x7fda900046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 msgr2=0x7fdaa802d880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fda9effd640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdaa802d450 0x7fdaa802d880 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda9001a000 tx=0x7fda900046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 170131, 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:34.313 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:34.314 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:34.315 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:34.316 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.317 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.318 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:34.319 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.320 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.321 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.325 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.327 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.328 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:34.329 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.330 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.331 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.334 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.335 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:34.335 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 10 librados: watch_flush enter 2024-09-17T16:46:34.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 10 librados: watch_flush exit 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 20 client.74911.objecter shutdown clearing up homeless session... 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 10 client.74911.objecter successfully canceled tick 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fda8803e5b0 msgr2=0x7fda88040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fda8803e5b0 0x7fda88040a70 secure :-1 s=READY pgs=3668 cs=0 l=1 rev1=1 crypto rx=0x7fda9801c380 tx=0x7fda980a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 10 monclient: shutdown 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 msgr2=0x7fdaa812a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 0x7fdaa812a6e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000cb90 tx=0x7fdaa000a430 comp rx=0 tx=0).stop 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaa6d9d640 1 -- 172.21.15.105:0/1568704925 reap_dead start 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 shutdown_connections 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 --2- 172.21.15.105:0/1568704925 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fdaa80a5f70 0x7fdaa812a6e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 >> 172.21.15.105:0/1568704925 conn(0x7fdaa80aa450 msgr2=0x7fdaa809fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 shutdown_connections 2024-09-17T16:46:34.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 -- 172.21.15.105:0/1568704925 wait complete. 2024-09-17T16:46:34.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.286+0000 7fdaacb64640 1 librados: shutdown 2024-09-17T16:46:34.338 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:34.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-17T16:46:34.338 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-17T16:46:34.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: get_monmap_and_config 2024-09-17T16:46:34.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: monmap: 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:34.453030+0000 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:34.453030+0000 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.457 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:34.458 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 Processor -- start 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 -- start start 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: init 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient: _add_conns ranks=[3,5,7,10,6,8,1,4,2,0,9] 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 0x7f1fb00a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-g con 0x7f1fb00a4750 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 0x7f1fb00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-h con 0x7f1fb006cb00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:34.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-i con 0x7f1fb0139530 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00a6610 con 0x7f1fb00a4750 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f1fb00a55b0 con 0x7f1fb006cb00 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f1fb006c7f0 con 0x7f1fb0139530 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb8131640 10 monclient(hunting): authenticate will time out at 2077.718753s 2024-09-17T16:46:34.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 0x7f1fb00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:59646/0 (socket says 172.21.15.105:59646) 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 1 -- 172.21.15.105:0/4280611788 learned_addr learned my addr 172.21.15.105:0/4280611788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): get_auth_request con 0x7f1fb0139530 auth_method 0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 0x7f1fb00a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): get_auth_request con 0x7f1fb006cb00 auth_method 0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request con 0x7f1fb00a4750 auth_method 0 2024-09-17T16:46:34.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_done global_id 104108 payload 274 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 -- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 msgr2=0x7f1fb0139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 -- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 msgr2=0x7f1fb00a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 0x7f1fb00a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 monclient: found mon.noname-h 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.160:3302/0 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb56cc640 1 -- 172.21.15.105:0/4280611788 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb006c1f0 con 0x7f1fb006cb00 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 0x7f1fb00a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.450+0000 7f1fb66ce640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 0x7f1fb00a5070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009dc0 tx=0x7f1f9c01c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=525dbb4945f01ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 1 -- 172.21.15.105:0/4280611788 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f9c0ac070 con 0x7f1fb006cb00 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: dump: 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.473 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 1 -- 172.21.15.105:0/4280611788 <== mon.5 v2:172.21.15.160:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0a68b0 con 0x7f1fb006cb00 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 5 monclient: authenticate success, global_id 104108 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:34.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 1 -- 172.21.15.105:0/4280611788 <== mon.5 v2:172.21.15.160:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f9c0a7420 con 0x7f1fb006cb00 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb4ecb640 10 monclient: dump: 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.475 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 4 set_mon_vals no callback set 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 10 monclient: shutdown 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 -- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 msgr2=0x7f1fb00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 0x7f1fb00a5070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009dc0 tx=0x7f1f9c01c850 comp rx=0 tx=0).stop 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 -- 172.21.15.105:0/4280611788 shutdown_connections 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f1fb0139530 0x7f1fb0139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f1fb006cb00 0x7f1fb00a5070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 --2- 172.21.15.105:0/4280611788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb00a4750 0x7f1fb00a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 -- 172.21.15.105:0/4280611788 >> 172.21.15.105:0/4280611788 conn(0x7f1fb00a93d0 msgr2=0x7f1fb00a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 -- 172.21.15.105:0/4280611788 shutdown_connections 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 1 -- 172.21.15.105:0/4280611788 wait complete. 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.454+0000 7f1fb8131640 10 monclient: monmap: 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:34.477 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 librados: starting msgr at 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 librados: starting objecter 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 Processor -- start 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- start start 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 librados: setting wanted keys 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 librados: calling monclient init 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: init 2024-09-17T16:46:34.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _add_conns ranks=[3,6,4,2,0,7,1,5,8] 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb006cb00 0x7f1fb01224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-g con 0x7f1fb006cb00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 0x7f1fb01d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-d con 0x7f1fb00a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1fb0139530 0x7f1fb011feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): picked mon.noname-c con 0x7f1fb0139530 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb01433a0 con 0x7f1fb006cb00 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f1fb0143220 con 0x7f1fb0139530 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f1fb0143520 con 0x7f1fb00a4750 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): authenticate will time out at 2077.725069s 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 0x7f1fb01d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 0x7f1fb01d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54828/0 (socket says 172.21.15.105:54828) 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 -- 172.21.15.105:0/2520957075 learned_addr learned my addr 172.21.15.105:0/2520957075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): get_auth_request con 0x7f1fb00a4750 auth_method 0 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1fb0139530 0x7f1fb011feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): get_auth_request con 0x7f1fb0139530 auth_method 0 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb006cb00 0x7f1fb01224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request con 0x7f1fb006cb00 auth_method 0 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient(hunting): handle_auth_done global_id 94389 payload 824 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1fb0139530 msgr2=0x7f1fb011feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f1fb0139530 0x7f1fb011feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb006cb00 msgr2=0x7f1fb01224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1fb006cb00 0x7f1fb01224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 monclient: found mon.noname-d 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb0123cb0 con 0x7f1fb00a4750 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb56cc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 0x7f1fb01d1b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009d60 tx=0x7f1f9c0195f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 client.?.objecter ms_handle_connect 0x7f1fb00a4750 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f9c0ac070 con 0x7f1fb00a4750 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: dump: 2024-09-17T16:46:34.482 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.483 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c003e10 con 0x7f1fb00a4750 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 5 monclient: authenticate success, global_id 94389 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _renew_subs 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb013f9e0 con 0x7f1fb00a4750 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f9c0a66b0 con 0x7f1fb00a4750 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:34.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1faffff640 4 set_mon_vals no callback set 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 client.94389.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: dump: 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.485 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _renew_subs 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb0122e40 con 0x7f1fb00a4750 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 librados: init done 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 0x7f1fb01203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1fb0051e80 con 0x7f1fb0123410 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 0x7f1fb01203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient: get_auth_request con 0x7f1fb0123410 auth_method 0 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f1f9c0af070 con 0x7f1fb00a4750 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_done global_id 55133 payload 824 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fb0139930 con 0x7f1fb0123410 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1f9803e580 0x7f1f98040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f1f9c0b7070 con 0x7f1fb00a4750 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 client.94389.objecter ms_dispatch 0x7f1fb0000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 3 client.94389.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:34.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 3 client.94389.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 0x7f1fb01203f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa0011ac0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 20 client.94389.objecter dump_active .. 0 homeless 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 client.94389.objecter ms_handle_connect 0x7f1fb0123410 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 client.94389.objecter resend_mon_ops 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa000fbb0 con 0x7f1fb0123410 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: dump: 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.488 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1f9803e580 0x7f1f98040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fb66ce640 10 monclient: get_auth_request con 0x7f1f9803e580 auth_method 0 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1fa000f740 con 0x7f1fb0123410 2024-09-17T16:46:34.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 msgr2=0x7f1fb01203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 0x7f1fb01203f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa0011ac0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 msgr2=0x7f1fb01203f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.458+0000 7f1fae7fc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0123410 0x7f1fb01203f0 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa0011ac0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.462+0000 7f1fae7fc640 10 client.94389.objecter ms_handle_connect 0x7f1f9803e580 2024-09-17T16:46:34.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.462+0000 7f1fb66ce640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1f9803e580 0x7f1f98040a40 secure :-1 s=READY pgs=3669 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80096b0 tx=0x7f1fa8009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 0x7f1fb013c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f1fb0120d10 con 0x7f1fb0120930 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 0x7f1fb013c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient: get_auth_request con 0x7f1fb0120930 auth_method 0 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 10 monclient(hunting): handle_auth_done global_id 55142 payload 824 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 1 -- 172.21.15.105:0/2520957075 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1fb0121950 con 0x7f1fb0120930 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 client.94389.objecter ms_handle_connect 0x7f1fb0120930 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 client.94389.objecter resend_mon_ops 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb5ecd640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 0x7f1fb013c7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa00046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa0004490 con 0x7f1fb0120930 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:34.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 monclient: dump: 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.494 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 <== mon.8 v2:172.21.15.105:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9457 (secure 0 0 0) 0x7f1fa0005f70 con 0x7f1fb0120930 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 msgr2=0x7f1fb013c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 0x7f1fb013c7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa00046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 msgr2=0x7f1fb013c7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fae7fc640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f1fb0120930 0x7f1fb013c7d0 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011600 tx=0x7f1fa00046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.495 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "leader", 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 701464, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:34.496 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.497 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.498 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:34.499 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.500 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:34.501 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.502 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:34.503 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.504 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.505 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.506 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.507 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.509 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.510 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.511 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.512 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.514 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.515 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.516 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": [ 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.517 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 3 2024-09-17T16:46:34.518 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.518 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.518 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 librados: watch_flush enter 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 librados: watch_flush exit 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 20 client.94389.objecter shutdown clearing up homeless session... 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 client.94389.objecter successfully canceled tick 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1f9803e580 msgr2=0x7f1f98040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f1f9803e580 0x7f1f98040a40 secure :-1 s=READY pgs=3669 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80096b0 tx=0x7f1fa8009240 comp rx=0 tx=0).stop 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 10 monclient: shutdown 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 msgr2=0x7f1fb01d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 --2- 172.21.15.105:0/2520957075 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f1fb00a4750 0x7f1fb01d1b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009d60 tx=0x7f1f9c0195f0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb66ce640 1 -- 172.21.15.105:0/2520957075 reap_dead start 2024-09-17T16:46:34.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb66ce640 1 -- 172.21.15.105:0/2520957075 reap_dead start 2024-09-17T16:46:34.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 shutdown_connections 2024-09-17T16:46:34.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 >> 172.21.15.105:0/2520957075 conn(0x7f1fb00a93d0 msgr2=0x7f1fb00a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 shutdown_connections 2024-09-17T16:46:34.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 -- 172.21.15.105:0/2520957075 wait complete. 2024-09-17T16:46:34.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.466+0000 7f1fb8131640 1 librados: shutdown 2024-09-17T16:46:34.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:34.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-17T16:46:34.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: get_monmap_and_config 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: monmap: 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:34.640199+0000 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:34.640199+0000 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:34.645 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 Processor -- start 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 -- start start 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: init 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient: _add_conns ranks=[5,2,10,0,9,1,3,4,7,6,8] 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8a40a6a30 0x7fe8a40a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-h con 0x7fe8a40a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-b con 0x7fe8a40a4ab0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe8a41397f0 0x7fe8a4139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-f con 0x7fe8a41397f0 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a40a7c40 con 0x7fe8a41397f0 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a40a7890 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe8a40a4080 con 0x7fe8a40a6a30 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a9e9b640 10 monclient(hunting): authenticate will time out at 2077.906059s 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49472/0 (socket says 172.21.15.105:49472) 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/2500572840 learned_addr learned my addr 172.21.15.105:0/2500572840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): get_auth_request con 0x7fe8a40a4ab0 auth_method 0 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8a40a6a30 0x7fe8a40a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a3fff640 1 --2- 172.21.15.105:0/2500572840 >> v2:172.21.15.160:3300/0 conn(0x7fe8a41397f0 0x7fe8a4139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_done global_id 84620 payload 274 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/2500572840 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8a40a6a30 msgr2=0x7fe8a40a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8a40a6a30 0x7fe8a40a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/2500572840 >> v2:172.21.15.160:3300/0 conn(0x7fe8a41397f0 msgr2=0x7fe8a4139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/2500572840 >> v2:172.21.15.160:3300/0 conn(0x7fe8a41397f0 0x7fe8a4139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 monclient: found mon.noname-b 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:34.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.638+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/2500572840 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a40a37e0 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c034c65c92e3de server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 1 -- 172.21.15.105:0/2500572840 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8980b1070 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: dump: 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.657 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 1 -- 172.21.15.105:0/2500572840 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8980a8050 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 5 monclient: authenticate success, global_id 84620 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 1 -- 172.21.15.105:0/2500572840 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8980abcc0 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 4 set_mon_vals no callback set 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a27fc640 10 monclient: dump: 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.660 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: shutdown 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/2500572840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 msgr2=0x7fe8a40a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89801c480 comp rx=0 tx=0).stop 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/2500572840 shutdown_connections 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/2500572840 >> v2:172.21.15.160:3300/0 conn(0x7fe8a41397f0 0x7fe8a4139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe8a40a4ab0 0x7fe8a40a7350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/2500572840 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe8a40a6a30 0x7fe8a40a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/2500572840 >> 172.21.15.105:0/2500572840 conn(0x7fe8a40a8d00 msgr2=0x7fe8a40a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/2500572840 shutdown_connections 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/2500572840 wait complete. 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: monmap: 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.662 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 librados: starting msgr at 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 librados: starting objecter 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 Processor -- start 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- start start 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 librados: setting wanted keys 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 librados: calling monclient init 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: init 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient: _add_conns ranks=[4,0,6,7,2,1,3,8,5] 2024-09-17T16:46:34.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe8a40a4ab0 0x7fe8a4138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-c con 0x7fe8a40a4ab0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 0x7fe8a4138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-a con 0x7fe8a40a6a30 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe8a41397f0 0x7fe8a413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): picked mon.noname-d con 0x7fe8a41397f0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a406e770 con 0x7fe8a40a6a30 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe8a413a5a0 con 0x7fe8a40a4ab0 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe8a4143260 con 0x7fe8a41397f0 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a9e9b640 10 monclient(hunting): authenticate will time out at 2077.911200s 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 0x7fe8a4138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe8a40a4ab0 0x7fe8a4138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe8a41397f0 0x7fe8a413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 0x7fe8a4138750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51416/0 (socket says 172.21.15.105:51416) 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/276361244 learned_addr learned my addr 172.21.15.105:0/276361244 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): get_auth_request con 0x7fe8a40a6a30 auth_method 0 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): get_auth_request con 0x7fe8a40a4ab0 auth_method 0 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): get_auth_request con 0x7fe8a41397f0 auth_method 0 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient(hunting): handle_auth_done global_id 104175 payload 824 2024-09-17T16:46:34.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe8a41397f0 msgr2=0x7fe8a413d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe8a41397f0 0x7fe8a413d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe8a40a4ab0 msgr2=0x7fe8a4138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe8a40a4ab0 0x7fe8a4138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 monclient: found mon.noname-a 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a2ffd640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a413fb30 con 0x7fe8a40a6a30 2024-09-17T16:46:34.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.642+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe8a40a4ab0 0x7fe8a4138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a2ffd640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 0x7fe8a4138750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8980af040 tx=0x7fe898019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.?.objecter ms_handle_connect 0x7fe8a40a6a30 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8980ba070 con 0x7fe8a40a6a30 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: dump: 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.668 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe898003dc0 con 0x7fe8a40a6a30 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 5 monclient: authenticate success, global_id 104175 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 monclient: _renew_subs 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a4124010 con 0x7fe8a40a6a30 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8980b1040 con 0x7fe8a40a6a30 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: dump: 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.670 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a1ffb640 4 set_mon_vals no callback set 2024-09-17T16:46:34.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 client.104175.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 monclient: _renew_subs 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a40abc90 con 0x7fe8a40a6a30 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 1 librados: init done 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 0x7fe8a413e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a4051e80 con 0x7fe8a41239b0 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe898003950 con 0x7fe8a40a6a30 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe89003e550 0x7fe890040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fe8980e90a0 con 0x7fe8a40a6a30 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.104175.objecter ms_dispatch 0x7fe8a4000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:34.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 3 client.104175.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 3 client.104175.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 0x7fe8a413e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient: get_auth_request con 0x7fe8a41239b0 auth_method 0 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 20 client.104175.objecter dump_active .. 0 homeless 2024-09-17T16:46:34.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a3fff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe89003e550 0x7fe890040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a3fff640 10 monclient: get_auth_request con 0x7fe89003e550 auth_method 0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_done global_id 104131 payload 824 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8a4139c50 con 0x7fe8a41239b0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.104175.objecter ms_handle_connect 0x7fe8a41239b0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.104175.objecter resend_mon_ops 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 0x7fe8a413e450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe89400d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 client.104175.objecter ms_handle_connect 0x7fe89003e550 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe8a3fff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe89003e550 0x7fe890040a10 secure :-1 s=READY pgs=3670 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4139040 tx=0x7fe89c0075a0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe89400fce0 con 0x7fe8a41239b0 2024-09-17T16:46:34.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: dump: 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.675 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe89400fce0 con 0x7fe8a41239b0 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 msgr2=0x7fe8a413e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 0x7fe8a413e450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe89400d590 comp rx=0 tx=0).stop 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 msgr2=0x7fe8a413e450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.646+0000 7fe88bfff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a41239b0 0x7fe8a413e450 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe89400d590 comp rx=0 tx=0).stop 2024-09-17T16:46:34.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a9e9b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 0x7fe8a402d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a9e9b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a406a200 con 0x7fe8a402d450 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 0x7fe8a402d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient: get_auth_request con 0x7fe8a402d450 auth_method 0 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 10 monclient(hunting): handle_auth_done global_id 104140 payload 824 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 1 -- 172.21.15.105:0/276361244 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe8a40a1120 con 0x7fe8a402d450 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe8a37fe640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 0x7fe8a402d860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe8940046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 client.104175.objecter ms_handle_connect 0x7fe8a402d450 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 client.104175.objecter resend_mon_ops 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8940067b0 con 0x7fe8a402d450 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 monclient: dump: 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 <== mon.1 v2:172.21.15.160:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7fe894006340 con 0x7fe8a402d450 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:34.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 msgr2=0x7fe8a402d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 0x7fe8a402d860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe8940046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 msgr2=0x7fe8a402d860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.654+0000 7fe88bfff640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8a402d450 0x7fe8a402d860 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fbf0 tx=0x7fe8940046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 29203, 2024-09-17T16:46:34.682 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:34.683 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.684 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.685 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:34.686 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.687 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.689 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.690 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.692 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.695 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.696 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:34.698 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.699 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.700 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.701 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.702 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.703 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:34.704 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:34.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 10 librados: watch_flush enter 2024-09-17T16:46:34.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 10 librados: watch_flush exit 2024-09-17T16:46:34.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 20 client.104175.objecter shutdown clearing up homeless session... 2024-09-17T16:46:34.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 10 client.104175.objecter successfully canceled tick 2024-09-17T16:46:34.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe89003e550 msgr2=0x7fe890040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe89003e550 0x7fe890040a10 secure :-1 s=READY pgs=3670 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4139040 tx=0x7fe89c0075a0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 10 monclient: shutdown 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 msgr2=0x7fe8a4138750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 --2- 172.21.15.105:0/276361244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a40a6a30 0x7fe8a4138750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe8980af040 tx=0x7fe898019220 comp rx=0 tx=0).stop 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a3fff640 1 -- 172.21.15.105:0/276361244 reap_dead start 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a3fff640 1 -- 172.21.15.105:0/276361244 reap_dead start 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 shutdown_connections 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 >> 172.21.15.105:0/276361244 conn(0x7fe8a40a8d00 msgr2=0x7fe8a413fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 shutdown_connections 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 -- 172.21.15.105:0/276361244 wait complete. 2024-09-17T16:46:34.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.658+0000 7fe8a9e9b640 1 librados: shutdown 2024-09-17T16:46:34.708 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:34.708 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-17T16:46:34.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: get_monmap_and_config 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: monmap: 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:34.820459+0000 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:34.820459+0000 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.824 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:34.825 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 Processor -- start 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 -- start start 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: init 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient: _add_conns ranks=[1,2,3,0,8,9,5,6,10,7,4] 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8e840a68e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-b con 0x7f8e840a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e840a73d0 0x7f8e84141040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-g con 0x7f8e840a73d0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8406ebc0 con 0x7f8e840a68e0 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e840a3fc0 con 0x7f8e840a4910 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8e840a63c0 con 0x7f8e840a73d0 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e88929640 10 monclient(hunting): authenticate will time out at 2078.086177s 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49480/0 (socket says 172.21.15.105:49480) 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 -- 172.21.15.105:0/548974124 learned_addr learned my addr 172.21.15.105:0/548974124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): get_auth_request con 0x7f8e840a4910 auth_method 0 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e82d9d640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e840a73d0 0x7f8e84141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e82d9d640 10 monclient(hunting): get_auth_request con 0x7f8e840a73d0 auth_method 0 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_done global_id 84629 payload 274 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 -- 172.21.15.105:0/548974124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e840a73d0 msgr2=0x7f8e84141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e840a73d0 0x7f8e84141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 -- 172.21.15.105:0/548974124 >> v1:172.21.15.160:6789/0 conn(0x7f8e840a68e0 legacy=0x7f8e840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 monclient: found mon.noname-b 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.818+0000 7f8e81d9b640 1 -- 172.21.15.105:0/548974124 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840a3690 con 0x7f8e840a4910 2024-09-17T16:46:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8e780098a0 tx=0x7f8e7801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a9da02b1a874e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 1 -- 172.21.15.105:0/548974124 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e780b2070 con 0x7f8e840a4910 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: dump: 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.832 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 1 -- 172.21.15.105:0/548974124 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e780a9070 con 0x7f8e840a4910 2024-09-17T16:46:34.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 5 monclient: authenticate success, global_id 84629 2024-09-17T16:46:34.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:34.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 1 -- 172.21.15.105:0/548974124 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e780abc40 con 0x7f8e840a4910 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 4 set_mon_vals no callback set 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8159a640 10 monclient: dump: 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.835 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.836 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: shutdown 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- 172.21.15.105:0/548974124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 msgr2=0x7f8e8413eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8e780098a0 tx=0x7f8e7801c410 comp rx=0 tx=0).stop 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- 172.21.15.105:0/548974124 shutdown_connections 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e840a73d0 0x7f8e84141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- 172.21.15.105:0/548974124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a4910 0x7f8e8413eaf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- 172.21.15.105:0/548974124 >> 172.21.15.105:0/548974124 conn(0x7f8e84068ba0 msgr2=0x7f8e84068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- 172.21.15.105:0/548974124 shutdown_connections 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- 172.21.15.105:0/548974124 wait complete. 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: build_initial_monmap 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: monmap: 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:34.844 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 librados: starting msgr at 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 librados: starting objecter 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 Processor -- start 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- start start 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 librados: setting wanted keys 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 librados: calling monclient init 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: init 2024-09-17T16:46:34.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient: _add_conns ranks=[7,2,8,6,1,4,3,5,0] 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e840a4910 0x7f8e84128a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-i con 0x7f8e840a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 0x7f8e841d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-b con 0x7f8e840a68e0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 0x7f8e84126470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): picked mon.noname-e con 0x7f8e840a73d0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e84143f20 con 0x7f8e840a68e0 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8e84143da0 con 0x7f8e840a4910 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f8e841440a0 con 0x7f8e840a73d0 2024-09-17T16:46:34.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e88929640 10 monclient(hunting): authenticate will time out at 2078.091242s 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 0x7f8e84126470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 0x7f8e841d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 0x7f8e841d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49490/0 (socket says 172.21.15.105:49490) 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 0x7f8e84126470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51206/0 (socket says 172.21.15.105:51206) 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 -- 172.21.15.105:0/185007437 learned_addr learned my addr 172.21.15.105:0/185007437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): get_auth_request con 0x7f8e840a68e0 auth_method 0 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): get_auth_request con 0x7f8e840a73d0 auth_method 0 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e840a4910 0x7f8e84128a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient(hunting): handle_auth_done global_id 84638 payload 824 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 msgr2=0x7f8e84126470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 0x7f8e84126470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e840a4910 msgr2=0x7f8e84128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e840a4910 0x7f8e84128a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 monclient: found mon.noname-b 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840a25a0 con 0x7f8e840a68e0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e82d9d640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f8e840a73d0 0x7f8e84126470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8259c640 10 monclient: get_auth_request con 0x7f8e840a4910 auth_method 0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8e840a4910 0x7f8e84128a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e81d9b640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 0x7f8e841d1620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e7801aac0 tx=0x7f8e780191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e72ffd640 10 client.?.objecter ms_handle_connect 0x7f8e840a68e0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.822+0000 7f8e72ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e780bb070 con 0x7f8e840a68e0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:34.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: dump: 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.850 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e78003b90 con 0x7f8e840a68e0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 5 monclient: authenticate success, global_id 84638 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 monclient: _renew_subs 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e780b2040 con 0x7f8e840a68e0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e84129f60 con 0x7f8e840a68e0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: dump: 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.852 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e80d99640 4 set_mon_vals no callback set 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 client.84638.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 monclient: _renew_subs 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8406e0b0 con 0x7f8e840a68e0 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 1 librados: init done 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 0x7f8e841269b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8e841c8f30 con 0x7f8e841d1b60 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8e780ab070 con 0x7f8e840a68e0 2024-09-17T16:46:34.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 0x7f8e841269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e6403e550 0x7f8e64040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient: get_auth_request con 0x7f8e841d1b60 auth_method 0 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f8e780e8ba0 con 0x7f8e840a68e0 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 client.84638.objecter ms_dispatch 0x7f8e84000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 3 client.84638.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 3 client.84638.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 20 client.84638.objecter dump_active .. 0 homeless 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e82d9d640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e6403e550 0x7f8e64040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e82d9d640 10 monclient: get_auth_request con 0x7f8e6403e550 auth_method 0 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 10 monclient(hunting): handle_auth_done global_id 55083 payload 824 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e8406db60 con 0x7f8e841d1b60 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 0x7f8e841269b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c00dd20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 client.84638.objecter ms_handle_connect 0x7f8e841d1b60 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 client.84638.objecter resend_mon_ops 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e7c003bb0 con 0x7f8e841d1b60 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: dump: 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.855 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 client.84638.objecter ms_handle_connect 0x7f8e6403e550 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e82d9d640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e6403e550 0x7f8e64040a10 secure :-1 s=READY pgs=3671 cs=0 l=1 rev1=1 crypto rx=0x7f8e840aae20 tx=0x7f8e74008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8e7c003bb0 con 0x7f8e841d1b60 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 msgr2=0x7f8e841269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 0x7f8e841269b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c00dd20 comp rx=0 tx=0).stop 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 msgr2=0x7f8e841269b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.826+0000 7f8e72ffd640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e841d1b60 0x7f8e841269b0 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c00dd20 comp rx=0 tx=0).stop 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e88929640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e88929640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 0x7f8e84122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e88929640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8e84127300 con 0x7f8e84126ef0 2024-09-17T16:46:34.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 0x7f8e84122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient: get_auth_request con 0x7f8e84126ef0 auth_method 0 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 10 monclient(hunting): handle_auth_done global_id 55092 payload 824 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 1 -- 172.21.15.105:0/185007437 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e840a6cc0 con 0x7f8e84126ef0 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 client.84638.objecter ms_handle_connect 0x7f8e84126ef0 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 client.84638.objecter resend_mon_ops 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e8259c640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 0x7f8e84122530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c001260 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:34.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e7c0054e0 con 0x7f8e84126ef0 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 monclient: dump: 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:34.859 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 <== mon.3 v2:172.21.15.160:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f8e7c005070 con 0x7f8e84126ef0 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 msgr2=0x7f8e84122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 0x7f8e84122530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c001260 comp rx=0 tx=0).stop 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 msgr2=0x7f8e84122530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:34.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e72ffd640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e84126ef0 0x7f8e84122530 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d860 tx=0x7f8e7c001260 comp rx=0 tx=0).stop 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 701843, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:34.861 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 3, 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.862 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.863 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.864 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.864 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.864 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.864 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.864 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:34.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:34.866 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.867 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:34.868 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:34.869 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.870 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.872 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.874 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:34.875 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:34.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.877 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:34.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.880 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:34.883 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:34.884 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:34.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:34.886 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:34.886 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.834+0000 7f8e88929640 10 librados: watch_flush enter 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 10 librados: watch_flush exit 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 20 client.84638.objecter shutdown clearing up homeless session... 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 10 client.84638.objecter successfully canceled tick 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e6403e550 msgr2=0x7f8e64040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8e6403e550 0x7f8e64040a10 secure :-1 s=READY pgs=3671 cs=0 l=1 rev1=1 crypto rx=0x7f8e840aae20 tx=0x7f8e74008040 comp rx=0 tx=0).stop 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 10 monclient: shutdown 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 msgr2=0x7f8e841d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 --2- 172.21.15.105:0/185007437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e840a68e0 0x7f8e841d1620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e7801aac0 tx=0x7f8e780191b0 comp rx=0 tx=0).stop 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e82d9d640 1 -- 172.21.15.105:0/185007437 reap_dead start 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e82d9d640 1 -- 172.21.15.105:0/185007437 reap_dead start 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 shutdown_connections 2024-09-17T16:46:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 >> 172.21.15.105:0/185007437 conn(0x7f8e84068ba0 msgr2=0x7f8e8409fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 shutdown_connections 2024-09-17T16:46:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 -- 172.21.15.105:0/185007437 wait complete. 2024-09-17T16:46:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:34.838+0000 7f8e88929640 1 librados: shutdown 2024-09-17T16:46:34.889 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:34.889 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-17T16:46:34.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: get_monmap_and_config 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: monmap: 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:35.008580+0000 2024-09-17T16:46:35.012 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:35.008580+0000 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.013 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:35.014 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 Processor -- start 2024-09-17T16:46:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 -- start start 2024-09-17T16:46:35.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: init 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient: _add_conns ranks=[8,3,4,9,1,2,5,7,0,10,6] 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-e con 0x7f42380a4750 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423806cb00 0x7f42380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-g con 0x7f423806cb00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-c con 0x7f4238139530 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f42380a6610 con 0x7f423806cb00 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f42380a55b0 con 0x7f4238139530 2024-09-17T16:46:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f423806c7f0 con 0x7f42380a4750 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423f40a640 10 monclient(hunting): authenticate will time out at 2078.274412s 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51208/0 (socket says 172.21.15.105:51208) 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34478/0 (socket says 172.21.15.105:34478) 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 1 -- 172.21.15.105:0/1636042148 learned_addr learned my addr 172.21.15.105:0/1636042148 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): get_auth_request con 0x7f42380a4750 auth_method 0 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): get_auth_request con 0x7f4238139530 auth_method 0 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423c9a5640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423806cb00 0x7f42380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423c9a5640 10 monclient(hunting): get_auth_request con 0x7f423806cb00 auth_method 0 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423c9a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423c9a5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423c9a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient(hunting): handle_auth_done global_id 74938 payload 274 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 -- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 msgr2=0x7f42380a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 -- 172.21.15.105:0/1636042148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423806cb00 msgr2=0x7f42380a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423806cb00 0x7f42380a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 monclient: found mon.noname-c 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d9a7640 1 -- 172.21.15.105:0/1636042148 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423806c1f0 con 0x7f4238139530 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.006+0000 7f423d1a6640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4230009dc0 tx=0x7f423001c850 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ad4840a88ffba924 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 1 -- 172.21.15.105:0/1636042148 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42300ac070 con 0x7f4238139530 2024-09-17T16:46:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: dump: 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.022 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 1 -- 172.21.15.105:0/1636042148 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42300a68b0 con 0x7f4238139530 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 5 monclient: authenticate success, global_id 74938 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 1 -- 172.21.15.105:0/1636042148 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42300a7420 con 0x7f4238139530 2024-09-17T16:46:35.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 4 set_mon_vals no callback set 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f422ffff640 10 monclient: dump: 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.024 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: shutdown 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 msgr2=0x7f4238139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4230009dc0 tx=0x7f423001c850 comp rx=0 tx=0).stop 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- 172.21.15.105:0/1636042148 shutdown_connections 2024-09-17T16:46:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423806cb00 0x7f42380a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f4238139530 0x7f4238139930 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- 172.21.15.105:0/1636042148 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f42380a4750 0x7f42380a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- 172.21.15.105:0/1636042148 >> 172.21.15.105:0/1636042148 conn(0x7f42380a93d0 msgr2=0x7f42380a5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- 172.21.15.105:0/1636042148 shutdown_connections 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- 172.21.15.105:0/1636042148 wait complete. 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: monmap: 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:35.026 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.027 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 librados: starting msgr at 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 librados: starting objecter 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 Processor -- start 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- start start 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 librados: setting wanted keys 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 librados: calling monclient init 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: init 2024-09-17T16:46:35.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient: _add_conns ranks=[4,6,5,8,3,7,1,0,2] 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 0x7f423811fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-c con 0x7f423806cb00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f42380a4750 0x7f42381203d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-d con 0x7f42380a4750 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238139530 0x7f4238120910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): picked mon.noname-h con 0x7f4238139530 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f42381433a0 con 0x7f423806cb00 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4238143220 con 0x7f4238139530 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f4238143520 con 0x7f42380a4750 2024-09-17T16:46:35.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423f40a640 10 monclient(hunting): authenticate will time out at 2078.279622s 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 0x7f423811fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f42380a4750 0x7f42381203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 0x7f423811fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:34490/0 (socket says 172.21.15.105:34490) 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 1 -- 172.21.15.105:0/1783618974 learned_addr learned my addr 172.21.15.105:0/1783618974 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): get_auth_request con 0x7f423806cb00 auth_method 0 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): get_auth_request con 0x7f42380a4750 auth_method 0 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.010+0000 7f423d1a6640 10 monclient(hunting): handle_auth_done global_id 74947 payload 824 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f42380a4750 msgr2=0x7f42381203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f42380a4750 0x7f42381203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238139530 msgr2=0x7f4238120910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238139530 0x7f4238120910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238139530 0x7f4238120910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 monclient: found mon.noname-c 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4238123b20 con 0x7f423806cb00 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d1a6640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 0x7f423811fe90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f423400ea00 tx=0x7f423400eec0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.?.objecter ms_handle_connect 0x7f423806cb00 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4234017070 con 0x7f423806cb00 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: dump: 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.038 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4234013470 con 0x7f423806cb00 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 5 monclient: authenticate success, global_id 74947 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 monclient: _renew_subs 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42380a2fb0 con 0x7f423806cb00 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4234013a90 con 0x7f423806cb00 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: dump: 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.040 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.041 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422f7fe640 4 set_mon_vals no callback set 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 client.74947.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 monclient: _renew_subs 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f42381217c0 con 0x7f423806cb00 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 1 librados: init done 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 0x7f42381d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f4238051e80 con 0x7f4238123fb0 2024-09-17T16:46:35.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f423401a070 con 0x7f423806cb00 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f421c03e580 0x7f421c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 0x7f42381d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient: get_auth_request con 0x7f4238123fb0 auth_method 0 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f423400f040 con 0x7f423806cb00 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.74947.objecter ms_dispatch 0x7f4238000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 3 client.74947.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 3 client.74947.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 20 client.74947.objecter dump_active .. 0 homeless 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f421c03e580 0x7f421c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d9a7640 10 monclient: get_auth_request con 0x7f421c03e580 auth_method 0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 10 monclient(hunting): handle_auth_done global_id 104117 payload 824 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4238139930 con 0x7f4238123fb0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.74947.objecter ms_handle_connect 0x7f4238123fb0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423c9a5640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 0x7f42381d1440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f4228010cb0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.74947.objecter resend_mon_ops 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 client.74947.objecter ms_handle_connect 0x7f421c03e580 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f423d9a7640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f421c03e580 0x7f421c040a40 secure :-1 s=READY pgs=3672 cs=0 l=1 rev1=1 crypto rx=0x7f42300ae000 tx=0x7f42300a7000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4228018070 con 0x7f4238123fb0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: dump: 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f422800ea30 con 0x7f4238123fb0 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 msgr2=0x7f42381d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 0x7f42381d1440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f4228010cb0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 msgr2=0x7f42381d1440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.014+0000 7f422dffb640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f4238123fb0 0x7f42381d1440 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f4228010cb0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423f40a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423f40a640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 0x7f42381d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423f40a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f42381d22d0 con 0x7f42381d1980 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 0x7f42381d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient: get_auth_request con 0x7f42381d1980 auth_method 0 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 10 monclient(hunting): handle_auth_done global_id 104126 payload 824 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 1 -- 172.21.15.105:0/1783618974 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f423806e230 con 0x7f42381d1980 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f423c9a5640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 0x7f42381d1d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f42280046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 10 client.74947.objecter ms_handle_connect 0x7f42381d1980 2024-09-17T16:46:35.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 10 client.74947.objecter resend_mon_ops 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.5 v2:172.21.15.160:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f422802b020 con 0x7f42381d1980 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.022+0000 7f422dffb640 10 monclient: dump: 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.048 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 <== mon.5 v2:172.21.15.160:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9287 (secure 0 0 0) 0x7f4228014430 con 0x7f42381d1980 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 msgr2=0x7f42381d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 0x7f42381d1d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f42280046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 msgr2=0x7f42381d1d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:35.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f422dffb640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f42381d1980 0x7f42381d1d90 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42280107f0 tx=0x7f42280046d0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 29579, 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:35.050 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.051 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:35.052 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:35.054 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.055 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:35.056 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.057 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.058 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:35.059 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.061 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.063 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.065 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:35.067 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:35.069 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.070 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:35.071 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:35.072 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 10 librados: watch_flush enter 2024-09-17T16:46:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 10 librados: watch_flush exit 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 20 client.74947.objecter shutdown clearing up homeless session... 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 10 client.74947.objecter successfully canceled tick 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f421c03e580 msgr2=0x7f421c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f421c03e580 0x7f421c040a40 secure :-1 s=READY pgs=3672 cs=0 l=1 rev1=1 crypto rx=0x7f42300ae000 tx=0x7f42300a7000 comp rx=0 tx=0).stop 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 10 monclient: shutdown 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 msgr2=0x7f423811fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 --2- 172.21.15.105:0/1783618974 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f423806cb00 0x7f423811fe90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f423400ea00 tx=0x7f423400eec0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423d9a7640 1 -- 172.21.15.105:0/1783618974 reap_dead start 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423d9a7640 1 -- 172.21.15.105:0/1783618974 reap_dead start 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 shutdown_connections 2024-09-17T16:46:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 >> 172.21.15.105:0/1783618974 conn(0x7f42380a93d0 msgr2=0x7f42380a81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 shutdown_connections 2024-09-17T16:46:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 -- 172.21.15.105:0/1783618974 wait complete. 2024-09-17T16:46:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.026+0000 7f423f40a640 1 librados: shutdown 2024-09-17T16:46:35.076 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:35.076 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-17T16:46:35.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: get_monmap_and_config 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: monmap: 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:35.189145+0000 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:35.189145+0000 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.193 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:35.194 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 Processor -- start 2024-09-17T16:46:35.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 -- start start 2024-09-17T16:46:35.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: init 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient: _add_conns ranks=[10,3,0,5,8,9,7,6,2,1,4] 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8dd809b510 0x7f8dd809b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-f con 0x7f8dd809b510 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8dd80a25a0 0x7f8dd80a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-g con 0x7f8dd80a25a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8dd80a42c0 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd80a18e0 con 0x7f8dd80a42c0 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd809f3b0 con 0x7f8dd809b510 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80a1bc0 con 0x7f8dd80a25a0 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 10 monclient(hunting): authenticate will time out at 2078.455103s 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8dd80a42c0 0x7f8dd813d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54716/0 (socket says 172.21.15.105:54716) 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd77fe640 1 -- 172.21.15.105:0/1212260440 learned_addr learned my addr 172.21.15.105:0/1212260440 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8dd80a25a0 0x7f8dd80a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/1212260440 >> v2:172.21.15.160:3300/0 conn(0x7f8dd809b510 0x7f8dd809b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2395875700 0 0) 0x7f8dd80a18e0 con 0x7f8dd80a42c0 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): _negotiate 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): my global_id is 104184 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dc4003530 con 0x7f8dd80a42c0 2024-09-17T16:46:35.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request con 0x7f8dd80a25a0 auth_method 0 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request con 0x7f8dd809b510 auth_method 0 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3835060830 0 0) 0x7f8dc4003530 con 0x7f8dd80a42c0 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8dd80a25a0 msgr2=0x7f8dd80a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8dd80a25a0 0x7f8dd80a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 >> v2:172.21.15.160:3300/0 conn(0x7f8dd809b510 msgr2=0x7f8dd809b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 --2- 172.21.15.105:0/1212260440 >> v2:172.21.15.160:3300/0 conn(0x7f8dd809b510 0x7f8dd809b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 monclient: found mon.noname-a-legacy 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.105:6789/0 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80a5c20 con 0x7f8dd80a42c0 2024-09-17T16:46:35.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 2614046410 0 0) 0x7f8dc8001970 con 0x7f8dd80a42c0 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 5 monclient: authenticate success, global_id 104184 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8ddd6d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: dump: 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.211 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] but i'm connected to v1:172.21.15.105:6789/0, reconnecting 2024-09-17T16:46:35.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 >> v1:172.21.15.105:6789/0 conn(0x7f8dd80a42c0 legacy=0x7f8dd813d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient: _add_conns ranks=[7,0,5,1,4,6,8,3,2] 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004210 0x7f8dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): picked mon.i con 0x7f8dc4004210 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 0x7f8dc4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 10 monclient(hunting): picked mon.a con 0x7f8dc4004da0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.186+0000 7f8dd5ffb640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8dc4009ab0 0x7f8dc400bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): picked mon.h con 0x7f8dc4009ab0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd809f3b0 con 0x7f8dc4004da0 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f8dd80a1bc0 con 0x7f8dc4009ab0 2024-09-17T16:46:35.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8dc400c850 con 0x7f8dc4004210 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient(hunting): _finish_auth 0 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 0x7f8dc4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request con 0x7f8dc4004da0 auth_method 0 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004210 0x7f8dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request con 0x7f8dc4004210 auth_method 0 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8dc4009ab0 0x7f8dc400bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 10 monclient(hunting): get_auth_request con 0x7f8dc4009ab0 auth_method 0 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_done global_id 104184 payload 293 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 -- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004210 msgr2=0x7f8dc4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004210 0x7f8dc4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 -- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8dc4009ab0 msgr2=0x7f8dc400bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f8dc4009ab0 0x7f8dc400bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 monclient: found mon.a 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd77fe640 1 -- 172.21.15.105:0/1212260440 reap_dead start 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 -- 172.21.15.105:0/1212260440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc4003530 con 0x7f8dc4004da0 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 0x7f8dc4005220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806d390 tx=0x7f8dc001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31e376d3d1bd313 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc00096b0 con 0x7f8dc4004da0 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: dump: 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.217 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc00a8880 con 0x7f8dc4004da0 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 4 set_mon_vals no callback set 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 1 -- 172.21.15.105:0/1212260440 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc00b0400 con 0x7f8dc4004da0 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8dd5ffb640 10 monclient: dump: 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.219 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 10 monclient: shutdown 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 msgr2=0x7f8dc4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 0x7f8dc4005220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806d390 tx=0x7f8dc001c9d0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/1212260440 shutdown_connections 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/1212260440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dc4004da0 0x7f8dc4005220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/1212260440 >> 172.21.15.105:0/1212260440 conn(0x7f8dd80a4e70 msgr2=0x7f8dd80a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/1212260440 shutdown_connections 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.190+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/1212260440 wait complete. 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: monmap: 2024-09-17T16:46:35.220 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 librados: starting msgr at 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 librados: starting objecter 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 Processor -- start 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- start start 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 librados: setting wanted keys 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 librados: calling monclient init 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: init 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _add_conns ranks=[7,0,2,6,5,8,3,1,4] 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004da0 0x7f8dd81d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-i con 0x7f8dc4004da0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 0x7f8dd81d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-a con 0x7f8dd81d1d60 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd81ce500 0x7f8dd81ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): picked mon.noname-b con 0x7f8dd81ce500 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd809fc00 con 0x7f8dd81d1d60 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd809fa80 con 0x7f8dd81ce500 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8dd809fd80 con 0x7f8dc4004da0 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): authenticate will time out at 2078.460963s 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 0x7f8dd81d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd81ce500 0x7f8dd81ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 0x7f8dd81d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51442/0 (socket says 172.21.15.105:51442) 2024-09-17T16:46:35.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 -- 172.21.15.105:0/3901678892 learned_addr learned my addr 172.21.15.105:0/3901678892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request con 0x7f8dd81d1d60 auth_method 0 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): get_auth_request con 0x7f8dd81ce500 auth_method 0 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004da0 0x7f8dd81d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request con 0x7f8dc4004da0 auth_method 0 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient(hunting): handle_auth_done global_id 104193 payload 824 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004da0 msgr2=0x7f8dd81d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dc4004da0 0x7f8dd81d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd81ce500 msgr2=0x7f8dd81ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd81ce500 0x7f8dd81ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 monclient: found mon.noname-a 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80a73c0 con 0x7f8dd81d1d60 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd67fc640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 0x7f8dd81d21a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001c800 tx=0x7f8dc0019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 client.?.objecter ms_handle_connect 0x7f8dd81d1d60 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc00a9d50 con 0x7f8dd81d1d60 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: dump: 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.226 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 5 monclient: authenticate success, global_id 104193 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc00bb8b0 con 0x7f8dd81d1d60 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _renew_subs 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd8077960 con 0x7f8dd81d1d60 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dc00b0400 con 0x7f8dd81d1d60 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 client.104193.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 monclient: dump: 2024-09-17T16:46:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.229 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd57fa640 4 set_mon_vals no callback set 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _renew_subs 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd80966c0 con 0x7f8dd81d1d60 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 librados: init done 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 0x7f8dd81cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8dd8051e80 con 0x7f8dd8097860 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f8dc00bb440 con 0x7f8dd81d1d60 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 0x7f8dd81cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient: get_auth_request con 0x7f8dd8097860 auth_method 0 2024-09-17T16:46:35.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8db803e550 0x7f8db8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f8dc000f040 con 0x7f8dd81d1d60 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 client.104193.objecter ms_dispatch 0x7f8dd8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 3 client.104193.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 3 client.104193.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 20 client.104193.objecter dump_active .. 0 homeless 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8db803e550 0x7f8db8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd77fe640 10 monclient: get_auth_request con 0x7f8db803e550 auth_method 0 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_done global_id 64825 payload 824 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8dd8097620 con 0x7f8dd8097860 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 client.104193.objecter ms_handle_connect 0x7f8dd8097860 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dcf7fe640 10 client.104193.objecter resend_mon_ops 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.194+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 0x7f8dd81cee80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8052890 tx=0x7f8dd0012610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd001c020 con 0x7f8dd8097860 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 monclient: dump: 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.232 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 client.104193.objecter ms_handle_connect 0x7f8db803e550 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dd77fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8db803e550 0x7f8db8040a10 secure :-1 s=READY pgs=3673 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806f9a0 tx=0x7f8dc8007040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8dd0018470 con 0x7f8dd8097860 2024-09-17T16:46:35.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:35.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 msgr2=0x7f8dd81cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 0x7f8dd81cee80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8052890 tx=0x7f8dd0012610 comp rx=0 tx=0).stop 2024-09-17T16:46:35.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 msgr2=0x7f8dd81cee80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:35.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.198+0000 7f8dcf7fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8097860 0x7f8dd81cee80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8ddd6d0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 0x7f8dd8072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8ddd6d0640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f8dd80971c0 con 0x7f8dd8071ff0 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 0x7f8dd8072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient: get_auth_request con 0x7f8dd8071ff0 auth_method 0 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 10 monclient(hunting): handle_auth_done global_id 64834 payload 824 2024-09-17T16:46:35.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.202+0000 7f8dd6ffd640 1 -- 172.21.15.105:0/3901678892 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8dd806a1c0 con 0x7f8dd8071ff0 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dd6ffd640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 0x7f8dd8072400 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000b010 tx=0x7f8dd0004ac0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 client.104193.objecter ms_handle_connect 0x7f8dd8071ff0 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 client.104193.objecter resend_mon_ops 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd001c040 con 0x7f8dd8071ff0 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 monclient: dump: 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.237 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 <== mon.7 v2:172.21.15.160:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9288 (secure 0 0 0) 0x7f8dd0016d00 con 0x7f8dd8071ff0 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 msgr2=0x7f8dd8072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 0x7f8dd8072400 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000b010 tx=0x7f8dd0004ac0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 msgr2=0x7f8dd8072400 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-17T16:46:35.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dcf7fe640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f8dd8071ff0 0x7f8dd8072400 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "peon", 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "election_epoch": 118, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 568308, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum": [ 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: 0, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: 1, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: 2, 2024-09-17T16:46:35.239 INFO:teuthology.orchestra.run.smithi105.stdout: 3, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: 4, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: 5, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: 6, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: 7, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: 8 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_age": 4, 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "required_con": "2450521708007866372", 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "required_mon": [ 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.240 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_con": "4540719139924082687", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "quorum_mon": [ 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.241 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "outside_quorum": [], 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "extra_probe_peers": [], 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "sync_provider": [], 2024-09-17T16:46:35.242 INFO:teuthology.orchestra.run.smithi105.stdout: "monmap": { 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 1, 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "fsid": "4403e27b-1d4f-4d49-a333-3b7d5cfcb76b", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "modified": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "created": "2024-09-17T16:27:05.094380Z", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release": 19, 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "min_mon_release_name": "squid", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "election_strategy": 3, 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "disallowed_leaders: ": "", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false, 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "tiebreaker_mon": "", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "removed_ranks: ": "", 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "features": { 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "persistent": [ 2024-09-17T16:46:35.243 INFO:teuthology.orchestra.run.smithi105.stdout: "kraken", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "luminous", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "mimic", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap-prune", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "nautilus", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "elector-pinging", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "reef", 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "squid" 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "optional": [] 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.244 INFO:teuthology.orchestra.run.smithi105.stdout: "mons": [ 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 0, 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "a", 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3300", 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.245 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789", 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6789/0", 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6789/0", 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 1, 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "f", 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.246 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3300", 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789", 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.247 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6789/0", 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6789/0", 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 2, 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "b", 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3301", 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790", 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6790/0", 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6790/0", 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.249 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 3, 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "g", 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3301", 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.250 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790", 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6790/0", 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6790/0", 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 4, 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "c", 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.251 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3302", 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791", 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.252 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6791/0", 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6791/0", 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 5, 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "h", 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3302", 2024-09-17T16:46:35.253 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791", 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6791/0", 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6791/0", 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.254 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 6, 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "d", 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3303", 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792", 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.255 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6792/0", 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6792/0", 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 7, 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "i", 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:3303", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.160:6792/0", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.160:6792/0", 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.257 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": 8, 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "e", 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addrs": { 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:3304", 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-17T16:46:35.258 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793", 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 0 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6793/0", 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "public_addr": "172.21.15.105:6793/0", 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "priority": 0, 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "weight": 0, 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_location": "{}" 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.259 INFO:teuthology.orchestra.run.smithi105.stdout: "feature_map": { 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": [ 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "client": [ 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "features": "0x3f03dffffffdffff", 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "release": "squid", 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 1 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-17T16:46:35.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-17T16:46:35.261 INFO:teuthology.orchestra.run.smithi105.stdout: "stretch_mode": false 2024-09-17T16:46:35.261 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:35.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 10 librados: watch_flush enter 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 10 librados: watch_flush exit 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 20 client.104193.objecter shutdown clearing up homeless session... 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 10 client.104193.objecter successfully canceled tick 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8db803e550 msgr2=0x7f8db8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f8db803e550 0x7f8db8040a10 secure :-1 s=READY pgs=3673 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806f9a0 tx=0x7f8dc8007040 comp rx=0 tx=0).stop 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 10 monclient: shutdown 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 msgr2=0x7f8dd81d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 --2- 172.21.15.105:0/3901678892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd81d1d60 0x7f8dd81d21a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8dc001c800 tx=0x7f8dc0019770 comp rx=0 tx=0).stop 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dd77fe640 1 -- 172.21.15.105:0/3901678892 reap_dead start 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8dd77fe640 1 -- 172.21.15.105:0/3901678892 reap_dead start 2024-09-17T16:46:35.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 shutdown_connections 2024-09-17T16:46:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 >> 172.21.15.105:0/3901678892 conn(0x7f8dd80a4e70 msgr2=0x7f8dd8099fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 shutdown_connections 2024-09-17T16:46:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 -- 172.21.15.105:0/3901678892 wait complete. 2024-09-17T16:46:35.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.206+0000 7f8ddd6d0640 1 librados: shutdown 2024-09-17T16:46:35.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-17T16:46:35.264 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-17T16:46:35.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-17T16:46:35.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: get_monmap_and_config 2024-09-17T16:46:35.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: monmap: 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:35.376953+0000 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:35.376953+0000 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.381 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:35.382 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 Processor -- start 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 -- start start 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: init 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient: _add_conns ranks=[2,5,3,4,1,6,0,10,7,8,9] 2024-09-17T16:46:35.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-b con 0x7fd8cc0a3720 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 0x7fd8cc0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-h con 0x7fd8cc0a3b00 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8cc0a3f00 0x7fd8cc0a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-g con 0x7fd8cc0a3f00 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc06db50 con 0x7fd8cc0a3720 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc0a0e80 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fd8cc0a1490 con 0x7fd8cc0a3b00 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d3e44640 10 monclient(hunting): authenticate will time out at 2078.642900s 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49502/0 (socket says 172.21.15.105:49502) 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 -- 172.21.15.105:0/1367897882 learned_addr learned my addr 172.21.15.105:0/1367897882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3720 auth_method 0 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8cc0a3f00 0x7fd8cc0a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 0x7fd8cc0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3f00 auth_method 0 2024-09-17T16:46:35.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3b00 auth_method 0 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_done global_id 84656 payload 274 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 -- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 msgr2=0x7fd8cc0a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 0x7fd8cc0a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 -- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8cc0a3f00 msgr2=0x7fd8cc0a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:35.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8cc0a3f00 0x7fd8cc0a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 monclient: found mon.noname-b 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d1be0640 1 -- 172.21.15.105:0/1367897882 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc0a7240 con 0x7fd8cc0a3720 2024-09-17T16:46:35.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.374+0000 7fd8d13df640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 0x7fd8cc0a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0098a0 tx=0x7fd8bc01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94fb3c1ba09bbe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 1 -- 172.21.15.105:0/1367897882 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8bc0b0070 con 0x7fd8cc0a3720 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: dump: 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.393 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 1 -- 172.21.15.105:0/1367897882 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0aa810 con 0x7fd8cc0a3720 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 5 monclient: authenticate success, global_id 84656 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 1 -- 172.21.15.105:0/1367897882 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8bc0a9300 con 0x7fd8cc0a3720 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 4 set_mon_vals no callback set 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d0bde640 10 monclient: dump: 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.395 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.396 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 10 monclient: shutdown 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- 172.21.15.105:0/1367897882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 msgr2=0x7fd8cc0a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0098a0 tx=0x7fd8bc01c440 comp rx=0 tx=0).stop 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- 172.21.15.105:0/1367897882 shutdown_connections 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8cc0a3f00 0x7fd8cc0a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fd8cc0a3b00 0x7fd8cc0a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/1367897882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc0a1e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- 172.21.15.105:0/1367897882 >> 172.21.15.105:0/1367897882 conn(0x7fd8cc0aa390 msgr2=0x7fd8cc0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- 172.21.15.105:0/1367897882 shutdown_connections 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- 172.21.15.105:0/1367897882 wait complete. 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 10 monclient: build_initial_monmap 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 10 monclient: monmap: 2024-09-17T16:46:35.397 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:35.398 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 librados: starting msgr at 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 librados: starting objecter 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 Processor -- start 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 -- start start 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 librados: setting wanted keys 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 1 librados: calling monclient init 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.378+0000 7fd8d3e44640 10 monclient: init 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _add_conns ranks=[2,8,6,4,7,3,5,1,0] 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-b con 0x7fd8cc0a3720 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 0x7fd8cc1d1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-e con 0x7fd8cc0a3b00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 0x7fd8cc1d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): picked mon.noname-d con 0x7fd8cc0a3f00 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc1432d0 con 0x7fd8cc0a3720 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fd8cc0a6550 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fd8cc143450 con 0x7fd8cc0a3b00 2024-09-17T16:46:35.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient(hunting): authenticate will time out at 2078.648447s 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 0x7fd8cc1d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 0x7fd8cc1d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49516/0 (socket says 172.21.15.105:49516) 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 -- 172.21.15.105:0/2320328018 learned_addr learned my addr 172.21.15.105:0/2320328018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3720 auth_method 0 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3f00 auth_method 0 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): get_auth_request con 0x7fd8cc0a3b00 auth_method 0 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient(hunting): handle_auth_done global_id 94416 payload 824 2024-09-17T16:46:35.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 -- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 msgr2=0x7fd8cc1d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 0x7fd8cc1d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 -- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 msgr2=0x7fd8cc124750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient: _finish_hunting 0 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 monclient: found mon.noname-d 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 -- 172.21.15.105:0/2320328018 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1c82e0 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d13df640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 0x7fd8cc1d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d23e1640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 0x7fd8cc1d1dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400eba0 tx=0x7fd8c400c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 client.?.objecter ms_handle_connect 0x7fd8cc0a3f00 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:35.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8c4018020 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: dump: 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.404 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4014480 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 5 monclient: authenticate success, global_id 94416 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _renew_subs 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc125c70 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8c4014ae0 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c3fff640 4 set_mon_vals no callback set 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 monclient: dump: 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:35.406 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 client.94416.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _renew_subs 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc125990 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 librados: init done 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0a1e70 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fd8c401b020 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 0x7fd8b0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fd8c4050820 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 client.94416.objecter ms_dispatch 0x7fd8cc000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 3 client.94416.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 3 client.94416.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 20 client.94416.objecter dump_active .. 0 homeless 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 0x7fd8b0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 10 monclient: get_auth_request con 0x7fd8b003e550 auth_method 0 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8d1be0640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 0x7fd8b0040a10 secure :-1 s=READY pgs=3674 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc01c270 tx=0x7fd8bc0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:35.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.382+0000 7fd8c27fc640 10 client.94416.objecter ms_handle_connect 0x7fd8b003e550 2024-09-17T16:46:35.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.386+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8c4022030 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.386+0000 7fd8c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:35.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.386+0000 7fd8c27fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.630+0000 7fd8d3e44640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-17T16:46:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.630+0000 7fd8d3e44640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-17T16:46:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.630+0000 7fd8d3e44640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:35.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.630+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd8cc0a09d0 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.630+0000 7fd8c27fc640 1 -- 172.21.15.105:0/2320328018 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd8cc1c82e0 con 0x7fd8cc0a3f00 2024-09-17T16:46:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-17T16:46:35.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8c27fc640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:35.642 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.642 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=12,kv=9,logm=58} crc {auth=1454400957,config=692619927,health=2809446585,kv=566996664,logm=974150525}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=18,config=2,health=10,kv=9,logm=61} crc {auth=3808092742,config=692619927,health=1824799481,kv=732018829,logm=3707625903}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=12,kv=7,logm=60} crc {auth=506856406,config=692619927,health=2809446585,kv=976254068,logm=2183911276}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=19,config=2,health=12,kv=10,logm=57} crc {auth=4076187814,config=692619927,health=2809446585,kv=2863082231,logm=73118736}) 2024-09-17T16:46:35.643 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=19,config=2,health=9,kv=9,logm=61} crc {auth=1290880976,config=692619927,health=409596735,kv=1551675710,logm=1268748488}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=3248707691}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=19,config=2,health=11,kv=10,logm=58} crc {auth=2025089190,config=692619927,health=4267428109,kv=2863082231,logm=3360214404}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=4023837418,config=692619927,health=2809446585,kv=2863082231,logm=212386026}) 2024-09-17T16:46:35.644 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.634+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=20,config=2,health=9,kv=10,logm=59} crc {auth=4023837418,config=692619927,health=1642917820,kv=2863082231,logm=2787741671}) 2024-09-17T16:46:35.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 10 librados: watch_flush enter 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 10 librados: watch_flush exit 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 20 client.94416.objecter shutdown clearing up homeless session... 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 10 client.94416.objecter successfully canceled tick 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 msgr2=0x7fd8b0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 0x7fd8b0040a10 secure :-1 s=READY pgs=3674 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc01c270 tx=0x7fd8bc0a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 10 monclient: shutdown 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 msgr2=0x7fd8cc1d1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 0x7fd8cc1d1dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400eba0 tx=0x7fd8c400c490 comp rx=0 tx=0).stop 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 shutdown_connections 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fd8b003e550 0x7fd8b0040a10 unknown :-1 s=CLOSED pgs=3674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fd8cc0a3b00 0x7fd8cc1d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fd8cc0a3f00 0x7fd8cc1d1dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 --2- 172.21.15.105:0/2320328018 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd8cc0a3720 0x7fd8cc124750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:35.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 >> 172.21.15.105:0/2320328018 conn(0x7fd8cc0aa390 msgr2=0x7fd8cc0a54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:35.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 shutdown_connections 2024-09-17T16:46:35.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 -- 172.21.15.105:0/2320328018 wait complete. 2024-09-17T16:46:35.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:35.634+0000 7fd8d3e44640 1 librados: shutdown 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183247104}) 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.649 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4284747385}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3657242884}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3842736497}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1482509526}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2651575670}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.650 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=492226776}) 2024-09-17T16:46:35.651 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.651 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1547083548}) 2024-09-17T16:46:35.651 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.642+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1980422952}) 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292146149}) 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1272848857}) 2024-09-17T16:46:35.655 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3486432254}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3529785022}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2380885660}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3705391702}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.656 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292133714}) 2024-09-17T16:46:35.657 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.657 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2273440651}) 2024-09-17T16:46:35.657 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.646+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=357569339}) 2024-09-17T16:46:35.659 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.659 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.659 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223260889}) 2024-09-17T16:46:35.659 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=917017553}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3889142363}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2455523968}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1260379769}) 2024-09-17T16:46:35.660 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1927141483}) 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2466470341}) 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753131134}) 2024-09-17T16:46:35.661 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.650+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3007920115}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1790308996}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1636354703}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730038818}) 2024-09-17T16:46:35.666 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2892518284}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3581714336}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1098118524}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4030616613}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=882997166}) 2024-09-17T16:46:35.667 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.658+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3659089973}) 2024-09-17T16:46:35.668 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.668 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.668 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=514307114}) 2024-09-17T16:46:35.668 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.668 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4290014568}) 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2465734450}) 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1091681974}) 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.672 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2224780274}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1139396146}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723030369}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3344312379}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3769826483}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.673 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157451574}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1794362269}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=399889171}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2306881238}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1995729720}) 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.674 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=374438929}) 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1118252377}) 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145629900}) 2024-09-17T16:46:35.675 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.662+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2954836148}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=711594683}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2890261907}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.682 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2936997044}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=345127852}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2230596690}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1820030389}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=759226828}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4028226242}) 2024-09-17T16:46:35.683 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.666+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=958771846}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1775550115}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1084287681}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2309492242}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3873017277}) 2024-09-17T16:46:35.684 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2173905088}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1306154304}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1310500559}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2511843514}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.670+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3829121256}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.685 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3894185307}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2882478405}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3985960798}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=631794902}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=623304598}) 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.686 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=68142999}) 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1683704069}) 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2410905958}) 2024-09-17T16:46:35.687 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1776323294}) 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2851447301}) 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.690 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=209083753}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=34520026}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3625211974}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1126031595}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3975424173}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.691 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3036220439}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19349218}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.674+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3460162157}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=69893638}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2355928968}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1260270958}) 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.692 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1574135030}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1040335515}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2441328250}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1417012975}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2307377486}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.678+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3787386182}) 2024-09-17T16:46:35.693 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725103712}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1213082697}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276410746}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4119725842}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1511582410}) 2024-09-17T16:46:35.694 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3932631843}) 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1400945148}) 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2198835006}) 2024-09-17T16:46:35.695 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.682+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2086282744}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3670030476}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=827281281}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.700 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1206150314}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2953139268}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1955206587}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3451700442}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2995017346}) 2024-09-17T16:46:35.701 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1648132528}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.686+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1872664060}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023107781}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=596411773}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2668042955}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1486749698}) 2024-09-17T16:46:35.702 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3144903856}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=346490368}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=961608375}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3868476568}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4247104383}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.703 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3413377552}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4012785630}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450102944}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3358892016}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1766950259}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.704 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3585954464}) 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4202232415}) 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3649161475}) 2024-09-17T16:46:35.705 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.690+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2229249618}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1928518338}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4052988460}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2520699185}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2279227258}) 2024-09-17T16:46:35.710 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2814977277}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=229512698}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2180531382}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2536414736}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.694+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1750831377}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.711 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966961234}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3267556807}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1020836611}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3285906128}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1025039600}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.712 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1384203911}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=952535701}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3773775294}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.698+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1819091711}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1062233336}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3338435181}) 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.713 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1886689453}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=739868315}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=80796352}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3072992425}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=5562197}) 2024-09-17T16:46:35.714 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.715 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=883123931}) 2024-09-17T16:46:35.715 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=308515744}) 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393445490}) 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.724 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3307041822}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1791379501}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3494752749}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=880264160}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1669457978}) 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.725 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2652932494}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6833682}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.702+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1555425795}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503416968}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3941273041}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.726 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410179112}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=421773492}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2371218960}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2952309596}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2046507433}) 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.727 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1736795969}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.706+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4063986900}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=885418966}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2496408161}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3007764613}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=825424948}) 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.728 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1343242625}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=245774924}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4209092855}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3542013207}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.710+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2223464669}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.729 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888323548}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4064546218}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=985540501}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1986694050}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1587358839}) 2024-09-17T16:46:35.730 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3091392932}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4177659811}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916115555}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3508686577}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3602847198}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.731 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2651215109}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2568299660}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2092334181}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.714+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=447124416}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3441239128}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.732 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2356006249}) 2024-09-17T16:46:35.733 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.733 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=662003977}) 2024-09-17T16:46:35.733 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3436118218}) 2024-09-17T16:46:35.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.749 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=884410902}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4134257857}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=383729170}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3170444881}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3635463768}) 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.750 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2863043023}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=646635639}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464621069}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.718+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=391271354}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072880933}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1256853001}) 2024-09-17T16:46:35.751 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=233030632}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1510698640}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2971335476}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1098628557}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.752 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2792126626}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2489987739}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3321521840}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114337370}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1622861976}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2751376924}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.753 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=21458266}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=530064245}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1799676496}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2263122473}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=975854177}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.722+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=17944803}) 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.754 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3226520096}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289948959}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3264864440}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4089134162}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1771145337}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.755 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1765797940}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525009034}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540252600}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.726+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1087236602}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3360635120}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4240176622}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.756 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4175446072}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3222828979}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=400084175}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2417961748}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2149464430}) 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.757 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=631130973}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.730+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1542961191}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592791852}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=179716714}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=885837606}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4007164651}) 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.758 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1061579523}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1107121265}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3713201403}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=972757702}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2129264118}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=936101891}) 2024-09-17T16:46:35.759 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=643269668}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4063854195}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3654285511}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3559577036}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2722177775}) 2024-09-17T16:46:35.760 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4254278470}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2355017773}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.734+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=840254285}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2827612734}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3627982217}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.761 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=125531049}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2307791495}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3534794138}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1847470910}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1576814948}) 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.762 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3769885687}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.738+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2754058510}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2471358300}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3401625827}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2205997147}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3523042932}) 2024-09-17T16:46:35.763 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1680308360}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2974995202}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=932841137}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3270162652}) 2024-09-17T16:46:35.764 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.742+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2714378469}) 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936983987}) 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.786 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=928722856}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1276412406}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3248333890}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1981971335}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1301863062}) 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.787 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2505612230}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=876185175}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2738256341}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=74955877}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3040754019}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3916157286}) 2024-09-17T16:46:35.788 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=691271541}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1322124546}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1136125747}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=248173351}) 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.789 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440429729}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.746+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=665997101}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=615529027}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2648167723}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1861637231}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3235730845}) 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.790 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4040268060}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2641903436}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1789490188}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699594207}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.750+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1945530648}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2917942180}) 2024-09-17T16:46:35.791 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=926757547}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2774633868}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1110328198}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=943230843}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.792 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2568104431}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2826778752}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396589197}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.754+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3687389524}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2009493876}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2198576974}) 2024-09-17T16:46:35.793 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1801962482}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2576824840}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1173396625}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=553957850}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3221843162}) 2024-09-17T16:46:35.794 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397416049}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.758+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2781260338}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220483817}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3499496749}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=935741977}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.795 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3705029617}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2973467728}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2895298237}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3658846677}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1710814574}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3043739923}) 2024-09-17T16:46:35.796 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3618310127}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2359668570}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4075109445}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1748239305}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.797 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3086518489}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1259971848}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1392600453}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=221944428}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.762+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2945943659}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3426397801}) 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.798 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=845510774}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2020095657}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.766+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2755111498}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2560825055}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=919137752}) 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.799 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1259808407}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3953834621}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=823194139}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=453280298}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540673696}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.800 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2323957143}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=169560802}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3351896254}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3867589253}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2999417934}) 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.801 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749040967}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.770+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801541529}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2742973652}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=71482781}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244486486}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.802 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4134116054}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2503503862}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273964563}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4203956685}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1463297308}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1236768562}) 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.803 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867498606}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4061225508}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997416342}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2339374261}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1900963660}) 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.804 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2970344668}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2552011806}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=843645048}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.774+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3929377270}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2775533088}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.805 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3361754343}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1794486590}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=462055653}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1316044316}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2241630996}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.806 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2575488724}) 2024-09-17T16:46:35.807 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.807 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413008534}) 2024-09-17T16:46:35.807 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3441164769}) 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.778+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3567689696}) 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2684804013}) 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.852 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3601895756}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1713157705}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2201733892}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2690785990}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.853 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4051148399}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1803400285}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=357999200}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=62025690}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=391970942}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3105528689}) 2024-09-17T16:46:35.854 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4127579299}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1892656332}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4110800571}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1584301813}) 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.855 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6113975}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.782+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4168686238}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3322091547}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=423690524}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3993142359}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2398691113}) 2024-09-17T16:46:35.856 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1007255838}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3038351759}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2141257304}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1309363490}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=481290558}) 2024-09-17T16:46:35.857 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3970836630}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1703118315}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3771619484}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4122506280}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.858 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=19976202}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1511609206}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.786+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2843870753}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=937482376}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2250034237}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=479559965}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.859 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=346324297}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4274083021}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=624931221}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2537099020}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3597940509}) 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.860 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2129655751}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978474332}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.790+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2907461992}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3547646053}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112919417}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=534771530}) 2024-09-17T16:46:35.861 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2870469700}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3525590113}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3359128979}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4180669079}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246418416}) 2024-09-17T16:46:35.862 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3032441908}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1228511359}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3135752300}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1479285904}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1055631121}) 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.863 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1942035096}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2499942521}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3694690311}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421517704}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.794+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=777601787}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=330332670}) 2024-09-17T16:46:35.864 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750507356}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596906678}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2598275331}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3634295270}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.865 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075142804}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3003840266}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679440335}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.798+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=689782462}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=153031259}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1914973370}) 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.866 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3732431578}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1068058083}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1599485216}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2019936422}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2251731367}) 2024-09-17T16:46:35.867 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3152755160}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.802+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2958045480}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722365074}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3945997864}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1942473621}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.868 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1827279042}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3830036507}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1322051594}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964019890}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4166260798}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=310780079}) 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.869 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=884375255}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3033139701}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4216369542}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3504219197}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2230864820}) 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.870 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3776216483}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1302473562}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768452815}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.806+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2624056655}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1333771393}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544377682}) 2024-09-17T16:46:35.871 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1829407494}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3361951189}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1845305073}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3527248194}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.872 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2798102927}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=396343010}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3539770340}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2510211858}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2421492384}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3958845810}) 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.873 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3818581493}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4057936199}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3743191816}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1060701913}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748347103}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.810+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4294530552}) 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.874 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=8005827}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285264167}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2669052007}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1897472171}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1017383871}) 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.875 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3141632413}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=873969498}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212251719}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1536545345}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699893589}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.876 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=789809347}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1125206091}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2084038032}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2855914603}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.814+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3642865879}) 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.877 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3810348849}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2301521345}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1010426017}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3336855185}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1938240109}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1418925613}) 2024-09-17T16:46:35.878 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=19233938}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2041260766}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3268722981}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=853859308}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773146028}) 2024-09-17T16:46:35.879 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.818+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1379865486}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4246496174}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2703257497}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3144449198}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3296413625}) 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.880 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1983936027}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2162061783}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1370992253}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029982698}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1065180521}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3648881301}) 2024-09-17T16:46:35.881 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3469049000}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3329089545}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4124658769}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3769009694}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1522864279}) 2024-09-17T16:46:35.882 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1530533542}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2114116254}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.822+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3080016693}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=138911471}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2369476079}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.883 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3790115307}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1089059594}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1032935743}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=533609674}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1601962124}) 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.884 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1674185564}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3071356983}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3122133310}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3704012955}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=628129995}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3122787030}) 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.885 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3114109464}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2526200223}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=799475147}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3135106594}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.826+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3155173622}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662387916}) 2024-09-17T16:46:35.886 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862597103}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1812639992}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4076667762}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2160492897}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.887 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=424997903}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1047597692}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2590069302}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3318632677}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2899151199}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2851972189}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.888 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2767027570}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3998043888}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.830+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1894662428}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3024763436}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2941638115}) 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.889 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339524255}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=93753007}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156965032}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4277762383}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3437340067}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=567634212}) 2024-09-17T16:46:35.890 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2539120325}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2226909723}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3547531221}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=16409345}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2670280762}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3756806695}) 2024-09-17T16:46:35.891 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=183129069}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4018683548}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4062659856}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.834+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1636976051}) 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.892 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1219926148}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1740677600}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1883626518}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=820396589}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=735812502}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=107207620}) 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.893 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=52588975}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2172449751}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3321397677}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2468802658}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1944520608}) 2024-09-17T16:46:35.894 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318406363}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.838+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=441796784}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1120633850}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1745550727}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.895 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=590447929}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=208552010}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=709632222}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=108095981}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3531980668}) 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.896 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=521334157}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2825673779}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3611897331}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2813400112}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=94986848}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3794220340}) 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.897 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2016040098}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2106392787}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3126650848}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2499211214}) 2024-09-17T16:46:35.898 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.842+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2000879030}) 2024-09-17T16:46:36.032 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.032 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3424761877}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=129385273}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3816872490}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3934499282}) 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.033 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1004737609}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3588622447}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=329623253}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2399317414}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.846+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=903656826}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4071452272}) 2024-09-17T16:46:36.034 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4184831878}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1819391450}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4006031844}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2752389250}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.035 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=458786105}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=459861992}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=430576463}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=453049907}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4256476489}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1504234986}) 2024-09-17T16:46:36.036 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102345320}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1410336103}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1350839443}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=452379300}) 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.037 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3842462530}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.850+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1115936973}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1623374181}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3911356270}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3284626316}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=919733238}) 2024-09-17T16:46:36.038 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2326123119}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2395307944}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1498384130}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1657486697}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1616415990}) 2024-09-17T16:46:36.039 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.854+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3509895177}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907404957}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2288633933}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194178272}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1279049141}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.040 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1839163600}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2742312691}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1285761746}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994712285}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2270616045}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1417156511}) 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.041 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1470295746}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3151359601}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=177191747}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3548971669}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.042 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1408660601}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3466495077}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3480282768}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.858+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2420812212}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1930033948}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1093325488}) 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.043 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1304233849}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=178928683}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2450881482}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=363400997}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3237567723}) 2024-09-17T16:46:36.044 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3471847410}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.862+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=973464994}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=234666065}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2144840759}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2051909619}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.045 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=782526700}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1575056152}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=570832099}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3648916603}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476605883}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1057084179}) 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.046 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3005062829}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=155270862}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3932330509}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1080813685}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=415501434}) 2024-09-17T16:46:36.047 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1296294029}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1362728127}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3766348618}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.866+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=361516127}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2481064139}) 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.048 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=337814292}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1490008567}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3785042212}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1247871353}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2976200940}) 2024-09-17T16:46:36.049 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3813433260}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305302825}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2006923238}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.870+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3251880688}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2172365702}) 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.050 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2143397319}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2165385274}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3856789079}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107017234}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2058453834}) 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.051 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607506565}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2375116318}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3147565637}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2404083189}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4249027592}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.052 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2752110433}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=896532369}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1103877694}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3029930683}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4043759729}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.874+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2158775429}) 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.053 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=410572430}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2519417488}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3061738915}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=288929326}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3973776129}) 2024-09-17T16:46:36.054 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1049225032}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1805921003}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3187928888}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2811995655}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=883426782}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.055 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3821297789}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3135604878}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3419269603}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=208918346}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2418205958}) 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.056 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=804038243}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1917613216}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.878+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4048297649}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2064499718}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552202302}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4146128575}) 2024-09-17T16:46:36.057 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2730744197}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4100585302}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2941489071}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1467382567}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246410973}) 2024-09-17T16:46:36.058 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.882+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4233984543}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281250243}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1998806688}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3423925561}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=621461540}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.059 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2479731770}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2990162413}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=783692795}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3027562628}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2581471646}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014291780}) 2024-09-17T16:46:36.060 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2745279868}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2130138298}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4155869156}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1437090037}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.061 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2620340900}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2983944417}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041475207}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.886+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2724485560}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=201637591}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2829928271}) 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.062 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1638963391}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3031089622}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1217980816}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=901154129}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1470163043}) 2024-09-17T16:46:36.063 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3297322748}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.890+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1338293323}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3704196742}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2766571253}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=919415369}) 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.064 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3373978779}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1742216111}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163121445}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2762389516}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2097191916}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=757544478}) 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.065 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4213173363}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2693943953}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1469737860}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1260455423}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=370085675}) 2024-09-17T16:46:36.066 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=30017608}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2654923739}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3735893253}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.894+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4064961890}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1022635727}) 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.067 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=805993597}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1851532807}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2756132231}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3164507422}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.068 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1663456680}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3634032600}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667851061}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=255453173}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601219062}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285791713}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.069 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3986394564}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1238238729}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=11310927}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3976007955}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3073188733}) 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.070 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3667297377}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.898+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1373474525}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848941314}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2361701388}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=451349883}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2214664400}) 2024-09-17T16:46:36.071 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1949855816}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1953602620}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=57738771}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2497451230}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3925101710}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.072 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356516539}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=114034247}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2119042244}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2950239858}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1143616733}) 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.073 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.902+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2469017687}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3350440680}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=60078885}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1533700966}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863062074}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.074 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2151176034}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570316384}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=933004477}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3735490884}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2045107887}) 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.075 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=555992612}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1708691950}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.906+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3529304294}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3766521837}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2340949690}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.076 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=311003146}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3572406582}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2233416268}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=527681356}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1223778911}) 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.077 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=41787173}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1000491711}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1260255986}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1805534351}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=98348026}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2283109544}) 2024-09-17T16:46:36.078 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4282653177}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2252015336}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2567732685}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699579104}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.910+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2285146355}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.079 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4041896438}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1288774428}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3871449335}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=209814898}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1593836773}) 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.080 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3910247586}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=162149646}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1730844578}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.914+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3123262065}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2049574713}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.081 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853442766}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3992386517}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3873252333}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2151575451}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939581591}) 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.082 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=714240453}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2592483879}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1702778234}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2230574512}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1921736242}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3631749051}) 2024-09-17T16:46:36.083 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3210039914}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1434268633}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=112719905}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1999038679}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=641809117}) 2024-09-17T16:46:36.084 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.918+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2882034183}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2353363960}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371822595}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4235017774}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2678379073}) 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.085 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4231456224}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1171960336}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1075076126}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033873558}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2051007361}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730175615}) 2024-09-17T16:46:36.086 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2212219227}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2460609199}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3515956222}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1415028787}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.087 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=930302179}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3272748506}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3982882364}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.922+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=147936027}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1627574557}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3819554502}) 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.088 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3306727900}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4096422578}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=962584575}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2404731814}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3683282043}) 2024-09-17T16:46:36.089 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=246201232}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2797463931}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1292700549}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2528979948}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.926+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2845832213}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.090 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1478718243}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=337492905}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=568825564}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3779990384}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3679894485}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2501814346}) 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.091 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330950006}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1280917941}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2863244253}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=779934031}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.092 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4062819788}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=215619584}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3391683451}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4220946125}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.930+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3592457465}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363899850}) 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.093 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3654974033}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2216793517}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=491937486}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3107714276}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=100779853}) 2024-09-17T16:46:36.094 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1124717574}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4266064578}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2902340334}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175153868}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3412326232}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.095 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=987560871}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3084641223}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3786877460}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=268051598}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1319158590}) 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.096 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649355183}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.934+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3683612180}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=752717685}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2966249005}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090333006}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=617858779}) 2024-09-17T16:46:36.097 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2858446628}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3397000297}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3911584587}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3009468098}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1921470995}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.098 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1588826789}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1030401355}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3185144366}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4084385306}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=331001199}) 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.099 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4218358157}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1763895436}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1594737348}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.938+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3062200882}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639379391}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2509175609}) 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.100 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3487353543}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=605835381}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=54164536}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3389744118}) 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.101 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1794467537}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1516838019}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.942+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1274183948}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1832589651}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2973014145}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3060178089}) 2024-09-17T16:46:36.102 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=472004527}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2868255570}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=135590245}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=44068066}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144613479}) 2024-09-17T16:46:36.103 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1900973497}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536589905}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737270100}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2474917357}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3041663904}) 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.104 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=557075160}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3632827918}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943900098}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=785069668}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.946+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=674061098}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1203592659}) 2024-09-17T16:46:36.105 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=576436483}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=216299445}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3729393193}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2840593906}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.106 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1819245047}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=649959759}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1673251627}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2111751987}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=200228627}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=208520437}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.107 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3963761083}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4064796914}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3437177210}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.950+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3758965913}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2040036110}) 2024-09-17T16:46:36.108 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2421605907}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1948713794}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=724082588}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1032411559}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1533490272}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.109 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=961187991}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1920869322}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3199244449}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=613305387}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=178171589}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.954+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3822948685}) 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.110 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1194643194}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2995512210}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1086940336}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1247079572}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1961643784}) 2024-09-17T16:46:36.111 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2815785783}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2911533631}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1142192196}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2565160811}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1538580443}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.112 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=810232712}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3479554013}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1911224839}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4247414579}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1688107893}) 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.113 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4134352454}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2214861832}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.958+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2120674988}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4022663451}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2982924998}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1894230112}) 2024-09-17T16:46:36.114 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1829626794}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2915926686}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2361588787}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4289813878}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2168273935}) 2024-09-17T16:46:36.115 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=715630702}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2059787552}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.962+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1018821631}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3889793216}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=602792079}) 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.116 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3537739060}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3957638652}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3536742314}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3748361627}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1394771820}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=660430690}) 2024-09-17T16:46:36.117 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3232848499}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1417919623}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1307594892}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3649729634}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.118 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3684294294}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3716178591}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776165055}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.966+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4114867298}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=967344691}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=685863708}) 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.119 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1009684715}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1785796238}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2910379014}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033824273}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.120 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2853972931}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3913517646}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=791262167}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=744407997}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2010177069}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2457210873}) 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.121 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1692557641}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1354894583}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1587522009}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3679616344}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588643664}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.970+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=890247856}) 2024-09-17T16:46:36.122 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1800049301}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3462394411}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717094729}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3255971884}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=419186200}) 2024-09-17T16:46:36.123 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1906796261}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1734978610}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=675659413}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.974+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1755113732}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2950217384}) 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.124 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2107781803}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1504594051}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=576244215}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3545779379}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=330921159}) 2024-09-17T16:46:36.125 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=536281907}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270599618}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2664917520}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2366294626}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=105887817}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.126 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2562819609}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=329115093}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2653570211}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3407986446}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3723362790}) 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.127 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433786596}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.978+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1479467054}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=892185335}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=442266927}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4088118470}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3330557607}) 2024-09-17T16:46:36.128 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1945252540}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37605782}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1473338417}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=20771091}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2449074016}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.129 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1175211466}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=484779000}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1737956494}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2875764046}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3975719259}) 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.130 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1147801208}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2533600824}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3560488831}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.982+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3279230429}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=485752961}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3584477298}) 2024-09-17T16:46:36.131 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=68335927}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3622379078}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=289065452}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423375082}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=573443812}) 2024-09-17T16:46:36.132 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916074414}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3206654987}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2640300986}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2700494243}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1826510700}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.133 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4065148516}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3466636587}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4014570288}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2077141752}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1688016720}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.986+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1968642072}) 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.134 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3366636449}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3454372672}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2759320358}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3589621097}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1416866678}) 2024-09-17T16:46:36.135 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2889350261}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3905150051}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100078630}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=306660330}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3877044511}) 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.136 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3487331209}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=519574517}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3338544204}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3295772297}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=483880370}) 2024-09-17T16:46:36.137 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3861380894}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465119757}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.990+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1176842427}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2267234975}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2914156939}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.138 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482821532}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1969710553}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4135019655}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=49456950}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1600188433}) 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.139 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316301810}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1860597407}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=2} crc {logm=1553736896,mdsmap=2435593270,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=981556218,mon_config_key=2681608797,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=2560959409,osdmap=569292936}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.994+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=41,osdmap=2} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=2681608797,monmap=1360384041,nvmeofgw=598337686,osd_metadata=352991311,osd_pg_creating=2801634870,osd_snap=1408702634,osdmap=964682662}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=4} crc {logm=4152266457,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgrstat=981556218,mon_config_key=4257234632,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=581286859,osdmap=342398979}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=2} crc {logm=381218200,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=4063125195,mon_config_key=346208105,monmap=1360384041,nvmeofgw=598337687,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=1092692323,osdmap=964682662}) 2024-09-17T16:46:36.140 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=41,osdmap=2} crc {logm=381218200,mdsmap=1248375647,mgr=647233216,mgr_command_descs=1770323759,mgrstat=981556218,mon_config_key=3381100223,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=2869229162,osdmap=964682662}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=12,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=4} crc {logm=316175748,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=981556218,mon_config_key=346208105,monmap=1360384041,nvmeofgw=2613832008,osd_metadata=49183994,osd_pg_creating=2801634870,osd_snap=2593883951,osdmap=342398979}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=41} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=981556218,mon_config_key=604563636,monmap=1360384041,nvmeofgw=598337686,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=1408702634}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,nvmeofgw=2,osd_metadata=6,osd_snap=40,osdmap=4} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=158765706,mon_config_key=1734705363,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_snap=275758491,osdmap=342398979}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=14,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=2} crc {logm=1553736896,mdsmap=1248375647,mgr=3567252286,mgr_command_descs=1770323759,mgr_metadata=3792311847,mgrstat=981556218,mon_config_key=346208105,monmap=1360384041,nvmeofgw=1596038039,osd_metadata=3224233958,osd_pg_creating=2801634870,osd_snap=236012911,osdmap=964682662}) 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.141 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3958369536}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3989738867}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2231225766}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4097957978}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1372920122}) 2024-09-17T16:46:36.142 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1001606182}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=956329126}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4274869863}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:35.998+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4220356289}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2935889379}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3660390846}) 2024-09-17T16:46:36.143 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1784784701}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4002119646}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3517884405}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=702767704}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.144 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1139517390}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2774610933}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.002+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=159620199}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=51} crc {osdmap=3665059980}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=51} crc {osdmap=2085797146}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=50} crc {osdmap=2485508834}) 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.145 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=50} crc {osdmap=2939516417}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=50} crc {osdmap=2065797076}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=52} crc {osdmap=2183689162}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=51} crc {osdmap=2878412536}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=52} crc {osdmap=350191177}) 2024-09-17T16:46:36.146 INFO:tasks.ceph.mon.e.smithi105.stderr:2024-09-17T16:46:36.018+0000 7f3ee1787640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=52} crc {osdmap=1647130532}) 2024-09-17T16:46:36.670 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-17T16:46:36.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-17T16:46:36.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: get_monmap_and_config 2024-09-17T16:46:36.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: build_initial_monmap 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: monmap: 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:36.813412+0000 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:36.813412+0000 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 Processor -- start 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 -- start start 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: init 2024-09-17T16:46:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient: _add_conns ranks=[9,3,1,2,7,6,4,8,5,0,10] 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdefc0a6a30 0x7fdefc0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-a con 0x7fdefc0a6a30 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdefc0a4ab0 0x7fdefc0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-g con 0x7fdefc0a4ab0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdefc1397f0 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdefc0a4080 con 0x7fdefc1397f0 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc0a7c40 con 0x7fdefc0a6a30 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc0a7890 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdf01d72640 10 monclient(hunting): authenticate will time out at 2080.078860s 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdefc0a6a30 0x7fdefc0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdefc0a6a30 0x7fdefc0a6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51458/0 (socket says 172.21.15.105:51458) 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 -- 172.21.15.105:0/692584278 learned_addr learned my addr 172.21.15.105:0/692584278 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): get_auth_request con 0x7fdefc0a6a30 auth_method 0 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_done global_id 104202 payload 274 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 -- 172.21.15.105:0/692584278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdefc0a4ab0 msgr2=0x7fdefc0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefaffd640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdefc0a4ab0 0x7fdefc0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdefc0a4ab0 0x7fdefc0a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 -- 172.21.15.105:0/692584278 >> v1:172.21.15.160:6789/0 conn(0x7fdefc1397f0 legacy=0x7fdefc139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 monclient: found mon.noname-a 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.810+0000 7fdefb7fe640 1 -- 172.21.15.105:0/692584278 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc0a37e0 con 0x7fdefc0a6a30 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefb7fe640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdefc0a6a30 0x7fdefc0a6e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009870 tx=0x7fdeec01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2738161b898721fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 1 -- 172.21.15.105:0/692584278 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdeec0b0070 con 0x7fdefc0a6a30 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: dump: 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:36.837 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 1 -- 172.21.15.105:0/692584278 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeec0044a0 con 0x7fdefc0a6a30 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 5 monclient: authenticate success, global_id 104202 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 1 -- 172.21.15.105:0/692584278 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdeec003a90 con 0x7fdefc0a6a30 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:36.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdefa7fc640 10 monclient: dump: 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:36.839 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 4 set_mon_vals no callback set 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 10 monclient: shutdown 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- 172.21.15.105:0/692584278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdefc0a6a30 msgr2=0x7fdefc0a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdefc0a6a30 0x7fdefc0a6e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009870 tx=0x7fdeec01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- 172.21.15.105:0/692584278 shutdown_connections 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdefc0a4ab0 0x7fdefc0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 --2- 172.21.15.105:0/692584278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdefc0a6a30 0x7fdefc0a6e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- 172.21.15.105:0/692584278 >> 172.21.15.105:0/692584278 conn(0x7fdefc0a8d00 msgr2=0x7fdefc0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:36.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- 172.21.15.105:0/692584278 shutdown_connections 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- 172.21.15.105:0/692584278 wait complete. 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 10 monclient: build_initial_monmap 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 10 monclient: monmap: 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:36.841 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 librados: starting msgr at 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 librados: starting objecter 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 Processor -- start 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 -- start start 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 librados: setting wanted keys 2024-09-17T16:46:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 1 librados: calling monclient init 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.814+0000 7fdf01d72640 10 monclient: init 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _add_conns ranks=[6,7,5,0,1,2,8,4,3] 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-d con 0x7fdefc0a4ab0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdefc0a6a30 0x7fdefc1386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-i con 0x7fdefc0a6a30 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdefc1397f0 0x7fdefc13d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): picked mon.noname-h con 0x7fdefc1397f0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fdefc144270 con 0x7fdefc1397f0 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fdefc06e770 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fdefc1443f0 con 0x7fdefc0a6a30 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient(hunting): authenticate will time out at 2080.084494s 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54858/0 (socket says 172.21.15.105:54858) 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 -- 172.21.15.105:0/1556832370 learned_addr learned my addr 172.21.15.105:0/1556832370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): get_auth_request con 0x7fdefc0a4ab0 auth_method 0 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdefc0a6a30 0x7fdefc1386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): get_auth_request con 0x7fdefc0a6a30 auth_method 0 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefbfff640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdefc1397f0 0x7fdefc13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefbfff640 10 monclient(hunting): get_auth_request con 0x7fdefc1397f0 auth_method 0 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefbfff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient(hunting): handle_auth_done global_id 94425 payload 824 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 -- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdefc0a6a30 msgr2=0x7fdefc1386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdefc0a6a30 0x7fdefc1386a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 -- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdefc1397f0 msgr2=0x7fdefc13d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdefc1397f0 0x7fdefc13d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 monclient: found mon.noname-d 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 -- 172.21.15.105:0/1556832370 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc1405b0 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefb7fe640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ae040 tx=0x7fdeec019190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 client.?.objecter ms_handle_connect 0x7fdefc0a4ab0 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdeec0bb070 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: dump: 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:36.848 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: _finish_auth 0 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 5 monclient: authenticate success, global_id 94425 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _renew_subs 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeec0ab440 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc122790 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdeec0b0040 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdef9ffb640 4 set_mon_vals no callback set 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 client.94425.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 monclient: dump: 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _renew_subs 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc13f0c0 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 librados: init done 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fdeec003b00 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 0x7fdee8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc1d20c0 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fdeec0e9440 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 client.94425.objecter ms_dispatch 0x7fdefc000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 3 client.94425.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 3 client.94425.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 20 client.94425.objecter dump_active .. 0 homeless 2024-09-17T16:46:36.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 0x7fdee8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 10 monclient: get_auth_request con 0x7fdee803e550 auth_method 0 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdee3fff640 10 client.94425.objecter ms_handle_connect 0x7fdee803e550 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.818+0000 7fdefaffd640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 0x7fdee8040a10 secure :-1 s=READY pgs=3675 cs=0 l=1 rev1=1 crypto rx=0x7fdefc13edd0 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.822+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdefc122790 con 0x7fdefc0a4ab0 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.822+0000 7fdee3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:36.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:36.822+0000 7fdee3fff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-17T16:46:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdefc139ca0 con 0x7fdefc0a4ab0 2024-09-17T16:46:37.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdee3fff640 1 -- 172.21.15.105:0/1556832370 <== mon.6 v2:172.21.15.105:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3213 (secure 0 0 0) 0x7fdeec016130 con 0x7fdefc0a4ab0 2024-09-17T16:46:37.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdee3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-17T16:46:37.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdee3fff640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:37.049 INFO:teuthology.orchestra.run.smithi105.stdout:{"election_epoch":118,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","modified":"2024-09-17T16:27:05.094380Z","created":"2024-09-17T16:27:05.094380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3301","nonce":0},{"type":"v1","addr":"172.21.15.160:6790","nonce":0}]},"addr":"172.21.15.160:6790/0","public_addr":"172.21.15.160:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3302","nonce":0},{"type":"v1","addr":"172.21.15.105:6791","nonce":0}]},"addr":"172.21.15.105:6791/0","public_addr":"172.21.15.105:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3302","nonce":0},{"type":"v1","addr":"172.21.15.160:6791","nonce":0}]},"addr":"172.21.15.160:6791/0","public_addr":"172.21.15.160:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3303","nonce":0},{"type":"v1","addr":"172.21.15.105:6792","nonce":0}]},"addr":"172.21.15.105:6792/0","public_addr":"172.21.15.105:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3303","nonce":0},{"type":"v1","addr":"172.21.15.160:6792","nonce":0}]},"addr":"172.21.15.160:6792/0","public_addr":"172.21.15.160:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3304","nonce":0},{"type":"v1","addr":"172.21.15.105:6793","nonce":0}]},"addr":"172.21.15.105:6793/0","public_addr":"172.21.15.105:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 librados: watch_flush enter 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 librados: watch_flush exit 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 20 client.94425.objecter shutdown clearing up homeless session... 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 client.94425.objecter successfully canceled tick 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 msgr2=0x7fdee8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 0x7fdee8040a10 secure :-1 s=READY pgs=3675 cs=0 l=1 rev1=1 crypto rx=0x7fdefc13edd0 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 10 monclient: shutdown 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 msgr2=0x7fdefc13fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ae040 tx=0x7fdeec019190 comp rx=0 tx=0).stop 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 shutdown_connections 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fdee803e550 0x7fdee8040a10 unknown :-1 s=CLOSED pgs=3675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fdefc1397f0 0x7fdefc13d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fdefc0a6a30 0x7fdefc1386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 --2- 172.21.15.105:0/1556832370 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fdefc0a4ab0 0x7fdefc13fc20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 >> 172.21.15.105:0/1556832370 conn(0x7fdefc0a8d00 msgr2=0x7fdefc0a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 shutdown_connections 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 -- 172.21.15.105:0/1556832370 wait complete. 2024-09-17T16:46:37.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.042+0000 7fdf01d72640 1 librados: shutdown 2024-09-17T16:46:37.066 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-17T16:46:37.066 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T16:46:37.075 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T16:46:37.075 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 10 monclient: get_monmap_and_config 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 10 monclient: monmap: 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:37.182282+0000 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:37.182282+0000 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:37.186 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:37.187 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 1 Processor -- start 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 1 -- start start 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.178+0000 7f21ccbc5640 10 monclient: init 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient: _add_conns ranks=[9,0,7,2,1,3,6,8,10,4,5] 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21c80a68e0 0x7f21c80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-a con 0x7f21c80a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f21c80a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c8139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-i con 0x7f21c80a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c806ebc0 con 0x7f21c80a4910 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80a3fc0 con 0x7f21c80a68e0 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f21c80a63c0 con 0x7f21c80a7200 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient(hunting): authenticate will time out at 2080.448603s 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f21c80a68e0 0x7f21c80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c5d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f21c80a4910 0x7f21c81396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54718/0 (socket says 172.21.15.105:54718) 2024-09-17T16:46:37.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c5d9b640 1 -- 172.21.15.105:0/2369966317 learned_addr learned my addr 172.21.15.105:0/2369966317 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): get_auth_request con 0x7f21c80a68e0 auth_method 0 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient(hunting): handle_auth_done global_id 104211 payload 274 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 -- 172.21.15.105:0/2369966317 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 msgr2=0x7f21c8139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 --2- 172.21.15.105:0/2369966317 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c8139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 -- 172.21.15.105:0/2369966317 >> v1:172.21.15.105:6789/0 conn(0x7f21c80a4910 legacy=0x7f21c81396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 monclient: found mon.noname-a 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:37.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 -- 172.21.15.105:0/2369966317 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80a3690 con 0x7f21c80a68e0 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c659c640 1 --2- 172.21.15.105:0/2369966317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c80a68e0 0x7f21c80a6cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f21b80097c0 tx=0x7f21b801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f825a688ff1e0f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 1 -- 172.21.15.105:0/2369966317 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21b80b0070 con 0x7f21c80a68e0 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: dump: 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.198 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 1 -- 172.21.15.105:0/2369966317 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b80aa810 con 0x7f21c80a68e0 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 5 monclient: authenticate success, global_id 104211 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 4 set_mon_vals no callback set 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 1 -- 172.21.15.105:0/2369966317 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21b80a9300 con 0x7f21c80a68e0 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21c559a640 10 monclient: dump: 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.200 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 10 monclient: shutdown 2024-09-17T16:46:37.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 -- 172.21.15.105:0/2369966317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c80a68e0 msgr2=0x7f21c80a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/2369966317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c80a68e0 0x7f21c80a6cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f21b80097c0 tx=0x7f21b801c330 comp rx=0 tx=0).stop 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 -- 172.21.15.105:0/2369966317 shutdown_connections 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/2369966317 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.182+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/2369966317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21c80a68e0 0x7f21c80a6cc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/2369966317 >> 172.21.15.105:0/2369966317 conn(0x7f21c8068ba0 msgr2=0x7f21c8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/2369966317 shutdown_connections 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/2369966317 wait complete. 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: monmap: 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:37.202 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 librados: starting msgr at 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 librados: starting objecter 2024-09-17T16:46:37.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 Processor -- start 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- start start 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 librados: setting wanted keys 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 librados: calling monclient init 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: init 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _add_conns ranks=[8,1,7,6,0,3,4,2,5] 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-e con 0x7f21c80a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21c80a68e0 0x7f21c81d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-f con 0x7f21c80a68e0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c813e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): picked mon.noname-i con 0x7f21c80a7200 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f21c813a2f0 con 0x7f21c80a68e0 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f21c806e630 con 0x7f21c80a7200 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f21c813a470 con 0x7f21c80a4910 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient(hunting): authenticate will time out at 2080.453478s 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51252/0 (socket says 172.21.15.105:51252) 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 -- 172.21.15.105:0/459736363 learned_addr learned my addr 172.21.15.105:0/459736363 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): get_auth_request con 0x7f21c80a4910 auth_method 0 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient(hunting): handle_auth_done global_id 55178 payload 824 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 msgr2=0x7f21c813e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c813e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21c80a68e0 msgr2=0x7f21c81d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21c80a68e0 0x7f21c81d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 monclient: found mon.noname-e 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 -- 172.21.15.105:0/459736363 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c81401e0 con 0x7f21c80a4910 2024-09-17T16:46:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c659c640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f21b80ae040 tx=0x7f21b80190d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 client.?.objecter ms_handle_connect 0x7f21c80a4910 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21b80b0040 con 0x7f21c80a4910 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: dump: 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.208 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b80b9070 con 0x7f21c80a4910 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 5 monclient: authenticate success, global_id 55178 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _renew_subs 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c81238c0 con 0x7f21c80a4910 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21b80aae30 con 0x7f21c80a4910 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c4d99640 4 set_mon_vals no callback set 2024-09-17T16:46:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 client.55178.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 monclient: dump: 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _renew_subs 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c80a21a0 con 0x7f21c80a4910 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 librados: init done 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c806df60 con 0x7f21c80a4910 2024-09-17T16:46:37.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f21b8004040 con 0x7f21c80a4910 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f21b80e9a50 con 0x7f21c80a4910 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 10 client.55178.objecter ms_dispatch 0x7f21c8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 3 client.55178.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 3 client.55178.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21b6ffd640 20 client.55178.objecter dump_active .. 0 homeless 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c5d9b640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.186+0000 7f21c5d9b640 10 monclient: get_auth_request con 0x7f21a803e550 auth_method 0 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21c5d9b640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 msgr2=0x7f21a8040a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21c5d9b640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 msgr2=0x7f21a8040a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21c5d9b640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-17T16:46:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21c5d9b640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-17T16:46:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21b8016070 con 0x7f21c80a4910 2024-09-17T16:46:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.190+0000 7f21b6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:37.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.346+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21c80a6cc0 con 0x7f21a803e550 2024-09-17T16:46:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.390+0000 7f21c5d9b640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.390+0000 7f21c5d9b640 10 monclient: get_auth_request con 0x7f21a803e550 auth_method 0 2024-09-17T16:46:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.390+0000 7f21c5d9b640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 secure :-1 s=READY pgs=3676 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005890 tx=0x7f21bc009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.390+0000 7f21b6ffd640 10 client.55178.objecter ms_handle_connect 0x7f21a803e550 2024-09-17T16:46:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.394+0000 7f21b6ffd640 1 -- 172.21.15.105:0/459736363 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+114046 (secure 0 0 0) 0x7f21c80a6cc0 con 0x7f21a803e550 2024-09-17T16:46:37.404 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:46:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 10 librados: watch_flush enter 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 10 librados: watch_flush exit 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 20 client.55178.objecter shutdown clearing up homeless session... 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 10 client.55178.objecter successfully canceled tick 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 msgr2=0x7f21a8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 secure :-1 s=READY pgs=3676 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005890 tx=0x7f21bc009040 comp rx=0 tx=0).stop 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 10 monclient: shutdown 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 msgr2=0x7f21c81d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f21b80ae040 tx=0x7f21b80190d0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 shutdown_connections 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f21a803e550 0x7f21a8040a10 unknown :-1 s=CLOSED pgs=3676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f21c80a7200 0x7f21c813e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21c80a68e0 0x7f21c81d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 --2- 172.21.15.105:0/459736363 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f21c80a4910 0x7f21c81d1380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 >> 172.21.15.105:0/459736363 conn(0x7f21c8068ba0 msgr2=0x7f21c809fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 shutdown_connections 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 -- 172.21.15.105:0/459736363 wait complete. 2024-09-17T16:46:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.398+0000 7f21ccbc5640 1 librados: shutdown 2024-09-17T16:46:37.422 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":1264,"stamp":"2024-09-17T16:46:36.193103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":61,"num_write":110,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470648,"kb_used_data":62808,"kb_used_omap":9,"kb_used_meta":407734,"kb_avail":565760392,"statfs":{"total":579820584960,"available":579338641408,"internally_reserved":0,"allocated":64315392,"data_stored":36273584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":417520288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004247"},"pg_stats":[{"pgid":"6.16","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356126+0000","last_change":"2024-09-17T16:29:02.018311+0000","last_active":"2024-09-17T16:46:00.356126+0000","last_peered":"2024-09-17T16:46:00.356126+0000","last_clean":"2024-09-17T16:46:00.356126+0000","last_became_active":"2024-09-17T16:29:02.017756+0000","last_became_peered":"2024-09-17T16:29:02.017756+0000","last_unstale":"2024-09-17T16:46:00.356126+0000","last_undegraded":"2024-09-17T16:46:00.356126+0000","last_fullsized":"2024-09-17T16:46:00.356126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:14.467205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360583+0000","last_change":"2024-09-17T16:29:02.054931+0000","last_active":"2024-09-17T16:46:00.360583+0000","last_peered":"2024-09-17T16:46:00.360583+0000","last_clean":"2024-09-17T16:46:00.360583+0000","last_became_active":"2024-09-17T16:29:02.054559+0000","last_became_peered":"2024-09-17T16:29:02.054559+0000","last_unstale":"2024-09-17T16:46:00.360583+0000","last_undegraded":"2024-09-17T16:46:00.360583+0000","last_fullsized":"2024-09-17T16:46:00.360583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:37.538511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239713+0000","last_change":"2024-09-17T16:29:02.079133+0000","last_active":"2024-09-17T16:46:00.239713+0000","last_peered":"2024-09-17T16:46:00.239713+0000","last_clean":"2024-09-17T16:46:00.239713+0000","last_became_active":"2024-09-17T16:29:02.076410+0000","last_became_peered":"2024-09-17T16:29:02.076410+0000","last_unstale":"2024-09-17T16:46:00.239713+0000","last_undegraded":"2024-09-17T16:46:00.239713+0000","last_fullsized":"2024-09-17T16:46:00.239713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:04.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243332+0000","last_change":"2024-09-17T16:29:02.123528+0000","last_active":"2024-09-17T16:46:00.243332+0000","last_peered":"2024-09-17T16:46:00.243332+0000","last_clean":"2024-09-17T16:46:00.243332+0000","last_became_active":"2024-09-17T16:29:02.122292+0000","last_became_peered":"2024-09-17T16:29:02.122292+0000","last_unstale":"2024-09-17T16:46:00.243332+0000","last_undegraded":"2024-09-17T16:46:00.243332+0000","last_fullsized":"2024-09-17T16:46:00.243332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:48.555269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360740+0000","last_change":"2024-09-17T16:29:02.064488+0000","last_active":"2024-09-17T16:46:00.360740+0000","last_peered":"2024-09-17T16:46:00.360740+0000","last_clean":"2024-09-17T16:46:00.360740+0000","last_became_active":"2024-09-17T16:29:02.064232+0000","last_became_peered":"2024-09-17T16:29:02.064232+0000","last_unstale":"2024-09-17T16:46:00.360740+0000","last_undegraded":"2024-09-17T16:46:00.360740+0000","last_fullsized":"2024-09-17T16:46:00.360740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:14.884902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354984+0000","last_change":"2024-09-17T16:29:02.121725+0000","last_active":"2024-09-17T16:46:00.354984+0000","last_peered":"2024-09-17T16:46:00.354984+0000","last_clean":"2024-09-17T16:46:00.354984+0000","last_became_active":"2024-09-17T16:29:02.116801+0000","last_became_peered":"2024-09-17T16:29:02.116801+0000","last_unstale":"2024-09-17T16:46:00.354984+0000","last_undegraded":"2024-09-17T16:46:00.354984+0000","last_fullsized":"2024-09-17T16:46:00.354984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:23.905864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242889+0000","last_change":"2024-09-17T16:29:02.131256+0000","last_active":"2024-09-17T16:46:00.242889+0000","last_peered":"2024-09-17T16:46:00.242889+0000","last_clean":"2024-09-17T16:46:00.242889+0000","last_became_active":"2024-09-17T16:29:02.129701+0000","last_became_peered":"2024-09-17T16:29:02.129701+0000","last_unstale":"2024-09-17T16:46:00.242889+0000","last_undegraded":"2024-09-17T16:46:00.242889+0000","last_fullsized":"2024-09-17T16:46:00.242889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:34:10.375884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356246+0000","last_change":"2024-09-17T16:29:02.079372+0000","last_active":"2024-09-17T16:46:00.356246+0000","last_peered":"2024-09-17T16:46:00.356246+0000","last_clean":"2024-09-17T16:46:00.356246+0000","last_became_active":"2024-09-17T16:29:02.078833+0000","last_became_peered":"2024-09-17T16:29:02.078833+0000","last_unstale":"2024-09-17T16:46:00.356246+0000","last_undegraded":"2024-09-17T16:46:00.356246+0000","last_fullsized":"2024-09-17T16:46:00.356246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:50.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360568+0000","last_change":"2024-09-17T16:29:02.185295+0000","last_active":"2024-09-17T16:46:00.360568+0000","last_peered":"2024-09-17T16:46:00.360568+0000","last_clean":"2024-09-17T16:46:00.360568+0000","last_became_active":"2024-09-17T16:29:02.179921+0000","last_became_peered":"2024-09-17T16:29:02.179921+0000","last_unstale":"2024-09-17T16:46:00.360568+0000","last_undegraded":"2024-09-17T16:46:00.360568+0000","last_fullsized":"2024-09-17T16:46:00.360568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:18.876647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.240108+0000","last_change":"2024-09-17T16:29:02.165149+0000","last_active":"2024-09-17T16:46:00.240108+0000","last_peered":"2024-09-17T16:46:00.240108+0000","last_clean":"2024-09-17T16:46:00.240108+0000","last_became_active":"2024-09-17T16:29:02.164714+0000","last_became_peered":"2024-09-17T16:29:02.164714+0000","last_unstale":"2024-09-17T16:46:00.240108+0000","last_undegraded":"2024-09-17T16:46:00.240108+0000","last_fullsized":"2024-09-17T16:46:00.240108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:19.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355098+0000","last_change":"2024-09-17T16:29:02.177944+0000","last_active":"2024-09-17T16:46:00.355098+0000","last_peered":"2024-09-17T16:46:00.355098+0000","last_clean":"2024-09-17T16:46:00.355098+0000","last_became_active":"2024-09-17T16:29:02.177674+0000","last_became_peered":"2024-09-17T16:29:02.177674+0000","last_unstale":"2024-09-17T16:46:00.355098+0000","last_undegraded":"2024-09-17T16:46:00.355098+0000","last_fullsized":"2024-09-17T16:46:00.355098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:42.365472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360691+0000","last_change":"2024-09-17T16:29:02.185360+0000","last_active":"2024-09-17T16:46:00.360691+0000","last_peered":"2024-09-17T16:46:00.360691+0000","last_clean":"2024-09-17T16:46:00.360691+0000","last_became_active":"2024-09-17T16:29:02.184301+0000","last_became_peered":"2024-09-17T16:29:02.184301+0000","last_unstale":"2024-09-17T16:46:00.360691+0000","last_undegraded":"2024-09-17T16:46:00.360691+0000","last_fullsized":"2024-09-17T16:46:00.360691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:12.678748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361203+0000","last_change":"2024-09-17T16:29:02.050147+0000","last_active":"2024-09-17T16:46:00.361203+0000","last_peered":"2024-09-17T16:46:00.361203+0000","last_clean":"2024-09-17T16:46:00.361203+0000","last_became_active":"2024-09-17T16:29:02.044488+0000","last_became_peered":"2024-09-17T16:29:02.044488+0000","last_unstale":"2024-09-17T16:46:00.361203+0000","last_undegraded":"2024-09-17T16:46:00.361203+0000","last_fullsized":"2024-09-17T16:46:00.361203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:16.232335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":1490,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243057+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:46:00.243057+0000","last_peered":"2024-09-17T16:46:00.243057+0000","last_clean":"2024-09-17T16:46:00.243057+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:46:00.243057+0000","last_undegraded":"2024-09-17T16:46:00.243057+0000","last_fullsized":"2024-09-17T16:46:00.243057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360719+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:46:00.360719+0000","last_peered":"2024-09-17T16:46:00.360719+0000","last_clean":"2024-09-17T16:46:00.360719+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:46:00.360719+0000","last_undegraded":"2024-09-17T16:46:00.360719+0000","last_fullsized":"2024-09-17T16:46:00.360719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355189+0000","last_change":"2024-09-17T16:29:02.189168+0000","last_active":"2024-09-17T16:46:00.355189+0000","last_peered":"2024-09-17T16:46:00.355189+0000","last_clean":"2024-09-17T16:46:00.355189+0000","last_became_active":"2024-09-17T16:29:02.186575+0000","last_became_peered":"2024-09-17T16:29:02.186575+0000","last_unstale":"2024-09-17T16:46:00.355189+0000","last_undegraded":"2024-09-17T16:46:00.355189+0000","last_fullsized":"2024-09-17T16:46:00.355189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:10.592381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1399,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242887+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:46:00.242887+0000","last_peered":"2024-09-17T16:46:00.242887+0000","last_clean":"2024-09-17T16:46:00.242887+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:46:00.242887+0000","last_undegraded":"2024-09-17T16:46:00.242887+0000","last_fullsized":"2024-09-17T16:46:00.242887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361012+0000","last_change":"2024-09-17T16:29:02.070104+0000","last_active":"2024-09-17T16:46:00.361012+0000","last_peered":"2024-09-17T16:46:00.361012+0000","last_clean":"2024-09-17T16:46:00.361012+0000","last_became_active":"2024-09-17T16:29:02.067560+0000","last_became_peered":"2024-09-17T16:29:02.067560+0000","last_unstale":"2024-09-17T16:46:00.361012+0000","last_undegraded":"2024-09-17T16:46:00.361012+0000","last_fullsized":"2024-09-17T16:46:00.361012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:51.122320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1397,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239633+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:46:00.239633+0000","last_peered":"2024-09-17T16:46:00.239633+0000","last_clean":"2024-09-17T16:46:00.239633+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:46:00.239633+0000","last_undegraded":"2024-09-17T16:46:00.239633+0000","last_fullsized":"2024-09-17T16:46:00.239633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355522+0000","last_change":"2024-09-17T16:29:02.175917+0000","last_active":"2024-09-17T16:46:00.355522+0000","last_peered":"2024-09-17T16:46:00.355522+0000","last_clean":"2024-09-17T16:46:00.355522+0000","last_became_active":"2024-09-17T16:29:02.175702+0000","last_became_peered":"2024-09-17T16:29:02.175702+0000","last_unstale":"2024-09-17T16:46:00.355522+0000","last_undegraded":"2024-09-17T16:46:00.355522+0000","last_fullsized":"2024-09-17T16:46:00.355522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:10.503343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1388,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355872+0000","last_change":"2024-09-17T16:29:02.160011+0000","last_active":"2024-09-17T16:46:00.355872+0000","last_peered":"2024-09-17T16:46:00.355872+0000","last_clean":"2024-09-17T16:46:00.355872+0000","last_became_active":"2024-09-17T16:29:02.158196+0000","last_became_peered":"2024-09-17T16:29:02.158196+0000","last_unstale":"2024-09-17T16:46:00.355872+0000","last_undegraded":"2024-09-17T16:46:00.355872+0000","last_fullsized":"2024-09-17T16:46:00.355872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:11.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355364+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:46:00.355364+0000","last_peered":"2024-09-17T16:46:00.355364+0000","last_clean":"2024-09-17T16:46:00.355364+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:46:00.355364+0000","last_undegraded":"2024-09-17T16:46:00.355364+0000","last_fullsized":"2024-09-17T16:46:00.355364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"22'7","reported_seq":1385,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354809+0000","last_change":"2024-09-17T16:29:02.118963+0000","last_active":"2024-09-17T16:46:00.354809+0000","last_peered":"2024-09-17T16:46:00.354809+0000","last_clean":"2024-09-17T16:46:00.354809+0000","last_became_active":"2024-09-17T16:29:02.116933+0000","last_became_peered":"2024-09-17T16:29:02.116933+0000","last_unstale":"2024-09-17T16:46:00.354809+0000","last_undegraded":"2024-09-17T16:46:00.354809+0000","last_fullsized":"2024-09-17T16:46:00.354809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:02.801705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243451+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:46:00.243451+0000","last_peered":"2024-09-17T16:46:00.243451+0000","last_clean":"2024-09-17T16:46:00.243451+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:46:00.243451+0000","last_undegraded":"2024-09-17T16:46:00.243451+0000","last_fullsized":"2024-09-17T16:46:00.243451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243022+0000","last_change":"2024-09-17T16:29:02.137515+0000","last_active":"2024-09-17T16:46:00.243022+0000","last_peered":"2024-09-17T16:46:00.243022+0000","last_clean":"2024-09-17T16:46:00.243022+0000","last_became_active":"2024-09-17T16:29:02.136808+0000","last_became_peered":"2024-09-17T16:29:02.136808+0000","last_unstale":"2024-09-17T16:46:00.243022+0000","last_undegraded":"2024-09-17T16:46:00.243022+0000","last_fullsized":"2024-09-17T16:46:00.243022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:14.068157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355638+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:46:00.355638+0000","last_peered":"2024-09-17T16:46:00.355638+0000","last_clean":"2024-09-17T16:46:00.355638+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:46:00.355638+0000","last_undegraded":"2024-09-17T16:46:00.355638+0000","last_fullsized":"2024-09-17T16:46:00.355638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1369,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360439+0000","last_change":"2024-09-17T16:29:02.049672+0000","last_active":"2024-09-17T16:46:00.360439+0000","last_peered":"2024-09-17T16:46:00.360439+0000","last_clean":"2024-09-17T16:46:00.360439+0000","last_became_active":"2024-09-17T16:29:02.048114+0000","last_became_peered":"2024-09-17T16:29:02.048114+0000","last_unstale":"2024-09-17T16:46:00.360439+0000","last_undegraded":"2024-09-17T16:46:00.360439+0000","last_fullsized":"2024-09-17T16:46:00.360439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:48.297807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243579+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:46:00.243579+0000","last_peered":"2024-09-17T16:46:00.243579+0000","last_clean":"2024-09-17T16:46:00.243579+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:46:00.243579+0000","last_undegraded":"2024-09-17T16:46:00.243579+0000","last_fullsized":"2024-09-17T16:46:00.243579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239579+0000","last_change":"2024-09-17T16:29:02.135845+0000","last_active":"2024-09-17T16:46:00.239579+0000","last_peered":"2024-09-17T16:46:00.239579+0000","last_clean":"2024-09-17T16:46:00.239579+0000","last_became_active":"2024-09-17T16:29:02.132790+0000","last_became_peered":"2024-09-17T16:29:02.132790+0000","last_unstale":"2024-09-17T16:46:00.239579+0000","last_undegraded":"2024-09-17T16:46:00.239579+0000","last_fullsized":"2024-09-17T16:46:00.239579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:19.264101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355839+0000","last_change":"2024-09-17T16:29:02.096285+0000","last_active":"2024-09-17T16:46:00.355839+0000","last_peered":"2024-09-17T16:46:00.355839+0000","last_clean":"2024-09-17T16:46:00.355839+0000","last_became_active":"2024-09-17T16:29:02.096072+0000","last_became_peered":"2024-09-17T16:29:02.096072+0000","last_unstale":"2024-09-17T16:46:00.355839+0000","last_undegraded":"2024-09-17T16:46:00.355839+0000","last_fullsized":"2024-09-17T16:46:00.355839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:56.215507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239869+0000","last_change":"2024-09-17T16:29:02.165494+0000","last_active":"2024-09-17T16:46:00.239869+0000","last_peered":"2024-09-17T16:46:00.239869+0000","last_clean":"2024-09-17T16:46:00.239869+0000","last_became_active":"2024-09-17T16:29:02.165109+0000","last_became_peered":"2024-09-17T16:29:02.165109+0000","last_unstale":"2024-09-17T16:46:00.239869+0000","last_undegraded":"2024-09-17T16:46:00.239869+0000","last_fullsized":"2024-09-17T16:46:00.239869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:38.687988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360820+0000","last_change":"2024-09-17T16:29:02.052616+0000","last_active":"2024-09-17T16:46:00.360820+0000","last_peered":"2024-09-17T16:46:00.360820+0000","last_clean":"2024-09-17T16:46:00.360820+0000","last_became_active":"2024-09-17T16:29:02.050385+0000","last_became_peered":"2024-09-17T16:29:02.050385+0000","last_unstale":"2024-09-17T16:46:00.360820+0000","last_undegraded":"2024-09-17T16:46:00.360820+0000","last_fullsized":"2024-09-17T16:46:00.360820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:14.785603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355442+0000","last_change":"2024-09-17T16:29:02.192467+0000","last_active":"2024-09-17T16:46:00.355442+0000","last_peered":"2024-09-17T16:46:00.355442+0000","last_clean":"2024-09-17T16:46:00.355442+0000","last_became_active":"2024-09-17T16:29:02.192181+0000","last_became_peered":"2024-09-17T16:29:02.192181+0000","last_unstale":"2024-09-17T16:46:00.355442+0000","last_undegraded":"2024-09-17T16:46:00.355442+0000","last_fullsized":"2024-09-17T16:46:00.355442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:14.890573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243241+0000","last_change":"2024-09-17T16:29:02.123632+0000","last_active":"2024-09-17T16:46:00.243241+0000","last_peered":"2024-09-17T16:46:00.243241+0000","last_clean":"2024-09-17T16:46:00.243241+0000","last_became_active":"2024-09-17T16:29:02.119724+0000","last_became_peered":"2024-09-17T16:29:02.119724+0000","last_unstale":"2024-09-17T16:46:00.243241+0000","last_undegraded":"2024-09-17T16:46:00.243241+0000","last_fullsized":"2024-09-17T16:46:00.243241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:59.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239848+0000","last_change":"2024-09-17T16:29:02.075907+0000","last_active":"2024-09-17T16:46:00.239848+0000","last_peered":"2024-09-17T16:46:00.239848+0000","last_clean":"2024-09-17T16:46:00.239848+0000","last_became_active":"2024-09-17T16:29:02.075599+0000","last_became_peered":"2024-09-17T16:29:02.075599+0000","last_unstale":"2024-09-17T16:46:00.239848+0000","last_undegraded":"2024-09-17T16:46:00.239848+0000","last_fullsized":"2024-09-17T16:46:00.239848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:34:21.469780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243362+0000","last_change":"2024-09-17T16:29:02.074165+0000","last_active":"2024-09-17T16:46:00.243362+0000","last_peered":"2024-09-17T16:46:00.243362+0000","last_clean":"2024-09-17T16:46:00.243362+0000","last_became_active":"2024-09-17T16:29:02.073908+0000","last_became_peered":"2024-09-17T16:29:02.073908+0000","last_unstale":"2024-09-17T16:46:00.243362+0000","last_undegraded":"2024-09-17T16:46:00.243362+0000","last_fullsized":"2024-09-17T16:46:00.243362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:09:53.379585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:29:02.151691+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:29:02.151449+0000","last_became_peered":"2024-09-17T16:29:02.151449+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:00.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356257+0000","last_change":"2024-09-17T16:29:02.159400+0000","last_active":"2024-09-17T16:46:00.356257+0000","last_peered":"2024-09-17T16:46:00.356257+0000","last_clean":"2024-09-17T16:46:00.356257+0000","last_became_active":"2024-09-17T16:29:02.157013+0000","last_became_peered":"2024-09-17T16:29:02.157013+0000","last_unstale":"2024-09-17T16:46:00.356257+0000","last_undegraded":"2024-09-17T16:46:00.356257+0000","last_fullsized":"2024-09-17T16:46:00.356257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.554975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356342+0000","last_change":"2024-09-17T16:29:02.146313+0000","last_active":"2024-09-17T16:46:00.356342+0000","last_peered":"2024-09-17T16:46:00.356342+0000","last_clean":"2024-09-17T16:46:00.356342+0000","last_became_active":"2024-09-17T16:29:02.144760+0000","last_became_peered":"2024-09-17T16:29:02.144760+0000","last_unstale":"2024-09-17T16:46:00.356342+0000","last_undegraded":"2024-09-17T16:46:00.356342+0000","last_fullsized":"2024-09-17T16:46:00.356342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:19.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360880+0000","last_change":"2024-09-17T16:29:02.050143+0000","last_active":"2024-09-17T16:46:00.360880+0000","last_peered":"2024-09-17T16:46:00.360880+0000","last_clean":"2024-09-17T16:46:00.360880+0000","last_became_active":"2024-09-17T16:29:02.049439+0000","last_became_peered":"2024-09-17T16:29:02.049439+0000","last_unstale":"2024-09-17T16:46:00.360880+0000","last_undegraded":"2024-09-17T16:46:00.360880+0000","last_fullsized":"2024-09-17T16:46:00.360880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:44.935606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607779,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607780,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10856,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11116544,"data_stored":6439173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607780,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78132,"kb_used_data":10852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293708,"statfs":{"total":96636764160,"available":96556756992,"internally_reserved":0,"allocated":11112448,"data_stored":6439168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81716,"kb_used_data":10276,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290124,"statfs":{"total":96636764160,"available":96553086976,"internally_reserved":0,"allocated":10522624,"data_stored":5848819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77560,"kb_used_data":10280,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294280,"statfs":{"total":96636764160,"available":96557342720,"internally_reserved":0,"allocated":10526720,"data_stored":5848824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:46:37.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 10 monclient: get_monmap_and_config 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 10 monclient: monmap: 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:37.542517+0000 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:37.542517+0000 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.546 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:37.547 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 1 Processor -- start 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 1 -- start start 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.538+0000 7f33bcc42640 10 monclient: init 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient: _add_conns ranks=[7,6,3,5,4,2,0,10,9,8,1] 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33b80a68e0 0x7f33b80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-i con 0x7f33b80a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-d con 0x7f33b80a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b81396a0 0x7f33b8139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-g con 0x7f33b81396a0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f33b80a3fc0 con 0x7f33b81396a0 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f33b80a63c0 con 0x7f33b80a4910 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f33b80a7740 con 0x7f33b80a68e0 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient(hunting): authenticate will time out at 2080.808212s 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:54862/0 (socket says 172.21.15.105:54862) 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 -- 172.21.15.105:0/1088224020 learned_addr learned my addr 172.21.15.105:0/1088224020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): get_auth_request con 0x7f33b80a4910 auth_method 0 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b659c640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33b80a68e0 0x7f33b80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b6d9d640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b81396a0 0x7f33b8139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b6d9d640 10 monclient(hunting): get_auth_request con 0x7f33b81396a0 auth_method 0 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b659c640 10 monclient(hunting): get_auth_request con 0x7f33b80a68e0 auth_method 0 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_done global_id 94434 payload 274 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 -- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33b80a68e0 msgr2=0x7f33b80a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33b80a68e0 0x7f33b80a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 -- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b81396a0 msgr2=0x7f33b8139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b81396a0 0x7f33b8139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 monclient: found mon.noname-d 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 -- 172.21.15.105:0/1088224020 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b806ec10 con 0x7f33b80a4910 2024-09-17T16:46:37.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b5d9b640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33a80098a0 tx=0x7f33a801cea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e5c949c8ecf4ecf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 1 -- 172.21.15.105:0/1088224020 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33a80b0070 con 0x7f33b80a4910 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: dump: 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.555 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 5 monclient: authenticate success, global_id 94434 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 1 -- 172.21.15.105:0/1088224020 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a80aa810 con 0x7f33b80a4910 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:37.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 4 set_mon_vals no callback set 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 1 -- 172.21.15.105:0/1088224020 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33a80a9300 con 0x7f33b80a4910 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33b559a640 10 monclient: dump: 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.557 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 10 monclient: shutdown 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- 172.21.15.105:0/1088224020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 msgr2=0x7f33b80a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33a80098a0 tx=0x7f33a801cea0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- 172.21.15.105:0/1088224020 shutdown_connections 2024-09-17T16:46:37.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b81396a0 0x7f33b8139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b80a4910 0x7f33b80a7200 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1088224020 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f33b80a68e0 0x7f33b80a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.542+0000 7f33bcc42640 1 -- 172.21.15.105:0/1088224020 >> 172.21.15.105:0/1088224020 conn(0x7f33b8068ba0 msgr2=0x7f33b8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- 172.21.15.105:0/1088224020 shutdown_connections 2024-09-17T16:46:37.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- 172.21.15.105:0/1088224020 wait complete. 2024-09-17T16:46:37.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: monmap: 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.564 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 librados: starting msgr at 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 librados: starting objecter 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 Processor -- start 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- start start 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 librados: setting wanted keys 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 librados: calling monclient init 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: init 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _add_conns ranks=[8,4,6,0,3,2,1,5,7] 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-e con 0x7f33b80a4910 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 0x7f33b81d1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-c con 0x7f33b80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 0x7f33b81d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): picked mon.noname-d con 0x7f33b81396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f33b8143320 con 0x7f33b80a68e0 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f33b806e630 con 0x7f33b81396a0 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f33b81434a0 con 0x7f33b80a4910 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient(hunting): authenticate will time out at 2080.813618s 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 0x7f33b81d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:51254/0 (socket says 172.21.15.105:51254) 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 1 -- 172.21.15.105:0/1059338260 learned_addr learned my addr 172.21.15.105:0/1059338260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): get_auth_request con 0x7f33b80a68e0 auth_method 0 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 0x7f33b81d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): get_auth_request con 0x7f33b80a4910 auth_method 0 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): get_auth_request con 0x7f33b81396a0 auth_method 0 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient(hunting): handle_auth_done global_id 94443 payload 824 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 -- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 msgr2=0x7f33b8122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 -- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 msgr2=0x7f33b81d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 0x7f33b81d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 monclient: found mon.noname-d 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 -- 172.21.15.105:0/1059338260 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8123a70 con 0x7f33b81396a0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b5d9b640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 0x7f33b81d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b659c640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b6d9d640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 0x7f33b81d1e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33b0007d20 tx=0x7f33b000e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 client.?.objecter ms_handle_connect 0x7f33b81396a0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33b00098b0 con 0x7f33b81396a0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: dump: 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.570 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 5 monclient: authenticate success, global_id 94443 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b000edd0 con 0x7f33b81396a0 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _renew_subs 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b8123d50 con 0x7f33b81396a0 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33b0014d00 con 0x7f33b81396a0 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33b4d99640 4 set_mon_vals no callback set 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 client.94443.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 monclient: dump: 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.572 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.573 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _renew_subs 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b80aa100 con 0x7f33b81396a0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 librados: init done 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f33b001b030 con 0x7f33b81396a0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8077c80 con 0x7f33b81396a0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 0x7f3398040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f33b0019020 con 0x7f33b81396a0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 10 client.94443.objecter ms_dispatch 0x7f33b8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 3 client.94443.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:37.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 3 client.94443.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.546+0000 7f33a6ffd640 20 client.94443.objecter dump_active .. 0 homeless 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.550+0000 7f33b659c640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 0x7f3398040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.550+0000 7f33b659c640 10 monclient: get_auth_request con 0x7f339803e550 auth_method 0 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.550+0000 7f33a6ffd640 10 client.94443.objecter ms_handle_connect 0x7f339803e550 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.550+0000 7f33b659c640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 0x7f3398040a10 secure :-1 s=READY pgs=3677 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bf80 tx=0x7f33ac008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.554+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33b0022060 con 0x7f33b81396a0 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.554+0000 7f33a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.554+0000 7f33a6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.730+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33b8139ae0 con 0x7f339803e550 2024-09-17T16:46:37.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.734+0000 7f33a6ffd640 1 -- 172.21.15.105:0/1059338260 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+114046 (secure 0 0 0) 0x7f33b8139ae0 con 0x7f339803e550 2024-09-17T16:46:37.742 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:46:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:46:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 10 librados: watch_flush enter 2024-09-17T16:46:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 10 librados: watch_flush exit 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 20 client.94443.objecter shutdown clearing up homeless session... 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 10 client.94443.objecter successfully canceled tick 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 msgr2=0x7f3398040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 0x7f3398040a10 secure :-1 s=READY pgs=3677 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bf80 tx=0x7f33ac008040 comp rx=0 tx=0).stop 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 10 monclient: shutdown 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 msgr2=0x7f33b81d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 0x7f33b81d1e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33b0007d20 tx=0x7f33b000e4b0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 shutdown_connections 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f339803e550 0x7f3398040a10 unknown :-1 s=CLOSED pgs=3677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f33b81396a0 0x7f33b81d1e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f33b80a68e0 0x7f33b81d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 --2- 172.21.15.105:0/1059338260 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f33b80a4910 0x7f33b8122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 >> 172.21.15.105:0/1059338260 conn(0x7f33b8068ba0 msgr2=0x7f33b80a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 shutdown_connections 2024-09-17T16:46:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 -- 172.21.15.105:0/1059338260 wait complete. 2024-09-17T16:46:37.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.738+0000 7f33bcc42640 1 librados: shutdown 2024-09-17T16:46:37.760 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":1264,"stamp":"2024-09-17T16:46:36.193103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":61,"num_write":110,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470648,"kb_used_data":62808,"kb_used_omap":9,"kb_used_meta":407734,"kb_avail":565760392,"statfs":{"total":579820584960,"available":579338641408,"internally_reserved":0,"allocated":64315392,"data_stored":36273584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":417520288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004247"},"pg_stats":[{"pgid":"6.16","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356126+0000","last_change":"2024-09-17T16:29:02.018311+0000","last_active":"2024-09-17T16:46:00.356126+0000","last_peered":"2024-09-17T16:46:00.356126+0000","last_clean":"2024-09-17T16:46:00.356126+0000","last_became_active":"2024-09-17T16:29:02.017756+0000","last_became_peered":"2024-09-17T16:29:02.017756+0000","last_unstale":"2024-09-17T16:46:00.356126+0000","last_undegraded":"2024-09-17T16:46:00.356126+0000","last_fullsized":"2024-09-17T16:46:00.356126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:14.467205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360583+0000","last_change":"2024-09-17T16:29:02.054931+0000","last_active":"2024-09-17T16:46:00.360583+0000","last_peered":"2024-09-17T16:46:00.360583+0000","last_clean":"2024-09-17T16:46:00.360583+0000","last_became_active":"2024-09-17T16:29:02.054559+0000","last_became_peered":"2024-09-17T16:29:02.054559+0000","last_unstale":"2024-09-17T16:46:00.360583+0000","last_undegraded":"2024-09-17T16:46:00.360583+0000","last_fullsized":"2024-09-17T16:46:00.360583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:37.538511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239713+0000","last_change":"2024-09-17T16:29:02.079133+0000","last_active":"2024-09-17T16:46:00.239713+0000","last_peered":"2024-09-17T16:46:00.239713+0000","last_clean":"2024-09-17T16:46:00.239713+0000","last_became_active":"2024-09-17T16:29:02.076410+0000","last_became_peered":"2024-09-17T16:29:02.076410+0000","last_unstale":"2024-09-17T16:46:00.239713+0000","last_undegraded":"2024-09-17T16:46:00.239713+0000","last_fullsized":"2024-09-17T16:46:00.239713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:04.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243332+0000","last_change":"2024-09-17T16:29:02.123528+0000","last_active":"2024-09-17T16:46:00.243332+0000","last_peered":"2024-09-17T16:46:00.243332+0000","last_clean":"2024-09-17T16:46:00.243332+0000","last_became_active":"2024-09-17T16:29:02.122292+0000","last_became_peered":"2024-09-17T16:29:02.122292+0000","last_unstale":"2024-09-17T16:46:00.243332+0000","last_undegraded":"2024-09-17T16:46:00.243332+0000","last_fullsized":"2024-09-17T16:46:00.243332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:48.555269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360740+0000","last_change":"2024-09-17T16:29:02.064488+0000","last_active":"2024-09-17T16:46:00.360740+0000","last_peered":"2024-09-17T16:46:00.360740+0000","last_clean":"2024-09-17T16:46:00.360740+0000","last_became_active":"2024-09-17T16:29:02.064232+0000","last_became_peered":"2024-09-17T16:29:02.064232+0000","last_unstale":"2024-09-17T16:46:00.360740+0000","last_undegraded":"2024-09-17T16:46:00.360740+0000","last_fullsized":"2024-09-17T16:46:00.360740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:14.884902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354984+0000","last_change":"2024-09-17T16:29:02.121725+0000","last_active":"2024-09-17T16:46:00.354984+0000","last_peered":"2024-09-17T16:46:00.354984+0000","last_clean":"2024-09-17T16:46:00.354984+0000","last_became_active":"2024-09-17T16:29:02.116801+0000","last_became_peered":"2024-09-17T16:29:02.116801+0000","last_unstale":"2024-09-17T16:46:00.354984+0000","last_undegraded":"2024-09-17T16:46:00.354984+0000","last_fullsized":"2024-09-17T16:46:00.354984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:23.905864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242889+0000","last_change":"2024-09-17T16:29:02.131256+0000","last_active":"2024-09-17T16:46:00.242889+0000","last_peered":"2024-09-17T16:46:00.242889+0000","last_clean":"2024-09-17T16:46:00.242889+0000","last_became_active":"2024-09-17T16:29:02.129701+0000","last_became_peered":"2024-09-17T16:29:02.129701+0000","last_unstale":"2024-09-17T16:46:00.242889+0000","last_undegraded":"2024-09-17T16:46:00.242889+0000","last_fullsized":"2024-09-17T16:46:00.242889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:34:10.375884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356246+0000","last_change":"2024-09-17T16:29:02.079372+0000","last_active":"2024-09-17T16:46:00.356246+0000","last_peered":"2024-09-17T16:46:00.356246+0000","last_clean":"2024-09-17T16:46:00.356246+0000","last_became_active":"2024-09-17T16:29:02.078833+0000","last_became_peered":"2024-09-17T16:29:02.078833+0000","last_unstale":"2024-09-17T16:46:00.356246+0000","last_undegraded":"2024-09-17T16:46:00.356246+0000","last_fullsized":"2024-09-17T16:46:00.356246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:50.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360568+0000","last_change":"2024-09-17T16:29:02.185295+0000","last_active":"2024-09-17T16:46:00.360568+0000","last_peered":"2024-09-17T16:46:00.360568+0000","last_clean":"2024-09-17T16:46:00.360568+0000","last_became_active":"2024-09-17T16:29:02.179921+0000","last_became_peered":"2024-09-17T16:29:02.179921+0000","last_unstale":"2024-09-17T16:46:00.360568+0000","last_undegraded":"2024-09-17T16:46:00.360568+0000","last_fullsized":"2024-09-17T16:46:00.360568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:18.876647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.240108+0000","last_change":"2024-09-17T16:29:02.165149+0000","last_active":"2024-09-17T16:46:00.240108+0000","last_peered":"2024-09-17T16:46:00.240108+0000","last_clean":"2024-09-17T16:46:00.240108+0000","last_became_active":"2024-09-17T16:29:02.164714+0000","last_became_peered":"2024-09-17T16:29:02.164714+0000","last_unstale":"2024-09-17T16:46:00.240108+0000","last_undegraded":"2024-09-17T16:46:00.240108+0000","last_fullsized":"2024-09-17T16:46:00.240108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:19.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355098+0000","last_change":"2024-09-17T16:29:02.177944+0000","last_active":"2024-09-17T16:46:00.355098+0000","last_peered":"2024-09-17T16:46:00.355098+0000","last_clean":"2024-09-17T16:46:00.355098+0000","last_became_active":"2024-09-17T16:29:02.177674+0000","last_became_peered":"2024-09-17T16:29:02.177674+0000","last_unstale":"2024-09-17T16:46:00.355098+0000","last_undegraded":"2024-09-17T16:46:00.355098+0000","last_fullsized":"2024-09-17T16:46:00.355098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:42.365472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360691+0000","last_change":"2024-09-17T16:29:02.185360+0000","last_active":"2024-09-17T16:46:00.360691+0000","last_peered":"2024-09-17T16:46:00.360691+0000","last_clean":"2024-09-17T16:46:00.360691+0000","last_became_active":"2024-09-17T16:29:02.184301+0000","last_became_peered":"2024-09-17T16:29:02.184301+0000","last_unstale":"2024-09-17T16:46:00.360691+0000","last_undegraded":"2024-09-17T16:46:00.360691+0000","last_fullsized":"2024-09-17T16:46:00.360691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:12.678748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361203+0000","last_change":"2024-09-17T16:29:02.050147+0000","last_active":"2024-09-17T16:46:00.361203+0000","last_peered":"2024-09-17T16:46:00.361203+0000","last_clean":"2024-09-17T16:46:00.361203+0000","last_became_active":"2024-09-17T16:29:02.044488+0000","last_became_peered":"2024-09-17T16:29:02.044488+0000","last_unstale":"2024-09-17T16:46:00.361203+0000","last_undegraded":"2024-09-17T16:46:00.361203+0000","last_fullsized":"2024-09-17T16:46:00.361203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:16.232335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":1490,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243057+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:46:00.243057+0000","last_peered":"2024-09-17T16:46:00.243057+0000","last_clean":"2024-09-17T16:46:00.243057+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:46:00.243057+0000","last_undegraded":"2024-09-17T16:46:00.243057+0000","last_fullsized":"2024-09-17T16:46:00.243057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360719+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:46:00.360719+0000","last_peered":"2024-09-17T16:46:00.360719+0000","last_clean":"2024-09-17T16:46:00.360719+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:46:00.360719+0000","last_undegraded":"2024-09-17T16:46:00.360719+0000","last_fullsized":"2024-09-17T16:46:00.360719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355189+0000","last_change":"2024-09-17T16:29:02.189168+0000","last_active":"2024-09-17T16:46:00.355189+0000","last_peered":"2024-09-17T16:46:00.355189+0000","last_clean":"2024-09-17T16:46:00.355189+0000","last_became_active":"2024-09-17T16:29:02.186575+0000","last_became_peered":"2024-09-17T16:29:02.186575+0000","last_unstale":"2024-09-17T16:46:00.355189+0000","last_undegraded":"2024-09-17T16:46:00.355189+0000","last_fullsized":"2024-09-17T16:46:00.355189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:10.592381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1399,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242887+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:46:00.242887+0000","last_peered":"2024-09-17T16:46:00.242887+0000","last_clean":"2024-09-17T16:46:00.242887+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:46:00.242887+0000","last_undegraded":"2024-09-17T16:46:00.242887+0000","last_fullsized":"2024-09-17T16:46:00.242887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361012+0000","last_change":"2024-09-17T16:29:02.070104+0000","last_active":"2024-09-17T16:46:00.361012+0000","last_peered":"2024-09-17T16:46:00.361012+0000","last_clean":"2024-09-17T16:46:00.361012+0000","last_became_active":"2024-09-17T16:29:02.067560+0000","last_became_peered":"2024-09-17T16:29:02.067560+0000","last_unstale":"2024-09-17T16:46:00.361012+0000","last_undegraded":"2024-09-17T16:46:00.361012+0000","last_fullsized":"2024-09-17T16:46:00.361012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:51.122320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1397,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239633+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:46:00.239633+0000","last_peered":"2024-09-17T16:46:00.239633+0000","last_clean":"2024-09-17T16:46:00.239633+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:46:00.239633+0000","last_undegraded":"2024-09-17T16:46:00.239633+0000","last_fullsized":"2024-09-17T16:46:00.239633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355522+0000","last_change":"2024-09-17T16:29:02.175917+0000","last_active":"2024-09-17T16:46:00.355522+0000","last_peered":"2024-09-17T16:46:00.355522+0000","last_clean":"2024-09-17T16:46:00.355522+0000","last_became_active":"2024-09-17T16:29:02.175702+0000","last_became_peered":"2024-09-17T16:29:02.175702+0000","last_unstale":"2024-09-17T16:46:00.355522+0000","last_undegraded":"2024-09-17T16:46:00.355522+0000","last_fullsized":"2024-09-17T16:46:00.355522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:10.503343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1388,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355872+0000","last_change":"2024-09-17T16:29:02.160011+0000","last_active":"2024-09-17T16:46:00.355872+0000","last_peered":"2024-09-17T16:46:00.355872+0000","last_clean":"2024-09-17T16:46:00.355872+0000","last_became_active":"2024-09-17T16:29:02.158196+0000","last_became_peered":"2024-09-17T16:29:02.158196+0000","last_unstale":"2024-09-17T16:46:00.355872+0000","last_undegraded":"2024-09-17T16:46:00.355872+0000","last_fullsized":"2024-09-17T16:46:00.355872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:11.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355364+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:46:00.355364+0000","last_peered":"2024-09-17T16:46:00.355364+0000","last_clean":"2024-09-17T16:46:00.355364+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:46:00.355364+0000","last_undegraded":"2024-09-17T16:46:00.355364+0000","last_fullsized":"2024-09-17T16:46:00.355364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"22'7","reported_seq":1385,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354809+0000","last_change":"2024-09-17T16:29:02.118963+0000","last_active":"2024-09-17T16:46:00.354809+0000","last_peered":"2024-09-17T16:46:00.354809+0000","last_clean":"2024-09-17T16:46:00.354809+0000","last_became_active":"2024-09-17T16:29:02.116933+0000","last_became_peered":"2024-09-17T16:29:02.116933+0000","last_unstale":"2024-09-17T16:46:00.354809+0000","last_undegraded":"2024-09-17T16:46:00.354809+0000","last_fullsized":"2024-09-17T16:46:00.354809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:02.801705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243451+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:46:00.243451+0000","last_peered":"2024-09-17T16:46:00.243451+0000","last_clean":"2024-09-17T16:46:00.243451+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:46:00.243451+0000","last_undegraded":"2024-09-17T16:46:00.243451+0000","last_fullsized":"2024-09-17T16:46:00.243451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243022+0000","last_change":"2024-09-17T16:29:02.137515+0000","last_active":"2024-09-17T16:46:00.243022+0000","last_peered":"2024-09-17T16:46:00.243022+0000","last_clean":"2024-09-17T16:46:00.243022+0000","last_became_active":"2024-09-17T16:29:02.136808+0000","last_became_peered":"2024-09-17T16:29:02.136808+0000","last_unstale":"2024-09-17T16:46:00.243022+0000","last_undegraded":"2024-09-17T16:46:00.243022+0000","last_fullsized":"2024-09-17T16:46:00.243022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:14.068157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355638+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:46:00.355638+0000","last_peered":"2024-09-17T16:46:00.355638+0000","last_clean":"2024-09-17T16:46:00.355638+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:46:00.355638+0000","last_undegraded":"2024-09-17T16:46:00.355638+0000","last_fullsized":"2024-09-17T16:46:00.355638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1369,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360439+0000","last_change":"2024-09-17T16:29:02.049672+0000","last_active":"2024-09-17T16:46:00.360439+0000","last_peered":"2024-09-17T16:46:00.360439+0000","last_clean":"2024-09-17T16:46:00.360439+0000","last_became_active":"2024-09-17T16:29:02.048114+0000","last_became_peered":"2024-09-17T16:29:02.048114+0000","last_unstale":"2024-09-17T16:46:00.360439+0000","last_undegraded":"2024-09-17T16:46:00.360439+0000","last_fullsized":"2024-09-17T16:46:00.360439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:48.297807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243579+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:46:00.243579+0000","last_peered":"2024-09-17T16:46:00.243579+0000","last_clean":"2024-09-17T16:46:00.243579+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:46:00.243579+0000","last_undegraded":"2024-09-17T16:46:00.243579+0000","last_fullsized":"2024-09-17T16:46:00.243579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239579+0000","last_change":"2024-09-17T16:29:02.135845+0000","last_active":"2024-09-17T16:46:00.239579+0000","last_peered":"2024-09-17T16:46:00.239579+0000","last_clean":"2024-09-17T16:46:00.239579+0000","last_became_active":"2024-09-17T16:29:02.132790+0000","last_became_peered":"2024-09-17T16:29:02.132790+0000","last_unstale":"2024-09-17T16:46:00.239579+0000","last_undegraded":"2024-09-17T16:46:00.239579+0000","last_fullsized":"2024-09-17T16:46:00.239579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:19.264101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355839+0000","last_change":"2024-09-17T16:29:02.096285+0000","last_active":"2024-09-17T16:46:00.355839+0000","last_peered":"2024-09-17T16:46:00.355839+0000","last_clean":"2024-09-17T16:46:00.355839+0000","last_became_active":"2024-09-17T16:29:02.096072+0000","last_became_peered":"2024-09-17T16:29:02.096072+0000","last_unstale":"2024-09-17T16:46:00.355839+0000","last_undegraded":"2024-09-17T16:46:00.355839+0000","last_fullsized":"2024-09-17T16:46:00.355839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:56.215507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239869+0000","last_change":"2024-09-17T16:29:02.165494+0000","last_active":"2024-09-17T16:46:00.239869+0000","last_peered":"2024-09-17T16:46:00.239869+0000","last_clean":"2024-09-17T16:46:00.239869+0000","last_became_active":"2024-09-17T16:29:02.165109+0000","last_became_peered":"2024-09-17T16:29:02.165109+0000","last_unstale":"2024-09-17T16:46:00.239869+0000","last_undegraded":"2024-09-17T16:46:00.239869+0000","last_fullsized":"2024-09-17T16:46:00.239869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:38.687988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360820+0000","last_change":"2024-09-17T16:29:02.052616+0000","last_active":"2024-09-17T16:46:00.360820+0000","last_peered":"2024-09-17T16:46:00.360820+0000","last_clean":"2024-09-17T16:46:00.360820+0000","last_became_active":"2024-09-17T16:29:02.050385+0000","last_became_peered":"2024-09-17T16:29:02.050385+0000","last_unstale":"2024-09-17T16:46:00.360820+0000","last_undegraded":"2024-09-17T16:46:00.360820+0000","last_fullsized":"2024-09-17T16:46:00.360820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:14.785603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355442+0000","last_change":"2024-09-17T16:29:02.192467+0000","last_active":"2024-09-17T16:46:00.355442+0000","last_peered":"2024-09-17T16:46:00.355442+0000","last_clean":"2024-09-17T16:46:00.355442+0000","last_became_active":"2024-09-17T16:29:02.192181+0000","last_became_peered":"2024-09-17T16:29:02.192181+0000","last_unstale":"2024-09-17T16:46:00.355442+0000","last_undegraded":"2024-09-17T16:46:00.355442+0000","last_fullsized":"2024-09-17T16:46:00.355442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:14.890573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243241+0000","last_change":"2024-09-17T16:29:02.123632+0000","last_active":"2024-09-17T16:46:00.243241+0000","last_peered":"2024-09-17T16:46:00.243241+0000","last_clean":"2024-09-17T16:46:00.243241+0000","last_became_active":"2024-09-17T16:29:02.119724+0000","last_became_peered":"2024-09-17T16:29:02.119724+0000","last_unstale":"2024-09-17T16:46:00.243241+0000","last_undegraded":"2024-09-17T16:46:00.243241+0000","last_fullsized":"2024-09-17T16:46:00.243241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:59.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239848+0000","last_change":"2024-09-17T16:29:02.075907+0000","last_active":"2024-09-17T16:46:00.239848+0000","last_peered":"2024-09-17T16:46:00.239848+0000","last_clean":"2024-09-17T16:46:00.239848+0000","last_became_active":"2024-09-17T16:29:02.075599+0000","last_became_peered":"2024-09-17T16:29:02.075599+0000","last_unstale":"2024-09-17T16:46:00.239848+0000","last_undegraded":"2024-09-17T16:46:00.239848+0000","last_fullsized":"2024-09-17T16:46:00.239848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:34:21.469780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243362+0000","last_change":"2024-09-17T16:29:02.074165+0000","last_active":"2024-09-17T16:46:00.243362+0000","last_peered":"2024-09-17T16:46:00.243362+0000","last_clean":"2024-09-17T16:46:00.243362+0000","last_became_active":"2024-09-17T16:29:02.073908+0000","last_became_peered":"2024-09-17T16:29:02.073908+0000","last_unstale":"2024-09-17T16:46:00.243362+0000","last_undegraded":"2024-09-17T16:46:00.243362+0000","last_fullsized":"2024-09-17T16:46:00.243362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:09:53.379585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:29:02.151691+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:29:02.151449+0000","last_became_peered":"2024-09-17T16:29:02.151449+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:00.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356257+0000","last_change":"2024-09-17T16:29:02.159400+0000","last_active":"2024-09-17T16:46:00.356257+0000","last_peered":"2024-09-17T16:46:00.356257+0000","last_clean":"2024-09-17T16:46:00.356257+0000","last_became_active":"2024-09-17T16:29:02.157013+0000","last_became_peered":"2024-09-17T16:29:02.157013+0000","last_unstale":"2024-09-17T16:46:00.356257+0000","last_undegraded":"2024-09-17T16:46:00.356257+0000","last_fullsized":"2024-09-17T16:46:00.356257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.554975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356342+0000","last_change":"2024-09-17T16:29:02.146313+0000","last_active":"2024-09-17T16:46:00.356342+0000","last_peered":"2024-09-17T16:46:00.356342+0000","last_clean":"2024-09-17T16:46:00.356342+0000","last_became_active":"2024-09-17T16:29:02.144760+0000","last_became_peered":"2024-09-17T16:29:02.144760+0000","last_unstale":"2024-09-17T16:46:00.356342+0000","last_undegraded":"2024-09-17T16:46:00.356342+0000","last_fullsized":"2024-09-17T16:46:00.356342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:19.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360880+0000","last_change":"2024-09-17T16:29:02.050143+0000","last_active":"2024-09-17T16:46:00.360880+0000","last_peered":"2024-09-17T16:46:00.360880+0000","last_clean":"2024-09-17T16:46:00.360880+0000","last_became_active":"2024-09-17T16:29:02.049439+0000","last_became_peered":"2024-09-17T16:29:02.049439+0000","last_unstale":"2024-09-17T16:46:00.360880+0000","last_undegraded":"2024-09-17T16:46:00.360880+0000","last_fullsized":"2024-09-17T16:46:00.360880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:44.935606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607779,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607780,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10856,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11116544,"data_stored":6439173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607780,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78132,"kb_used_data":10852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293708,"statfs":{"total":96636764160,"available":96556756992,"internally_reserved":0,"allocated":11112448,"data_stored":6439168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81716,"kb_used_data":10276,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290124,"statfs":{"total":96636764160,"available":96553086976,"internally_reserved":0,"allocated":10522624,"data_stored":5848819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77560,"kb_used_data":10280,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294280,"statfs":{"total":96636764160,"available":96557342720,"internally_reserved":0,"allocated":10526720,"data_stored":5848824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:46:37.762 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T16:46:37.762 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.870+0000 7f8620a4a640 10 monclient: get_monmap_and_config 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.870+0000 7f8620a4a640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.870+0000 7f8620a4a640 10 monclient: monmap: 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:37.875470+0000 2024-09-17T16:46:37.879 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:37.875470+0000 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:37.880 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 Processor -- start 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 -- start start 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient: init 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient: _add_conns ranks=[5,7,2,1,9,0,8,10,6,4,3] 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c0a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-h con 0x7f861c0a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f861c0a4ab0 0x7f861c0a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-i con 0x7f861c0a4ab0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-b con 0x7f861c1397f0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:37.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f861c0a7c40 con 0x7f861c1397f0 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f861c0a7890 con 0x7f861c0a6a30 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f861c0a4080 con 0x7f861c0a4ab0 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8620a4a640 10 monclient(hunting): authenticate will time out at 2081.141525s 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38594/0 (socket says 172.21.15.105:38594) 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 -- 172.21.15.105:0/2709309163 learned_addr learned my addr 172.21.15.105:0/2709309163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): get_auth_request con 0x7f861c1397f0 auth_method 0 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861a59c640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c0a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8619d9b640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f861c0a4ab0 0x7f861c0a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8619d9b640 10 monclient(hunting): get_auth_request con 0x7f861c0a4ab0 auth_method 0 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8619d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8619d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f8619d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861a59c640 10 monclient(hunting): get_auth_request con 0x7f861c0a6a30 auth_method 0 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient(hunting): handle_auth_done global_id 84674 payload 274 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 -- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f861c0a4ab0 msgr2=0x7f861c0a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f861c0a4ab0 0x7f861c0a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 -- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 msgr2=0x7f861c0a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c0a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 monclient: found mon.noname-b 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:37.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.874+0000 7f861ad9d640 1 -- 172.21.15.105:0/2709309163 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c0a37e0 con 0x7f861c1397f0 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8614009eb0 tx=0x7f861401c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930d387645b95823 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 1 -- 172.21.15.105:0/2709309163 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86140090d0 con 0x7f861c1397f0 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: dump: 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.894 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 1 -- 172.21.15.105:0/2709309163 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86140a5af0 con 0x7f861c1397f0 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 5 monclient: authenticate success, global_id 84674 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 1 -- 172.21.15.105:0/2709309163 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86140a5ce0 con 0x7f861c1397f0 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 4 set_mon_vals no callback set 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861959a640 10 monclient: dump: 2024-09-17T16:46:37.896 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: shutdown 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/2709309163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 msgr2=0x7f861c139c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8614009eb0 tx=0x7f861401c8c0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/2709309163 shutdown_connections 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f861c1397f0 0x7f861c139c50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f861c0a4ab0 0x7f861c0a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- 172.21.15.105:0/2709309163 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c0a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/2709309163 >> 172.21.15.105:0/2709309163 conn(0x7f861c0a8d00 msgr2=0x7f861c0a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/2709309163 shutdown_connections 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/2709309163 wait complete. 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: build_initial_monmap 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: monmap: 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 librados: starting msgr at 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 librados: starting objecter 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 Processor -- start 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- start start 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 librados: setting wanted keys 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 librados: calling monclient init 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: init 2024-09-17T16:46:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _add_conns ranks=[8,5,4,1,6,0,7,2,3] 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-e con 0x7f861c0a4ab0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-h con 0x7f861c0a6a30 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f861c1397f0 0x7f861c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): picked mon.noname-c con 0x7f861c1397f0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f861c06e770 con 0x7f861c1397f0 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f861c13a5a0 con 0x7f861c0a6a30 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f861c143260 con 0x7f861c0a4ab0 2024-09-17T16:46:37.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient(hunting): authenticate will time out at 2081.146829s 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41244/0 (socket says 172.21.15.105:41244) 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f861c1397f0 0x7f861c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 -- 172.21.15.105:0/3651776541 learned_addr learned my addr 172.21.15.105:0/3651776541 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): get_auth_request con 0x7f861c0a4ab0 auth_method 0 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): get_auth_request con 0x7f861c1397f0 auth_method 0 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 10 monclient(hunting): get_auth_request con 0x7f861c0a6a30 auth_method 0 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient(hunting): handle_auth_done global_id 55196 payload 824 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 -- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 msgr2=0x7f861c138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c138730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 -- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f861c1397f0 msgr2=0x7f861c13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f861c1397f0 0x7f861c13d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 monclient: found mon.noname-e 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 -- 172.21.15.105:0/3651776541 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c123ae0 con 0x7f861c0a4ab0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8619d9b640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f861a59c640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d800 tx=0x7f860c00dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 client.?.objecter ms_handle_connect 0x7f861c0a4ab0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f860c00bb30 con 0x7f861c0a4ab0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: dump: 2024-09-17T16:46:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 5 monclient: authenticate success, global_id 55196 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c019070 con 0x7f861c0a4ab0 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _renew_subs 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f860c00bcf0 con 0x7f861c0a4ab0 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c123dc0 con 0x7f861c0a4ab0 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8618d99640 4 set_mon_vals no callback set 2024-09-17T16:46:37.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 client.55196.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _renew_subs 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c0a9fc0 con 0x7f861c0a4ab0 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8620a4a640 1 librados: init done 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.878+0000 7f8612ffd640 10 monclient: dump: 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:37.907 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f860c00b6c0 con 0x7f861c0a4ab0 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8620a4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8620a4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8620a4a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c139c50 con 0x7f861c0a4ab0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 0x7f85fc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f860c00f040 con 0x7f861c0a4ab0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 10 client.55196.objecter ms_dispatch 0x7f861c000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 3 client.55196.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 3 client.55196.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 20 client.55196.objecter dump_active .. 0 homeless 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8619d9b640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 0x7f85fc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8619d9b640 10 monclient: get_auth_request con 0x7f85fc03e550 auth_method 0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8612ffd640 10 client.55196.objecter ms_handle_connect 0x7f85fc03e550 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.882+0000 7f8619d9b640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 0x7f85fc040a10 secure :-1 s=READY pgs=3678 cs=0 l=1 rev1=1 crypto rx=0x7f861c0a5ee0 tx=0x7f8604008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.886+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f860c022020 con 0x7f861c0a4ab0 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.886+0000 7f8612ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:37.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:37.886+0000 7f8612ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.050+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f861c1394a0 con 0x7f85fc03e550 2024-09-17T16:46:38.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.058+0000 7f8612ffd640 1 -- 172.21.15.105:0/3651776541 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+114046 (secure 0 0 0) 0x7f861c1394a0 con 0x7f85fc03e550 2024-09-17T16:46:38.066 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:46:38.067 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 10 librados: watch_flush enter 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 10 librados: watch_flush exit 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 20 client.55196.objecter shutdown clearing up homeless session... 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 10 client.55196.objecter successfully canceled tick 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 msgr2=0x7f85fc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 0x7f85fc040a10 secure :-1 s=READY pgs=3678 cs=0 l=1 rev1=1 crypto rx=0x7f861c0a5ee0 tx=0x7f8604008040 comp rx=0 tx=0).stop 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 10 monclient: shutdown 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 msgr2=0x7f861c1381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f860c00d800 tx=0x7f860c00dcc0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 shutdown_connections 2024-09-17T16:46:38.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f85fc03e550 0x7f85fc040a10 unknown :-1 s=CLOSED pgs=3678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f861c0a6a30 0x7f861c138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f861c1397f0 0x7f861c13d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 --2- 172.21.15.105:0/3651776541 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f861c0a4ab0 0x7f861c1381f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 >> 172.21.15.105:0/3651776541 conn(0x7f861c0a8d00 msgr2=0x7f861c0a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 shutdown_connections 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 -- 172.21.15.105:0/3651776541 wait complete. 2024-09-17T16:46:38.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.062+0000 7f8620a4a640 1 librados: shutdown 2024-09-17T16:46:38.084 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":1264,"stamp":"2024-09-17T16:46:36.193103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":61,"num_write":110,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470648,"kb_used_data":62808,"kb_used_omap":9,"kb_used_meta":407734,"kb_avail":565760392,"statfs":{"total":579820584960,"available":579338641408,"internally_reserved":0,"allocated":64315392,"data_stored":36273584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":417520288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004247"},"pg_stats":[{"pgid":"6.16","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356126+0000","last_change":"2024-09-17T16:29:02.018311+0000","last_active":"2024-09-17T16:46:00.356126+0000","last_peered":"2024-09-17T16:46:00.356126+0000","last_clean":"2024-09-17T16:46:00.356126+0000","last_became_active":"2024-09-17T16:29:02.017756+0000","last_became_peered":"2024-09-17T16:29:02.017756+0000","last_unstale":"2024-09-17T16:46:00.356126+0000","last_undegraded":"2024-09-17T16:46:00.356126+0000","last_fullsized":"2024-09-17T16:46:00.356126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:14.467205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360583+0000","last_change":"2024-09-17T16:29:02.054931+0000","last_active":"2024-09-17T16:46:00.360583+0000","last_peered":"2024-09-17T16:46:00.360583+0000","last_clean":"2024-09-17T16:46:00.360583+0000","last_became_active":"2024-09-17T16:29:02.054559+0000","last_became_peered":"2024-09-17T16:29:02.054559+0000","last_unstale":"2024-09-17T16:46:00.360583+0000","last_undegraded":"2024-09-17T16:46:00.360583+0000","last_fullsized":"2024-09-17T16:46:00.360583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:37.538511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239713+0000","last_change":"2024-09-17T16:29:02.079133+0000","last_active":"2024-09-17T16:46:00.239713+0000","last_peered":"2024-09-17T16:46:00.239713+0000","last_clean":"2024-09-17T16:46:00.239713+0000","last_became_active":"2024-09-17T16:29:02.076410+0000","last_became_peered":"2024-09-17T16:29:02.076410+0000","last_unstale":"2024-09-17T16:46:00.239713+0000","last_undegraded":"2024-09-17T16:46:00.239713+0000","last_fullsized":"2024-09-17T16:46:00.239713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:04.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243332+0000","last_change":"2024-09-17T16:29:02.123528+0000","last_active":"2024-09-17T16:46:00.243332+0000","last_peered":"2024-09-17T16:46:00.243332+0000","last_clean":"2024-09-17T16:46:00.243332+0000","last_became_active":"2024-09-17T16:29:02.122292+0000","last_became_peered":"2024-09-17T16:29:02.122292+0000","last_unstale":"2024-09-17T16:46:00.243332+0000","last_undegraded":"2024-09-17T16:46:00.243332+0000","last_fullsized":"2024-09-17T16:46:00.243332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:48.555269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360740+0000","last_change":"2024-09-17T16:29:02.064488+0000","last_active":"2024-09-17T16:46:00.360740+0000","last_peered":"2024-09-17T16:46:00.360740+0000","last_clean":"2024-09-17T16:46:00.360740+0000","last_became_active":"2024-09-17T16:29:02.064232+0000","last_became_peered":"2024-09-17T16:29:02.064232+0000","last_unstale":"2024-09-17T16:46:00.360740+0000","last_undegraded":"2024-09-17T16:46:00.360740+0000","last_fullsized":"2024-09-17T16:46:00.360740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:14.884902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354984+0000","last_change":"2024-09-17T16:29:02.121725+0000","last_active":"2024-09-17T16:46:00.354984+0000","last_peered":"2024-09-17T16:46:00.354984+0000","last_clean":"2024-09-17T16:46:00.354984+0000","last_became_active":"2024-09-17T16:29:02.116801+0000","last_became_peered":"2024-09-17T16:29:02.116801+0000","last_unstale":"2024-09-17T16:46:00.354984+0000","last_undegraded":"2024-09-17T16:46:00.354984+0000","last_fullsized":"2024-09-17T16:46:00.354984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:23.905864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242889+0000","last_change":"2024-09-17T16:29:02.131256+0000","last_active":"2024-09-17T16:46:00.242889+0000","last_peered":"2024-09-17T16:46:00.242889+0000","last_clean":"2024-09-17T16:46:00.242889+0000","last_became_active":"2024-09-17T16:29:02.129701+0000","last_became_peered":"2024-09-17T16:29:02.129701+0000","last_unstale":"2024-09-17T16:46:00.242889+0000","last_undegraded":"2024-09-17T16:46:00.242889+0000","last_fullsized":"2024-09-17T16:46:00.242889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:34:10.375884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356246+0000","last_change":"2024-09-17T16:29:02.079372+0000","last_active":"2024-09-17T16:46:00.356246+0000","last_peered":"2024-09-17T16:46:00.356246+0000","last_clean":"2024-09-17T16:46:00.356246+0000","last_became_active":"2024-09-17T16:29:02.078833+0000","last_became_peered":"2024-09-17T16:29:02.078833+0000","last_unstale":"2024-09-17T16:46:00.356246+0000","last_undegraded":"2024-09-17T16:46:00.356246+0000","last_fullsized":"2024-09-17T16:46:00.356246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:50.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360568+0000","last_change":"2024-09-17T16:29:02.185295+0000","last_active":"2024-09-17T16:46:00.360568+0000","last_peered":"2024-09-17T16:46:00.360568+0000","last_clean":"2024-09-17T16:46:00.360568+0000","last_became_active":"2024-09-17T16:29:02.179921+0000","last_became_peered":"2024-09-17T16:29:02.179921+0000","last_unstale":"2024-09-17T16:46:00.360568+0000","last_undegraded":"2024-09-17T16:46:00.360568+0000","last_fullsized":"2024-09-17T16:46:00.360568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:18.876647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.240108+0000","last_change":"2024-09-17T16:29:02.165149+0000","last_active":"2024-09-17T16:46:00.240108+0000","last_peered":"2024-09-17T16:46:00.240108+0000","last_clean":"2024-09-17T16:46:00.240108+0000","last_became_active":"2024-09-17T16:29:02.164714+0000","last_became_peered":"2024-09-17T16:29:02.164714+0000","last_unstale":"2024-09-17T16:46:00.240108+0000","last_undegraded":"2024-09-17T16:46:00.240108+0000","last_fullsized":"2024-09-17T16:46:00.240108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:19.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355098+0000","last_change":"2024-09-17T16:29:02.177944+0000","last_active":"2024-09-17T16:46:00.355098+0000","last_peered":"2024-09-17T16:46:00.355098+0000","last_clean":"2024-09-17T16:46:00.355098+0000","last_became_active":"2024-09-17T16:29:02.177674+0000","last_became_peered":"2024-09-17T16:29:02.177674+0000","last_unstale":"2024-09-17T16:46:00.355098+0000","last_undegraded":"2024-09-17T16:46:00.355098+0000","last_fullsized":"2024-09-17T16:46:00.355098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:42.365472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360691+0000","last_change":"2024-09-17T16:29:02.185360+0000","last_active":"2024-09-17T16:46:00.360691+0000","last_peered":"2024-09-17T16:46:00.360691+0000","last_clean":"2024-09-17T16:46:00.360691+0000","last_became_active":"2024-09-17T16:29:02.184301+0000","last_became_peered":"2024-09-17T16:29:02.184301+0000","last_unstale":"2024-09-17T16:46:00.360691+0000","last_undegraded":"2024-09-17T16:46:00.360691+0000","last_fullsized":"2024-09-17T16:46:00.360691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:12.678748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361203+0000","last_change":"2024-09-17T16:29:02.050147+0000","last_active":"2024-09-17T16:46:00.361203+0000","last_peered":"2024-09-17T16:46:00.361203+0000","last_clean":"2024-09-17T16:46:00.361203+0000","last_became_active":"2024-09-17T16:29:02.044488+0000","last_became_peered":"2024-09-17T16:29:02.044488+0000","last_unstale":"2024-09-17T16:46:00.361203+0000","last_undegraded":"2024-09-17T16:46:00.361203+0000","last_fullsized":"2024-09-17T16:46:00.361203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:16.232335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":1490,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243057+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:46:00.243057+0000","last_peered":"2024-09-17T16:46:00.243057+0000","last_clean":"2024-09-17T16:46:00.243057+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:46:00.243057+0000","last_undegraded":"2024-09-17T16:46:00.243057+0000","last_fullsized":"2024-09-17T16:46:00.243057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360719+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:46:00.360719+0000","last_peered":"2024-09-17T16:46:00.360719+0000","last_clean":"2024-09-17T16:46:00.360719+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:46:00.360719+0000","last_undegraded":"2024-09-17T16:46:00.360719+0000","last_fullsized":"2024-09-17T16:46:00.360719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355189+0000","last_change":"2024-09-17T16:29:02.189168+0000","last_active":"2024-09-17T16:46:00.355189+0000","last_peered":"2024-09-17T16:46:00.355189+0000","last_clean":"2024-09-17T16:46:00.355189+0000","last_became_active":"2024-09-17T16:29:02.186575+0000","last_became_peered":"2024-09-17T16:29:02.186575+0000","last_unstale":"2024-09-17T16:46:00.355189+0000","last_undegraded":"2024-09-17T16:46:00.355189+0000","last_fullsized":"2024-09-17T16:46:00.355189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:10.592381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1399,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242887+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:46:00.242887+0000","last_peered":"2024-09-17T16:46:00.242887+0000","last_clean":"2024-09-17T16:46:00.242887+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:46:00.242887+0000","last_undegraded":"2024-09-17T16:46:00.242887+0000","last_fullsized":"2024-09-17T16:46:00.242887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361012+0000","last_change":"2024-09-17T16:29:02.070104+0000","last_active":"2024-09-17T16:46:00.361012+0000","last_peered":"2024-09-17T16:46:00.361012+0000","last_clean":"2024-09-17T16:46:00.361012+0000","last_became_active":"2024-09-17T16:29:02.067560+0000","last_became_peered":"2024-09-17T16:29:02.067560+0000","last_unstale":"2024-09-17T16:46:00.361012+0000","last_undegraded":"2024-09-17T16:46:00.361012+0000","last_fullsized":"2024-09-17T16:46:00.361012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:51.122320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1397,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239633+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:46:00.239633+0000","last_peered":"2024-09-17T16:46:00.239633+0000","last_clean":"2024-09-17T16:46:00.239633+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:46:00.239633+0000","last_undegraded":"2024-09-17T16:46:00.239633+0000","last_fullsized":"2024-09-17T16:46:00.239633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355522+0000","last_change":"2024-09-17T16:29:02.175917+0000","last_active":"2024-09-17T16:46:00.355522+0000","last_peered":"2024-09-17T16:46:00.355522+0000","last_clean":"2024-09-17T16:46:00.355522+0000","last_became_active":"2024-09-17T16:29:02.175702+0000","last_became_peered":"2024-09-17T16:29:02.175702+0000","last_unstale":"2024-09-17T16:46:00.355522+0000","last_undegraded":"2024-09-17T16:46:00.355522+0000","last_fullsized":"2024-09-17T16:46:00.355522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:10.503343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1388,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355872+0000","last_change":"2024-09-17T16:29:02.160011+0000","last_active":"2024-09-17T16:46:00.355872+0000","last_peered":"2024-09-17T16:46:00.355872+0000","last_clean":"2024-09-17T16:46:00.355872+0000","last_became_active":"2024-09-17T16:29:02.158196+0000","last_became_peered":"2024-09-17T16:29:02.158196+0000","last_unstale":"2024-09-17T16:46:00.355872+0000","last_undegraded":"2024-09-17T16:46:00.355872+0000","last_fullsized":"2024-09-17T16:46:00.355872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:11.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355364+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:46:00.355364+0000","last_peered":"2024-09-17T16:46:00.355364+0000","last_clean":"2024-09-17T16:46:00.355364+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:46:00.355364+0000","last_undegraded":"2024-09-17T16:46:00.355364+0000","last_fullsized":"2024-09-17T16:46:00.355364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"22'7","reported_seq":1385,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354809+0000","last_change":"2024-09-17T16:29:02.118963+0000","last_active":"2024-09-17T16:46:00.354809+0000","last_peered":"2024-09-17T16:46:00.354809+0000","last_clean":"2024-09-17T16:46:00.354809+0000","last_became_active":"2024-09-17T16:29:02.116933+0000","last_became_peered":"2024-09-17T16:29:02.116933+0000","last_unstale":"2024-09-17T16:46:00.354809+0000","last_undegraded":"2024-09-17T16:46:00.354809+0000","last_fullsized":"2024-09-17T16:46:00.354809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:02.801705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243451+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:46:00.243451+0000","last_peered":"2024-09-17T16:46:00.243451+0000","last_clean":"2024-09-17T16:46:00.243451+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:46:00.243451+0000","last_undegraded":"2024-09-17T16:46:00.243451+0000","last_fullsized":"2024-09-17T16:46:00.243451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243022+0000","last_change":"2024-09-17T16:29:02.137515+0000","last_active":"2024-09-17T16:46:00.243022+0000","last_peered":"2024-09-17T16:46:00.243022+0000","last_clean":"2024-09-17T16:46:00.243022+0000","last_became_active":"2024-09-17T16:29:02.136808+0000","last_became_peered":"2024-09-17T16:29:02.136808+0000","last_unstale":"2024-09-17T16:46:00.243022+0000","last_undegraded":"2024-09-17T16:46:00.243022+0000","last_fullsized":"2024-09-17T16:46:00.243022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:14.068157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355638+0000","last_change":"2024-09-17T16:28:03.463418+0000","last_active":"2024-09-17T16:46:00.355638+0000","last_peered":"2024-09-17T16:46:00.355638+0000","last_clean":"2024-09-17T16:46:00.355638+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:46:00.355638+0000","last_undegraded":"2024-09-17T16:46:00.355638+0000","last_fullsized":"2024-09-17T16:46:00.355638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:00.366319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1369,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360439+0000","last_change":"2024-09-17T16:29:02.049672+0000","last_active":"2024-09-17T16:46:00.360439+0000","last_peered":"2024-09-17T16:46:00.360439+0000","last_clean":"2024-09-17T16:46:00.360439+0000","last_became_active":"2024-09-17T16:29:02.048114+0000","last_became_peered":"2024-09-17T16:29:02.048114+0000","last_unstale":"2024-09-17T16:46:00.360439+0000","last_undegraded":"2024-09-17T16:46:00.360439+0000","last_fullsized":"2024-09-17T16:46:00.360439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:48.297807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243579+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:46:00.243579+0000","last_peered":"2024-09-17T16:46:00.243579+0000","last_clean":"2024-09-17T16:46:00.243579+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:46:00.243579+0000","last_undegraded":"2024-09-17T16:46:00.243579+0000","last_fullsized":"2024-09-17T16:46:00.243579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239579+0000","last_change":"2024-09-17T16:29:02.135845+0000","last_active":"2024-09-17T16:46:00.239579+0000","last_peered":"2024-09-17T16:46:00.239579+0000","last_clean":"2024-09-17T16:46:00.239579+0000","last_became_active":"2024-09-17T16:29:02.132790+0000","last_became_peered":"2024-09-17T16:29:02.132790+0000","last_unstale":"2024-09-17T16:46:00.239579+0000","last_undegraded":"2024-09-17T16:46:00.239579+0000","last_fullsized":"2024-09-17T16:46:00.239579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:19.264101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355839+0000","last_change":"2024-09-17T16:29:02.096285+0000","last_active":"2024-09-17T16:46:00.355839+0000","last_peered":"2024-09-17T16:46:00.355839+0000","last_clean":"2024-09-17T16:46:00.355839+0000","last_became_active":"2024-09-17T16:29:02.096072+0000","last_became_peered":"2024-09-17T16:29:02.096072+0000","last_unstale":"2024-09-17T16:46:00.355839+0000","last_undegraded":"2024-09-17T16:46:00.355839+0000","last_fullsized":"2024-09-17T16:46:00.355839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:56.215507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239869+0000","last_change":"2024-09-17T16:29:02.165494+0000","last_active":"2024-09-17T16:46:00.239869+0000","last_peered":"2024-09-17T16:46:00.239869+0000","last_clean":"2024-09-17T16:46:00.239869+0000","last_became_active":"2024-09-17T16:29:02.165109+0000","last_became_peered":"2024-09-17T16:29:02.165109+0000","last_unstale":"2024-09-17T16:46:00.239869+0000","last_undegraded":"2024-09-17T16:46:00.239869+0000","last_fullsized":"2024-09-17T16:46:00.239869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:09:38.687988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360820+0000","last_change":"2024-09-17T16:29:02.052616+0000","last_active":"2024-09-17T16:46:00.360820+0000","last_peered":"2024-09-17T16:46:00.360820+0000","last_clean":"2024-09-17T16:46:00.360820+0000","last_became_active":"2024-09-17T16:29:02.050385+0000","last_became_peered":"2024-09-17T16:29:02.050385+0000","last_unstale":"2024-09-17T16:46:00.360820+0000","last_undegraded":"2024-09-17T16:46:00.360820+0000","last_fullsized":"2024-09-17T16:46:00.360820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:14.785603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355442+0000","last_change":"2024-09-17T16:29:02.192467+0000","last_active":"2024-09-17T16:46:00.355442+0000","last_peered":"2024-09-17T16:46:00.355442+0000","last_clean":"2024-09-17T16:46:00.355442+0000","last_became_active":"2024-09-17T16:29:02.192181+0000","last_became_peered":"2024-09-17T16:29:02.192181+0000","last_unstale":"2024-09-17T16:46:00.355442+0000","last_undegraded":"2024-09-17T16:46:00.355442+0000","last_fullsized":"2024-09-17T16:46:00.355442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:14.890573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243241+0000","last_change":"2024-09-17T16:29:02.123632+0000","last_active":"2024-09-17T16:46:00.243241+0000","last_peered":"2024-09-17T16:46:00.243241+0000","last_clean":"2024-09-17T16:46:00.243241+0000","last_became_active":"2024-09-17T16:29:02.119724+0000","last_became_peered":"2024-09-17T16:29:02.119724+0000","last_unstale":"2024-09-17T16:46:00.243241+0000","last_undegraded":"2024-09-17T16:46:00.243241+0000","last_fullsized":"2024-09-17T16:46:00.243241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:59.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239848+0000","last_change":"2024-09-17T16:29:02.075907+0000","last_active":"2024-09-17T16:46:00.239848+0000","last_peered":"2024-09-17T16:46:00.239848+0000","last_clean":"2024-09-17T16:46:00.239848+0000","last_became_active":"2024-09-17T16:29:02.075599+0000","last_became_peered":"2024-09-17T16:29:02.075599+0000","last_unstale":"2024-09-17T16:46:00.239848+0000","last_undegraded":"2024-09-17T16:46:00.239848+0000","last_fullsized":"2024-09-17T16:46:00.239848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:34:21.469780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243362+0000","last_change":"2024-09-17T16:29:02.074165+0000","last_active":"2024-09-17T16:46:00.243362+0000","last_peered":"2024-09-17T16:46:00.243362+0000","last_clean":"2024-09-17T16:46:00.243362+0000","last_became_active":"2024-09-17T16:29:02.073908+0000","last_became_peered":"2024-09-17T16:29:02.073908+0000","last_unstale":"2024-09-17T16:46:00.243362+0000","last_undegraded":"2024-09-17T16:46:00.243362+0000","last_fullsized":"2024-09-17T16:46:00.243362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:09:53.379585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:29:02.151691+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:29:02.151449+0000","last_became_peered":"2024-09-17T16:29:02.151449+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:00.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356257+0000","last_change":"2024-09-17T16:29:02.159400+0000","last_active":"2024-09-17T16:46:00.356257+0000","last_peered":"2024-09-17T16:46:00.356257+0000","last_clean":"2024-09-17T16:46:00.356257+0000","last_became_active":"2024-09-17T16:29:02.157013+0000","last_became_peered":"2024-09-17T16:29:02.157013+0000","last_unstale":"2024-09-17T16:46:00.356257+0000","last_undegraded":"2024-09-17T16:46:00.356257+0000","last_fullsized":"2024-09-17T16:46:00.356257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.554975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356342+0000","last_change":"2024-09-17T16:29:02.146313+0000","last_active":"2024-09-17T16:46:00.356342+0000","last_peered":"2024-09-17T16:46:00.356342+0000","last_clean":"2024-09-17T16:46:00.356342+0000","last_became_active":"2024-09-17T16:29:02.144760+0000","last_became_peered":"2024-09-17T16:29:02.144760+0000","last_unstale":"2024-09-17T16:46:00.356342+0000","last_undegraded":"2024-09-17T16:46:00.356342+0000","last_fullsized":"2024-09-17T16:46:00.356342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:19.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360880+0000","last_change":"2024-09-17T16:29:02.050143+0000","last_active":"2024-09-17T16:46:00.360880+0000","last_peered":"2024-09-17T16:46:00.360880+0000","last_clean":"2024-09-17T16:46:00.360880+0000","last_became_active":"2024-09-17T16:29:02.049439+0000","last_became_peered":"2024-09-17T16:29:02.049439+0000","last_unstale":"2024-09-17T16:46:00.360880+0000","last_undegraded":"2024-09-17T16:46:00.360880+0000","last_fullsized":"2024-09-17T16:46:00.360880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:44.935606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607779,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607780,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10856,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11116544,"data_stored":6439173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607780,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78132,"kb_used_data":10852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293708,"statfs":{"total":96636764160,"available":96556756992,"internally_reserved":0,"allocated":11112448,"data_stored":6439168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81716,"kb_used_data":10276,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290124,"statfs":{"total":96636764160,"available":96553086976,"internally_reserved":0,"allocated":10522624,"data_stored":5848819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640483,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77560,"kb_used_data":10280,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294280,"statfs":{"total":96636764160,"available":96557342720,"internally_reserved":0,"allocated":10526720,"data_stored":5848824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:46:38.086 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: get_monmap_and_config 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: build_initial_monmap 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: monmap: 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:38.198431+0000 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:38.198431+0000 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:38.202 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:38.203 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 1 Processor -- start 2024-09-17T16:46:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 1 -- start start 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: init 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.194+0000 7f4962701640 10 monclient: _add_conns ranks=[10,2,6,7,0,9,1,5,4,3,8] 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f495c0a68e0 0x7f495c0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): picked mon.noname-f con 0x7f495c0a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): picked mon.noname-b con 0x7f495c0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f495c1396a0 0x7f495c139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): picked mon.noname-d con 0x7f495c1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f495c0a3fc0 con 0x7f495c0a68e0 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f495c0a63c0 con 0x7f495c0a4910 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f495c0a7740 con 0x7f495c1396a0 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient(hunting): authenticate will time out at 2081.464106s 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38608/0 (socket says 172.21.15.105:38608) 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 -- 172.21.15.105:0/3031963315 learned_addr learned my addr 172.21.15.105:0/3031963315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f495c1396a0 0x7f495c139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): get_auth_request con 0x7f495c0a4910 auth_method 0 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:38.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): get_auth_request con 0x7f495c1396a0 auth_method 0 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient(hunting): handle_auth_done global_id 84683 payload 274 2024-09-17T16:46:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 -- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f495c1396a0 msgr2=0x7f495c139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 --2- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f495c1396a0 0x7f495c139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 -- 172.21.15.105:0/3031963315 >> v2:172.21.15.160:3300/0 conn(0x7f495c0a68e0 msgr2=0x7f495c0a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 --2- 172.21.15.105:0/3031963315 >> v2:172.21.15.160:3300/0 conn(0x7f495c0a68e0 0x7f495c0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 monclient: found mon.noname-b 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 -- 172.21.15.105:0/3031963315 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c06ebc0 con 0x7f495c0a4910 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495b7fe640 1 --2- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f494401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bd23a7fbef0561 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 1 -- 172.21.15.105:0/3031963315 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49440b2070 con 0x7f495c0a4910 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: dump: 2024-09-17T16:46:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: _finish_auth 0 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 1 -- 172.21.15.105:0/3031963315 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49440ab7b0 con 0x7f495c0a4910 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 5 monclient: authenticate success, global_id 84683 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 1 -- 172.21.15.105:0/3031963315 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49440abe50 con 0x7f495c0a4910 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f4962701640 4 set_mon_vals no callback set 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.198+0000 7f495affd640 10 monclient: dump: 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:38.214 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:38.214 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:38.214 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:38.214 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:38.214 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: shutdown 2024-09-17T16:46:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 msgr2=0x7f495c0a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4944009870 tx=0x7f494401c3f0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/3031963315 shutdown_connections 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f495c1396a0 0x7f495c139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- 172.21.15.105:0/3031963315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f495c0a4910 0x7f495c0a7200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- 172.21.15.105:0/3031963315 >> v2:172.21.15.160:3300/0 conn(0x7f495c0a68e0 0x7f495c0a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/3031963315 >> 172.21.15.105:0/3031963315 conn(0x7f495c068ba0 msgr2=0x7f495c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/3031963315 shutdown_connections 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/3031963315 wait complete. 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: build_initial_monmap 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: monmap: 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:38.220 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 librados: starting msgr at 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 librados: starting objecter 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 Processor -- start 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- start start 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 librados: setting wanted keys 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 librados: calling monclient init 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: init 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _add_conns ranks=[1,4,0,8,2,3,6,7,5] 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 0x7f495c137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): picked mon.noname-f con 0x7f495c0a4910 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): picked mon.noname-c con 0x7f495c0a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495c1396a0 0x7f495c138600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): picked mon.noname-a con 0x7f495c1396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f495c06e630 con 0x7f495c1396a0 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0a7af0 con 0x7f495c0a4910 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f495c143320 con 0x7f495c0a68e0 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient(hunting): authenticate will time out at 2081.469679s 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495c1396a0 0x7f495c138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:42808/0 (socket says 172.21.15.105:42808) 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 -- 172.21.15.105:0/531029874 learned_addr learned my addr 172.21.15.105:0/531029874 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): get_auth_request con 0x7f495c0a68e0 auth_method 0 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): get_auth_request con 0x7f495c1396a0 auth_method 0 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4960c9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient(hunting): handle_auth_done global_id 74974 payload 824 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 -- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 msgr2=0x7f495c137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495bfff640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 0x7f495c137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 0x7f495c137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 -- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495c1396a0 msgr2=0x7f495c138600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495c1396a0 0x7f495c138600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 monclient: found mon.noname-c 2024-09-17T16:46:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 -- 172.21.15.105:0/531029874 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c13fbb0 con 0x7f495c0a68e0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495bfff640 10 monclient: get_auth_request con 0x7f495c0a4910 auth_method 0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495bfff640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 0x7f495c137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495b7fe640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f494401cc90 tx=0x7f4944019190 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 client.?.objecter ms_handle_connect 0x7f495c0a68e0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49440b2070 con 0x7f495c0a68e0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: dump: 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:38.226 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: _finish_auth 0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49440abdd0 con 0x7f495c0a68e0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 5 monclient: authenticate success, global_id 74974 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _renew_subs 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c123820 con 0x7f495c0a68e0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4944003a90 con 0x7f495c0a68e0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 monclient: dump: 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f495a7fc640 4 set_mon_vals no callback set 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 client.74974.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _renew_subs 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c0a9fc0 con 0x7f495c0a68e0 2024-09-17T16:46:38.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 librados: init done 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f49440bf060 con 0x7f495c0a68e0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c13f0d0 con 0x7f495c0a68e0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 0x7f493c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f49440e9700 con 0x7f495c0a68e0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 10 client.74974.objecter ms_dispatch 0x7f495c000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 3 client.74974.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 3 client.74974.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.202+0000 7f4958ff9640 20 client.74974.objecter dump_active .. 0 homeless 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f495bfff640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 0x7f493c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f495bfff640 10 monclient: get_auth_request con 0x7f493c03e580 auth_method 0 2024-09-17T16:46:38.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f4958ff9640 10 client.74974.objecter ms_handle_connect 0x7f493c03e580 2024-09-17T16:46:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f495bfff640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 0x7f493c040a40 secure :-1 s=READY pgs=3679 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a8b0 tx=0x7f494c008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f495c13fbb0 con 0x7f495c0a68e0 2024-09-17T16:46:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f4958ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.206+0000 7f4958ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:46:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:46:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f495c077b60 con 0x7f495c0a68e0 2024-09-17T16:46:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4958ff9640 1 -- 172.21.15.105:0/531029874 <== mon.4 v2:172.21.15.105:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v707) ==== 74+0+15504 (secure 0 0 0) 0x7f4944016130 con 0x7f495c0a68e0 2024-09-17T16:46:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4958ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-17T16:46:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4958ff9640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:46:38.413 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:46:38.413 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":707,"fsid":"4403e27b-1d4f-4d49-a333-3b7d5cfcb76b","created":"2024-09-17T16:27:42.683912+0000","modified":"2024-09-17T16:46:00.229417+0000","last_up_change":"2024-09-17T16:27:52.867595+0000","last_in_change":"2024-09-17T16:27:44.458290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":105,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":340,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T16:27:53.825252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T16:27:59.687166+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-09-17T16:29:00.736098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da6872ee-3d6d-43b3-9bf3-57913747db7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6817","nonce":315040748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6819","nonce":315040748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6823","nonce":315040748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":315040748},{"type":"v1","addr":"172.21.15.105:6821","nonce":315040748}]},"public_addr":"172.21.15.105:6817/315040748","cluster_addr":"172.21.15.105:6819/315040748","heartbeat_back_addr":"172.21.15.105:6823/315040748","heartbeat_front_addr":"172.21.15.105:6821/315040748","state":["exists","up"]},{"osd":1,"uuid":"ed40322f-b150-4b91-94f8-556fad6e845b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6801","nonce":3243801162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6803","nonce":3243801162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6807","nonce":3243801162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3243801162},{"type":"v1","addr":"172.21.15.105:6805","nonce":3243801162}]},"public_addr":"172.21.15.105:6801/3243801162","cluster_addr":"172.21.15.105:6803/3243801162","heartbeat_back_addr":"172.21.15.105:6807/3243801162","heartbeat_front_addr":"172.21.15.105:6805/3243801162","state":["exists","up"]},{"osd":2,"uuid":"a4157a35-9183-4b1a-9afc-4e08def6d0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6809","nonce":1006010180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6811","nonce":1006010180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6815","nonce":1006010180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1006010180},{"type":"v1","addr":"172.21.15.105:6813","nonce":1006010180}]},"public_addr":"172.21.15.105:6809/1006010180","cluster_addr":"172.21.15.105:6811/1006010180","heartbeat_back_addr":"172.21.15.105:6815/1006010180","heartbeat_front_addr":"172.21.15.105:6813/1006010180","state":["exists","up"]},{"osd":3,"uuid":"994d2203-e554-437e-b09e-3b2861c4415e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6809","nonce":3218035673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6811","nonce":3218035673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6815","nonce":3218035673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":3218035673},{"type":"v1","addr":"172.21.15.160:6813","nonce":3218035673}]},"public_addr":"172.21.15.160:6809/3218035673","cluster_addr":"172.21.15.160:6811/3218035673","heartbeat_back_addr":"172.21.15.160:6815/3218035673","heartbeat_front_addr":"172.21.15.160:6813/3218035673","state":["exists","up"]},{"osd":4,"uuid":"ce840dc5-8cda-4070-9106-591c85c3f28b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6817","nonce":639753971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6819","nonce":639753971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6823","nonce":639753971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":639753971},{"type":"v1","addr":"172.21.15.160:6821","nonce":639753971}]},"public_addr":"172.21.15.160:6817/639753971","cluster_addr":"172.21.15.160:6819/639753971","heartbeat_back_addr":"172.21.15.160:6823/639753971","heartbeat_front_addr":"172.21.15.160:6821/639753971","state":["exists","up"]},{"osd":5,"uuid":"4a264646-336c-4e1e-97e0-bbc5e144e545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":702,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6801","nonce":2641095577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6803","nonce":2641095577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6807","nonce":2641095577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2641095577},{"type":"v1","addr":"172.21.15.160:6805","nonce":2641095577}]},"public_addr":"172.21.15.160:6801/2641095577","cluster_addr":"172.21.15.160:6803/2641095577","heartbeat_back_addr":"172.21.15.160:6807/2641095577","heartbeat_front_addr":"172.21.15.160:6805/2641095577","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.073377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.536863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.709983+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.944027+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:51.614775+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T16:27:50.703905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":13,"snaps":[{"begin":3,"length":4}]},{"pool":17,"snaps":[{"begin":1,"length":4}]},{"pool":35,"snaps":[{"begin":1,"length":1}]},{"pool":66,"snaps":[{"begin":1,"length":1}]},{"pool":93,"snaps":[{"begin":4,"length":2},{"begin":7,"length":2}]},{"pool":104,"snaps":[{"begin":1,"length":1}]},{"pool":125,"snaps":[{"begin":1,"length":2}]},{"pool":140,"snaps":[{"begin":3,"length":1}]},{"pool":144,"snaps":[{"begin":1,"length":4}]},{"pool":148,"snaps":[{"begin":2,"length":1}]},{"pool":154,"snaps":[{"begin":2,"length":2}]},{"pool":163,"snaps":[{"begin":2,"length":3}]},{"pool":168,"snaps":[{"begin":3,"length":4}]},{"pool":171,"snaps":[{"begin":2,"length":4}]},{"pool":173,"snaps":[{"begin":2,"length":2}]},{"pool":182,"snaps":[{"begin":2,"length":1}]},{"pool":188,"snaps":[{"begin":3,"length":2}]},{"pool":197,"snaps":[{"begin":4,"length":3},{"begin":8,"length":1}]},{"pool":199,"snaps":[{"begin":7,"length":1}]},{"pool":224,"snaps":[{"begin":8,"length":1}]},{"pool":235,"snaps":[{"begin":10,"length":1}]},{"pool":241,"snaps":[{"begin":11,"length":2}]},{"pool":256,"snaps":[{"begin":14,"length":1}]},{"pool":321,"snaps":[{"begin":2,"length":1}]},{"pool":322,"snaps":[{"begin":3,"length":1}]},{"pool":325,"snaps":[{"begin":4,"length":1}]},{"pool":329,"snaps":[{"begin":6,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T16:46:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 librados: watch_flush enter 2024-09-17T16:46:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 librados: watch_flush exit 2024-09-17T16:46:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 20 client.74974.objecter shutdown clearing up homeless session... 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 client.74974.objecter successfully canceled tick 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 msgr2=0x7f493c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 0x7f493c040a40 secure :-1 s=READY pgs=3679 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a8b0 tx=0x7f494c008040 comp rx=0 tx=0).stop 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 10 monclient: shutdown 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 msgr2=0x7f495c1380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.406+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f494401cc90 tx=0x7f4944019190 comp rx=0 tx=0).stop 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 shutdown_connections 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f493c03e580 0x7f493c040a40 unknown :-1 s=CLOSED pgs=3679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f495c1396a0 0x7f495c138600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f495c0a68e0 0x7f495c1380c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 --2- 172.21.15.105:0/531029874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f495c0a4910 0x7f495c137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 >> 172.21.15.105:0/531029874 conn(0x7f495c068ba0 msgr2=0x7f495c0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 shutdown_connections 2024-09-17T16:46:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 -- 172.21.15.105:0/531029874 wait complete. 2024-09-17T16:46:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:38.410+0000 7f4962701640 1 librados: shutdown 2024-09-17T16:46:39.434 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T16:46:39.434 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: get_monmap_and_config 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: build_initial_monmap 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: monmap: 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:39.564541+0000 2024-09-17T16:46:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:39.564541+0000 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:39.569 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 Processor -- start 2024-09-17T16:46:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 -- start start 2024-09-17T16:46:39.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: init 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient: _add_conns ranks=[8,2,3,10,7,5,0,4,1,6,9] 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 0x7f94f40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-e con 0x7f94f40a35a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-b con 0x7f94f40a0d60 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 0x7f94f41377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-g con 0x7f94f4137370 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f94f407d3b0 con 0x7f94f40a0d60 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f94f409e480 con 0x7f94f4137370 2024-09-17T16:46:39.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f94f40a4400 con 0x7f94f40a35a0 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94fb885640 10 monclient(hunting): authenticate will time out at 2082.830268s 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38620/0 (socket says 172.21.15.105:38620) 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 -- 172.21.15.105:0/3052367861 learned_addr learned my addr 172.21.15.105:0/3052367861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): get_auth_request con 0x7f94f40a0d60 auth_method 0 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 0x7f94f40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): get_auth_request con 0x7f94f40a35a0 auth_method 0 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient(hunting): handle_auth_done global_id 84692 payload 274 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9e22640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 0x7f94f41377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 -- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 msgr2=0x7f94f40a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 0x7f94f40a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 -- 172.21.15.105:0/3052367861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 msgr2=0x7f94f41377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 0x7f94f41377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient: _finish_hunting 0 2024-09-17T16:46:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 monclient: found mon.noname-b 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f8e20640 1 -- 172.21.15.105:0/3052367861 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40a1fd0 con 0x7f94f40a0d60 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9621640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 0x7f94f40a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9e22640 10 monclient: get_auth_request con 0x7f94f4137370 auth_method 0 2024-09-17T16:46:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.562+0000 7f94f9e22640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 0x7f94f41377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098a0 tx=0x7f94e401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894b50d83a263ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 1 -- 172.21.15.105:0/3052367861 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94e40b0070 con 0x7f94f40a0d60 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: dump: 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.582 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: _finish_auth 0 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 1 -- 172.21.15.105:0/3052367861 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e40aa810 con 0x7f94f40a0d60 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 1 -- 172.21.15.105:0/3052367861 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94e40a9300 con 0x7f94f40a0d60 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 5 monclient: authenticate success, global_id 84692 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 4 set_mon_vals no callback set 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94ebfff640 10 monclient: dump: 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: shutdown 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 msgr2=0x7f94f40a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098a0 tx=0x7f94e401c440 comp rx=0 tx=0).stop 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- 172.21.15.105:0/3052367861 shutdown_connections 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f4137370 0x7f94f41377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94f40a0d60 0x7f94f40a3ec0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- 172.21.15.105:0/3052367861 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f94f40a35a0 0x7f94f40a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- 172.21.15.105:0/3052367861 >> 172.21.15.105:0/3052367861 conn(0x7f94f40689d0 msgr2=0x7f94f4068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- 172.21.15.105:0/3052367861 shutdown_connections 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- 172.21.15.105:0/3052367861 wait complete. 2024-09-17T16:46:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: build_initial_monmap 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: monmap: 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:39.587 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 librados: starting msgr at 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 librados: starting objecter 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 Processor -- start 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- start start 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 librados: setting wanted keys 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 librados: calling monclient init 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: init 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient: _add_conns ranks=[6,0,5,4,7,1,3,8,2] 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 0x7f94f4135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-d con 0x7f94f40a0d60 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94f40a35a0 0x7f94f4136320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-a con 0x7f94f40a35a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f94f4137370 0x7f94f409eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): picked mon.noname-h con 0x7f94f4137370 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94f409e690 con 0x7f94f40a35a0 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f94f4069c10 con 0x7f94f4137370 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f94f4141080 con 0x7f94f40a0d60 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94fb885640 10 monclient(hunting): authenticate will time out at 2082.835322s 2024-09-17T16:46:39.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 0x7f94f4135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 0x7f94f4135de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:45710/0 (socket says 172.21.15.105:45710) 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 -- 172.21.15.105:0/2246800712 learned_addr learned my addr 172.21.15.105:0/2246800712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): get_auth_request con 0x7f94f40a0d60 auth_method 0 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94f40a35a0 0x7f94f4136320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): get_auth_request con 0x7f94f40a35a0 auth_method 0 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9e22640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f94f4137370 0x7f94f409eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient(hunting): handle_auth_done global_id 94461 payload 824 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 -- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f94f4137370 msgr2=0x7f94f409eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f94f4137370 0x7f94f409eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 -- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94f40a35a0 msgr2=0x7f94f4136320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94f40a35a0 0x7f94f4136320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient: _finish_hunting 0 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 monclient: found mon.noname-d 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:39.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9621640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40a0740 con 0x7f94f40a0d60 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f8e20640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94f40a35a0 0x7f94f4136320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9e22640 10 monclient: get_auth_request con 0x7f94f4137370 auth_method 0 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.566+0000 7f94f9e22640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f94f4137370 0x7f94f409eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f9621640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 0x7f94f4135de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f60 tx=0x7f94f0009f90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.?.objecter ms_handle_connect 0x7f94f40a0d60 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94f0018020 con 0x7f94f40a0d60 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: dump: 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.592 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:39.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f00148f0 con 0x7f94f40a0d60 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 5 monclient: authenticate success, global_id 94461 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 monclient: _renew_subs 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f40a0500 con 0x7f94f40a0d60 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94f00133e0 con 0x7f94f40a0d60 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: dump: 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.594 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 client.94461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:39.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94eb7fe640 4 set_mon_vals no callback set 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 monclient: _renew_subs 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f40a0940 con 0x7f94f40a0d60 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 1 librados: init done 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 client.94461.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 15 client.94461.objecter _session_command_op_assign -1 1 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 20 client.94461.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 client.94461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94f41377f0 con 0x7f94f40a0d60 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f94f001b020 con 0x7f94f40a0d60 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f94d803e550 0x7f94d8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f94f0050520 con 0x7f94f40a0d60 2024-09-17T16:46:39.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter ms_dispatch 0x7f94f4000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 3 client.94461.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 3 client.94461.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter checking command 1 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f94d8045d10 0x7f94d80481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter _get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 15 client.94461.objecter _session_command_op_remove -1 1 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter _get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 15 client.94461.objecter _session_command_op_assign 0 1 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 4 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter _send_command 1 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94f0012e50 con 0x7f94d8045d10 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter dump_active .. 0 homeless 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7f94f0057050 con 0x7f94f40a0d60 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f9e22640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f94d8045d10 0x7f94d80481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f9e22640 10 monclient: get_auth_request con 0x7f94d8045d10 auth_method 0 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f8e20640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f94d803e550 0x7f94d8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f8e20640 10 monclient: get_auth_request con 0x7f94d803e550 auth_method 0 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter ms_handle_connect 0x7f94d8045d10 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f9e22640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f94d8045d10 0x7f94d80481b0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter ms_handle_connect 0x7f94d803e550 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94f8e20640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f94d803e550 0x7f94d8040a10 secure :-1 s=READY pgs=3680 cs=0 l=1 rev1=1 crypto rx=0x7f94e40ad000 tx=0x7f94e40a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== osd.0 v2:172.21.15.105:6816/315040748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94f0012e50 con 0x7f94d8045d10 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter ms_dispatch 0x7f94f4000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 10 client.94461.objecter _finish_command 1 = system:0 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 4 2024-09-17T16:46:39.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.570+0000 7f94e9ffb640 15 client.94461.objecter _session_command_op_remove 0 1 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 client.94461.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 15 client.94461.objecter _session_command_op_assign -1 2 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter _get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter _get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 15 client.94461.objecter _session_command_op_remove -1 2 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter get_session s=0x7f94d8045640 osd=0 3 2024-09-17T16:46:39.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 15 client.94461.objecter _session_command_op_assign 0 2 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 4 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 client.94461.objecter _send_command 2 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 --> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f94f41233f0 con 0x7f94d8045d10 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94e9ffb640 1 -- 172.21.15.105:0/2246800712 <== osd.0 v2:172.21.15.105:6816/315040748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f94f41233f0 con 0x7f94d8045d10 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94e9ffb640 10 client.94461.objecter ms_dispatch 0x7f94f4000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94e9ffb640 10 client.94461.objecter _finish_command 2 = system:0 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94e9ffb640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 4 2024-09-17T16:46:39.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94e9ffb640 15 client.94461.objecter _session_command_op_remove 0 2 2024-09-17T16:46:39.602 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:39.602 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:39.602 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 librados: watch_flush enter 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 librados: watch_flush exit 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 client.94461.objecter close_session for osd.0 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f94d8045d10 msgr2=0x7f94d80481b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:6816/315040748,v1:172.21.15.105:6817/315040748] conn(0x7f94d8045d10 0x7f94d80481b0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter put_session s=0x7f94d8045640 osd=0 1 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 client.94461.objecter shutdown clearing up homeless session... 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 client.94461.objecter successfully canceled tick 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f94d803e550 msgr2=0x7f94d8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f94d803e550 0x7f94d8040a10 secure :-1 s=READY pgs=3680 cs=0 l=1 rev1=1 crypto rx=0x7f94e40ad000 tx=0x7f94e40a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 10 monclient: shutdown 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 msgr2=0x7f94f4135de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 --2- 172.21.15.105:0/2246800712 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f94f40a0d60 0x7f94f4135de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009f60 tx=0x7f94f0009f90 comp rx=0 tx=0).stop 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94f9e22640 1 -- 172.21.15.105:0/2246800712 reap_dead start 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 shutdown_connections 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 >> 172.21.15.105:0/2246800712 conn(0x7f94f40689d0 msgr2=0x7f94f40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 shutdown_connections 2024-09-17T16:46:39.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 -- 172.21.15.105:0/2246800712 wait complete. 2024-09-17T16:46:39.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.586+0000 7f94fb885640 1 librados: shutdown 2024-09-17T16:46:39.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T16:46:39.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 10 monclient: get_monmap_and_config 2024-09-17T16:46:39.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 10 monclient: build_initial_monmap 2024-09-17T16:46:39.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 10 monclient: monmap: 2024-09-17T16:46:39.722 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:39.722 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:39.718811+0000 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:39.718811+0000 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:39.723 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:39.724 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:39.724 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 1 Processor -- start 2024-09-17T16:46:39.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 1 -- start start 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.714+0000 7f667e512640 10 monclient: init 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient: _add_conns ranks=[7,8,1,0,10,3,5,9,4,2,6] 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f66780a2620 0x7f66780a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): picked mon.noname-i con 0x7f66780a2620 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): picked mon.noname-e con 0x7f66780a2f40 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6678137940 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667806f2e0 con 0x7f6678137940 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f667806e580 con 0x7f66780a2620 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f66780a15e0 con 0x7f66780a2f40 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient(hunting): authenticate will time out at 2082.984532s 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41266/0 (socket says 172.21.15.105:41266) 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 -- 172.21.15.105:0/1477322009 learned_addr learned my addr 172.21.15.105:0/1477322009 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): get_auth_request con 0x7f66780a2f40 auth_method 0 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6677fff640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f66780a2620 0x7f66780a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient(hunting): handle_auth_done global_id 55214 payload 274 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 -- 172.21.15.105:0/1477322009 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f66780a2620 msgr2=0x7f66780a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f66780a2620 0x7f66780a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 -- 172.21.15.105:0/1477322009 >> v1:172.21.15.160:6789/0 conn(0x7f6678137940 legacy=0x7f667813e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 monclient: found mon.noname-e 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:39.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 -- 172.21.15.105:0/1477322009 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66780a3c30 con 0x7f66780a2f40 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f66777fe640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e80 tx=0x7f667001c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c1b86cbbdef6b7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 1 -- 172.21.15.105:0/1477322009 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66700090d0 con 0x7f66780a2f40 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: dump: 2024-09-17T16:46:39.734 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.735 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 5 monclient: authenticate success, global_id 55214 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 1 -- 172.21.15.105:0/1477322009 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66700a5b80 con 0x7f66780a2f40 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 4 set_mon_vals no callback set 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 1 -- 172.21.15.105:0/1477322009 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66700a5d70 con 0x7f66780a2f40 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:39.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f6676ffd640 10 monclient: dump: 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.737 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 10 monclient: shutdown 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 -- 172.21.15.105:0/1477322009 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 msgr2=0x7f6678137400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.718+0000 7f667e512640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e80 tx=0x7f667001c910 comp rx=0 tx=0).stop 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/1477322009 shutdown_connections 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f66780a2f40 0x7f6678137400 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 --2- 172.21.15.105:0/1477322009 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f66780a2620 0x7f66780a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/1477322009 >> 172.21.15.105:0/1477322009 conn(0x7f6678068ba0 msgr2=0x7f6678068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/1477322009 shutdown_connections 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/1477322009 wait complete. 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: build_initial_monmap 2024-09-17T16:46:39.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: monmap: 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:39.739 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 librados: starting msgr at 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 librados: starting objecter 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 Processor -- start 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- start start 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 librados: setting wanted keys 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 librados: calling monclient init 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: init 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _add_conns ranks=[5,0,4,3,8,7,2,6,1] 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f66780a2620 0x7f667811ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): picked mon.noname-h con 0x7f66780a2620 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:39.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): picked mon.noname-a con 0x7f66780a2f40 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 0x7f667811f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): picked mon.noname-c con 0x7f6678137940 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6678069ad0 con 0x7f66780a2f40 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f6678069950 con 0x7f6678137940 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f66780a1dd0 con 0x7f66780a2620 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:39.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient(hunting): authenticate will time out at 2082.989597s 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 0x7f667811f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37752/0 (socket says 172.21.15.105:37752) 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 1 -- 172.21.15.105:0/4231312791 learned_addr learned my addr 172.21.15.105:0/4231312791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): get_auth_request con 0x7f66780a2f40 auth_method 0 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): get_auth_request con 0x7f6678137940 auth_method 0 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6677fff640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f66780a2620 0x7f667811ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6677fff640 10 monclient(hunting): get_auth_request con 0x7f66780a2620 auth_method 0 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6677fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6677fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6677fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient(hunting): handle_auth_done global_id 74983 payload 824 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 -- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f66780a2620 msgr2=0x7f667811ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f66780a2620 0x7f667811ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 -- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 msgr2=0x7f667811efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient: _finish_hunting 0 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 monclient: found mon.noname-c 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 -- 172.21.15.105:0/4231312791 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66781222a0 con 0x7f6678137940 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66777fe640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667caaf640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 0x7f667811f510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f666000b3e0 tx=0x7f666000b8a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 client.?.objecter ms_handle_connect 0x7f6678137940 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6660015020 con 0x7f6678137940 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:39.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: dump: 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.745 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: _finish_auth 0 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: _check_auth_tickets 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6660011aa0 con 0x7f6678137940 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6660011cd0 con 0x7f6678137940 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 5 monclient: authenticate success, global_id 74983 2024-09-17T16:46:39.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _renew_subs 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6678122580 con 0x7f6678137940 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f66767fc640 4 set_mon_vals no callback set 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 10 monclient: dump: 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:39.747 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 client.74983.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _renew_subs 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6678120500 con 0x7f6678137940 2024-09-17T16:46:39.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 librados: init done 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667809e1e0 con 0x7f6678137940 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f6660018020 con 0x7f6678137940 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.722+0000 7f6674ff9640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 0x7f665c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f666004f620 con 0x7f6678137940 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 10 client.74983.objecter ms_dispatch 0x7f6678000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 3 client.74983.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 3 client.74983.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 20 client.74983.objecter dump_active .. 0 homeless 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6677fff640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 0x7f665c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6677fff640 10 monclient: get_auth_request con 0x7f665c03e550 auth_method 0 2024-09-17T16:46:39.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6674ff9640 10 client.74983.objecter ms_handle_connect 0x7f665c03e550 2024-09-17T16:46:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.726+0000 7f6677fff640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 0x7f665c040a10 secure :-1 s=READY pgs=3681 cs=0 l=1 rev1=1 crypto rx=0x7f666800a970 tx=0x7f6668008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.730+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f666001f030 con 0x7f6678137940 2024-09-17T16:46:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.730+0000 7f6674ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:39.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.730+0000 7f6674ff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:39.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.898+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f66781371f0 con 0x7f665c03e550 2024-09-17T16:46:39.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.898+0000 7f6674ff9640 1 -- 172.21.15.105:0/4231312791 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f66781371f0 con 0x7f665c03e550 2024-09-17T16:46:39.907 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 10 librados: watch_flush enter 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 10 librados: watch_flush exit 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 20 client.74983.objecter shutdown clearing up homeless session... 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 10 client.74983.objecter successfully canceled tick 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 msgr2=0x7f665c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 0x7f665c040a10 secure :-1 s=READY pgs=3681 cs=0 l=1 rev1=1 crypto rx=0x7f666800a970 tx=0x7f6668008040 comp rx=0 tx=0).stop 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 10 monclient: shutdown 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 msgr2=0x7f667811f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 0x7f667811f510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f666000b3e0 tx=0x7f666000b8a0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 shutdown_connections 2024-09-17T16:46:39.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f665c03e550 0x7f665c040a10 unknown :-1 s=CLOSED pgs=3681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f6678137940 0x7f667811f510 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66780a2f40 0x7f667811efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 --2- 172.21.15.105:0/4231312791 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f66780a2620 0x7f667811ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 >> 172.21.15.105:0/4231312791 conn(0x7f6678068ba0 msgr2=0x7f66780a6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 shutdown_connections 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 -- 172.21.15.105:0/4231312791 wait complete. 2024-09-17T16:46:39.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:39.902+0000 7f667e512640 1 librados: shutdown 2024-09-17T16:46:39.927 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T16:46:39.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: get_monmap_and_config 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: monmap: 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:40.048707+0000 2024-09-17T16:46:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:40.048707+0000 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:40.054 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 Processor -- start 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 -- start start 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: init 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient: _add_conns ranks=[7,8,6,2,1,4,0,9,5,3,10] 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c0a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-i con 0x7fe96c0a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-e con 0x7fe96c0a6cc0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 0x7fe96c139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-d con 0x7fe96c1396a0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7fe96c07d3b0 con 0x7fe96c1396a0 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe96c0a87d0 con 0x7fe96c0a68e0 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe96c0a4de0 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe9739f8640 10 monclient(hunting): authenticate will time out at 2083.314496s 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41280/0 (socket says 172.21.15.105:41280) 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 0x7fe96c139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 -- 172.21.15.105:0/1508078130 learned_addr learned my addr 172.21.15.105:0/1508078130 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): get_auth_request con 0x7fe96c0a6cc0 auth_method 0 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): get_auth_request con 0x7fe96c1396a0 auth_method 0 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971794640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c0a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971794640 10 monclient(hunting): get_auth_request con 0x7fe96c0a68e0 auth_method 0 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971794640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971794640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971794640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient(hunting): handle_auth_done global_id 55223 payload 274 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 -- 172.21.15.105:0/1508078130 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 msgr2=0x7fe96c0a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c0a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 -- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 msgr2=0x7fe96c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 0x7fe96c139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 monclient: found mon.noname-e 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe970f93640 1 -- 172.21.15.105:0/1508078130 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c0a7bf0 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.046+0000 7fe971f95640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 0x7fe96c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe970f93640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0098d0 tx=0x7fe95c01c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=42a2bcd609ad62cf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 1 -- 172.21.15.105:0/1508078130 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe95c0b0070 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: dump: 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.062 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 1 -- 172.21.15.105:0/1508078130 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95c0aa810 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 5 monclient: authenticate success, global_id 55223 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 1 -- 172.21.15.105:0/1508078130 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe95c0a9300 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 4 set_mon_vals no callback set 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe963fff640 10 monclient: dump: 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.064 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: shutdown 2024-09-17T16:46:40.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 msgr2=0x7fe96c0a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0098d0 tx=0x7fe95c01c440 comp rx=0 tx=0).stop 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- 172.21.15.105:0/1508078130 shutdown_connections 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7fe96c1396a0 0x7fe96c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe96c0a6cc0 0x7fe96c0a7100 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1508078130 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c0a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- 172.21.15.105:0/1508078130 >> 172.21.15.105:0/1508078130 conn(0x7fe96c068ba0 msgr2=0x7fe96c068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- 172.21.15.105:0/1508078130 shutdown_connections 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- 172.21.15.105:0/1508078130 wait complete. 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: monmap: 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.066 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.067 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 librados: starting msgr at 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 librados: starting objecter 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 Processor -- start 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- start start 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 librados: setting wanted keys 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 librados: calling monclient init 2024-09-17T16:46:40.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: init 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient: _add_conns ranks=[7,5,1,0,3,4,6,2,8] 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c1d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-i con 0x7fe96c0a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe96c0a6cc0 0x7fe96c1d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-h con 0x7fe96c0a6cc0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe96c1396a0 0x7fe96c13d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): picked mon.noname-f con 0x7fe96c1396a0 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:40.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe96c0a7640 con 0x7fe96c1396a0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fe96c13a430 con 0x7fe96c0a6cc0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe96c143390 con 0x7fe96c0a68e0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.050+0000 7fe9739f8640 10 monclient(hunting): authenticate will time out at 2083.319749s 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c1d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe96c1396a0 0x7fe96c13d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe96c0a6cc0 0x7fe96c1d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c1d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:35120/0 (socket says 172.21.15.105:35120) 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 -- 172.21.15.105:0/1579385521 learned_addr learned my addr 172.21.15.105:0/1579385521 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): get_auth_request con 0x7fe96c0a68e0 auth_method 0 2024-09-17T16:46:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): get_auth_request con 0x7fe96c0a6cc0 auth_method 0 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): get_auth_request con 0x7fe96c1396a0 auth_method 0 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient(hunting): handle_auth_done global_id 64879 payload 824 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 -- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe96c0a6cc0 msgr2=0x7fe96c1d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fe96c0a6cc0 0x7fe96c1d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 -- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe96c1396a0 msgr2=0x7fe96c13d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe96c1396a0 0x7fe96c13d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 monclient: found mon.noname-i 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.160:3303/0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96c123cd0 con 0x7fe96c0a68e0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe96c1396a0 0x7fe96c13d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971794640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c1d1540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe964007ef0 tx=0x7fe96400e4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.?.objecter ms_handle_connect 0x7fe96c0a68e0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe964014890 con 0x7fe96c0a68e0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: dump: 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 5 monclient: authenticate success, global_id 64879 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 monclient: _renew_subs 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96c13fc20 con 0x7fe96c0a68e0 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964019070 con 0x7fe96c0a68e0 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe964014a30 con 0x7fe96c0a68e0 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: dump: 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.079 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9637fe640 4 set_mon_vals no callback set 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 client.64879.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 monclient: _renew_subs 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96c0a48a0 con 0x7fe96c0a68e0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 1 librados: init done 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 client.64879.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 15 client.64879.objecter _session_command_op_assign -1 1 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 20 client.64879.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 client.64879.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.160:3303/0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe96c0a7100 con 0x7fe96c0a68e0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe964014bd0 con 0x7fe96c0a68e0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe95003e550 0x7fe950040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fe9640513a0 con 0x7fe96c0a68e0 2024-09-17T16:46:40.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.64879.objecter ms_dispatch 0x7fe96c000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 3 client.64879.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 3 client.64879.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.64879.objecter checking command 1 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fe950045d10 0x7fe9500481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter _get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 15 client.64879.objecter _session_command_op_remove -1 1 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter _get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 15 client.64879.objecter _session_command_op_assign 1 1 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 4 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.64879.objecter _send_command 1 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe964021e80 con 0x7fe950045d10 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 20 client.64879.objecter dump_active .. 0 homeless 2024-09-17T16:46:40.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== mon.7 v2:172.21.15.160:3303/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7fe964051750 con 0x7fe96c0a68e0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe95003e550 0x7fe950040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fe950045d10 0x7fe9500481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 10 monclient: get_auth_request con 0x7fe950045d10 auth_method 0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe970f93640 10 monclient: get_auth_request con 0x7fe95003e550 auth_method 0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe971f95640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fe950045d10 0x7fe9500481b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.054+0000 7fe961ffb640 10 client.64879.objecter ms_handle_connect 0x7fe950045d10 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe970f93640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe95003e550 0x7fe950040a10 secure :-1 s=READY pgs=3682 cs=0 l=1 rev1=1 crypto rx=0x7fe96c069c20 tx=0x7fe95c0a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 10 client.64879.objecter ms_handle_connect 0x7fe95003e550 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== osd.1 v2:172.21.15.105:6800/3243801162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe964021e80 con 0x7fe950045d10 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 10 client.64879.objecter ms_dispatch 0x7fe96c000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 10 client.64879.objecter _finish_command 1 = system:0 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 4 2024-09-17T16:46:40.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.058+0000 7fe961ffb640 15 client.64879.objecter _session_command_op_remove 1 1 2024-09-17T16:46:40.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 client.64879.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:40.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 15 client.64879.objecter _session_command_op_assign -1 2 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter _get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter _get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 15 client.64879.objecter _session_command_op_remove -1 2 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter get_session s=0x7fe950045640 osd=1 3 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 15 client.64879.objecter _session_command_op_assign 1 2 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 4 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 client.64879.objecter _send_command 2 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 --> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe96c123950 con 0x7fe950045d10 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe961ffb640 1 -- 172.21.15.105:0/1579385521 <== osd.1 v2:172.21.15.105:6800/3243801162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe96c123950 con 0x7fe950045d10 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe961ffb640 10 client.64879.objecter ms_dispatch 0x7fe96c000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe961ffb640 10 client.64879.objecter _finish_command 2 = system:0 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe961ffb640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 4 2024-09-17T16:46:40.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe961ffb640 15 client.64879.objecter _session_command_op_remove 1 2 2024-09-17T16:46:40.086 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:40.086 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:40.086 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 librados: watch_flush enter 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 librados: watch_flush exit 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 client.64879.objecter close_session for osd.1 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fe950045d10 msgr2=0x7fe9500481b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.105:6800/3243801162,v1:172.21.15.105:6801/3243801162] conn(0x7fe950045d10 0x7fe9500481b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter put_session s=0x7fe950045640 osd=1 1 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 client.64879.objecter shutdown clearing up homeless session... 2024-09-17T16:46:40.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 client.64879.objecter successfully canceled tick 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe95003e550 msgr2=0x7fe950040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe95003e550 0x7fe950040a10 secure :-1 s=READY pgs=3682 cs=0 l=1 rev1=1 crypto rx=0x7fe96c069c20 tx=0x7fe95c0a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 10 monclient: shutdown 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 msgr2=0x7fe96c1d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 --2- 172.21.15.105:0/1579385521 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe96c0a68e0 0x7fe96c1d1540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe964007ef0 tx=0x7fe96400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe971f95640 1 -- 172.21.15.105:0/1579385521 reap_dead start 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 shutdown_connections 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 >> 172.21.15.105:0/1579385521 conn(0x7fe96c068ba0 msgr2=0x7fe96c0a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 shutdown_connections 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 -- 172.21.15.105:0/1579385521 wait complete. 2024-09-17T16:46:40.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.070+0000 7fe9739f8640 1 librados: shutdown 2024-09-17T16:46:40.094 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T16:46:40.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.198+0000 7f6254f75640 10 monclient: get_monmap_and_config 2024-09-17T16:46:40.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.198+0000 7f6254f75640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.198+0000 7f6254f75640 10 monclient: monmap: 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:40.203239+0000 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:40.203239+0000 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.207 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:40.208 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 Processor -- start 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 -- start start 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient: init 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient: _add_conns ranks=[4,5,1,0,6,10,9,3,8,7,2] 2024-09-17T16:46:40.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-c con 0x7f62500a4e00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6250137b50 0x7f6250137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-h con 0x7f6250137b50 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f62500a51e0 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62500a1c30 con 0x7f62500a51e0 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f62500a18f0 con 0x7f62500a4e00 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f62500a2c40 con 0x7f6250137b50 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f6254f75640 10 monclient(hunting): authenticate will time out at 2083.468921s 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:42824/0 (socket says 172.21.15.105:42824) 2024-09-17T16:46:40.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 -- 172.21.15.105:0/132260845 learned_addr learned my addr 172.21.15.105:0/132260845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): get_auth_request con 0x7f62500a4e00 auth_method 0 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624dd9b640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6250137b50 0x7f6250137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624dd9b640 10 monclient(hunting): get_auth_request con 0x7f6250137b50 auth_method 0 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient(hunting): handle_auth_done global_id 74992 payload 274 2024-09-17T16:46:40.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 -- 172.21.15.105:0/132260845 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6250137b50 msgr2=0x7f6250137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6250137b50 0x7f6250137f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 -- 172.21.15.105:0/132260845 >> v1:172.21.15.160:6789/0 conn(0x7f62500a51e0 legacy=0x7f62500a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 monclient: found mon.noname-c 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:40.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.202+0000 7f624e59c640 1 -- 172.21.15.105:0/132260845 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62500a2f20 con 0x7f62500a4e00 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f624001c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=36264addbf61f904 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 1 -- 172.21.15.105:0/132260845 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62400b1070 con 0x7f62500a4e00 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: dump: 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 5 monclient: authenticate success, global_id 74992 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 1 -- 172.21.15.105:0/132260845 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62400aa7b0 con 0x7f62500a4e00 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 4 set_mon_vals no callback set 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 1 -- 172.21.15.105:0/132260845 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62400aae10 con 0x7f62500a4e00 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624d59a640 10 monclient: dump: 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: shutdown 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/132260845 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 msgr2=0x7f6250137610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62400098a0 tx=0x7f624001c440 comp rx=0 tx=0).stop 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/132260845 shutdown_connections 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f6250137b50 0x7f6250137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- 172.21.15.105:0/132260845 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f6250137610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/132260845 >> 172.21.15.105:0/132260845 conn(0x7f62500aa250 msgr2=0x7f62500aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/132260845 shutdown_connections 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/132260845 wait complete. 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: monmap: 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 librados: starting msgr at 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 librados: starting objecter 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 Processor -- start 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- start start 2024-09-17T16:46:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 librados: setting wanted keys 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 librados: calling monclient init 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: init 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _add_conns ranks=[4,3,6,8,1,0,2,5,7] 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-c con 0x7f62500a4e00 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62500a51e0 0x7f62501d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-g con 0x7f62500a51e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6250137b50 0x7f62501d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): picked mon.noname-d con 0x7f6250137b50 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f6250142420 con 0x7f62500a51e0 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f62501422a0 con 0x7f62500a4e00 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f62501425a0 con 0x7f6250137b50 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient(hunting): authenticate will time out at 2083.474739s 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:42830/0 (socket says 172.21.15.105:42830) 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 -- 172.21.15.105:0/3386889150 learned_addr learned my addr 172.21.15.105:0/3386889150 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): get_auth_request con 0x7f62500a4e00 auth_method 0 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6250137b50 0x7f62501d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): get_auth_request con 0x7f6250137b50 auth_method 0 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624dd9b640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62500a51e0 0x7f62501d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624dd9b640 10 monclient(hunting): get_auth_request con 0x7f62500a51e0 auth_method 0 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient(hunting): handle_auth_done global_id 75001 payload 824 2024-09-17T16:46:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 -- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6250137b50 msgr2=0x7f62501d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6250137b50 0x7f62501d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 -- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62500a51e0 msgr2=0x7f62501d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62500a51e0 0x7f62501d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 monclient: found mon.noname-c 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 -- 172.21.15.105:0/3386889150 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6250122c90 con 0x7f62500a4e00 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624e59c640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62400af040 tx=0x7f62400191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 client.?.objecter ms_handle_connect 0x7f62500a4e00 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62400ba070 con 0x7f62500a4e00 2024-09-17T16:46:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: dump: 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62400aadd0 con 0x7f62500a4e00 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 5 monclient: authenticate success, global_id 75001 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _renew_subs 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6250073d30 con 0x7f62500a4e00 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62400b1040 con 0x7f62500a4e00 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f624cd99640 4 set_mon_vals no callback set 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 client.75001.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6246ffd640 10 monclient: dump: 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _renew_subs 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6250073890 con 0x7f62500a4e00 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.206+0000 7f6254f75640 1 librados: init done 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6254f75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6254f75640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6254f75640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62500008e0 con 0x7f62500a4e00 2024-09-17T16:46:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f62400aa780 con 0x7f62500a4e00 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 0x7f6230040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f62400e8e70 con 0x7f62500a4e00 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 10 client.75001.objecter ms_dispatch 0x7f6250000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 3 client.75001.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 3 client.75001.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 20 client.75001.objecter dump_active .. 0 homeless 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f624dd9b640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 0x7f6230040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f624dd9b640 10 monclient: get_auth_request con 0x7f623003e550 auth_method 0 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f6246ffd640 10 client.75001.objecter ms_handle_connect 0x7f623003e550 2024-09-17T16:46:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.210+0000 7f624dd9b640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 0x7f6230040a10 secure :-1 s=READY pgs=3683 cs=0 l=1 rev1=1 crypto rx=0x7f62500aae10 tx=0x7f6238008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.214+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6240016150 con 0x7f62500a4e00 2024-09-17T16:46:40.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.214+0000 7f6246ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.214+0000 7f6246ffd640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:40.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.390+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f6250122910 con 0x7f623003e550 2024-09-17T16:46:40.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6246ffd640 1 -- 172.21.15.105:0/3386889150 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6250122910 con 0x7f623003e550 2024-09-17T16:46:40.400 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T16:46:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 10 librados: watch_flush enter 2024-09-17T16:46:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 10 librados: watch_flush exit 2024-09-17T16:46:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 20 client.75001.objecter shutdown clearing up homeless session... 2024-09-17T16:46:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 10 client.75001.objecter successfully canceled tick 2024-09-17T16:46:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 msgr2=0x7f6230040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 0x7f6230040a10 secure :-1 s=READY pgs=3683 cs=0 l=1 rev1=1 crypto rx=0x7f62500aae10 tx=0x7f6238008040 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 10 monclient: shutdown 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 msgr2=0x7f62501d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62400af040 tx=0x7f62400191e0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 shutdown_connections 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f623003e550 0x7f6230040a10 unknown :-1 s=CLOSED pgs=3683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f6250137b50 0x7f62501d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62500a51e0 0x7f62501d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 --2- 172.21.15.105:0/3386889150 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f62500a4e00 0x7f62501d1620 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 >> 172.21.15.105:0/3386889150 conn(0x7f62500aa250 msgr2=0x7f62500a8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 shutdown_connections 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 -- 172.21.15.105:0/3386889150 wait complete. 2024-09-17T16:46:40.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.394+0000 7f6254f75640 1 librados: shutdown 2024-09-17T16:46:40.418 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T16:46:40.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: get_monmap_and_config 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: monmap: 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.548 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:40.544721+0000 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:40.544721+0000 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:40.549 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:40.550 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 Processor -- start 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 -- start start 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: init 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient: _add_conns ranks=[10,3,9,1,5,2,8,6,7,4,0] 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f98300a68e0 0x7f98300a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): picked mon.noname-f con 0x7f98300a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98300a6cc0 0x7f98300a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): picked mon.noname-g con 0x7f98300a6cc0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98301396a0 0x7f9830139b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): picked mon.noname-a con 0x7f98301396a0 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f983007d3b0 con 0x7f98301396a0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f98300a87d0 con 0x7f98300a68e0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f98300a4de0 con 0x7f98300a6cc0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f983680c640 10 monclient(hunting): authenticate will time out at 2083.811325s 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98301396a0 0x7f9830139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98301396a0 0x7f9830139b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37762/0 (socket says 172.21.15.105:37762) 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 -- 172.21.15.105:0/618653719 learned_addr learned my addr 172.21.15.105:0/618653719 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): get_auth_request con 0x7f98301396a0 auth_method 0 2024-09-17T16:46:40.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982ffff640 1 --2- 172.21.15.105:0/618653719 >> v2:172.21.15.160:3300/0 conn(0x7f98300a68e0 0x7f98300a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982f7fe640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98300a6cc0 0x7f98300a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982ffff640 10 monclient(hunting): get_auth_request con 0x7f98300a68e0 auth_method 0 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982f7fe640 10 monclient(hunting): get_auth_request con 0x7f98300a6cc0 auth_method 0 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f982f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient(hunting): handle_auth_done global_id 104256 payload 274 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 -- 172.21.15.105:0/618653719 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98300a6cc0 msgr2=0x7f98300a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98300a6cc0 0x7f98300a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 -- 172.21.15.105:0/618653719 >> v2:172.21.15.160:3300/0 conn(0x7f98300a68e0 msgr2=0x7f98300a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 --2- 172.21.15.105:0/618653719 >> v2:172.21.15.160:3300/0 conn(0x7f98300a68e0 0x7f98300a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 monclient: found mon.noname-a 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:40.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.542+0000 7f9834da9640 1 -- 172.21.15.105:0/618653719 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98300a7bf0 con 0x7f98301396a0 2024-09-17T16:46:40.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f9834da9640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301396a0 0x7f9830139b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f983006d2f0 tx=0x7f982801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb468d1fdca1163 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 1 -- 172.21.15.105:0/618653719 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98280b0070 con 0x7f98301396a0 2024-09-17T16:46:40.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:40.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: dump: 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.563 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 1 -- 172.21.15.105:0/618653719 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98280aa810 con 0x7f98301396a0 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 5 monclient: authenticate success, global_id 104256 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 1 -- 172.21.15.105:0/618653719 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98280a9300 con 0x7f98301396a0 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 4 set_mon_vals no callback set 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f982effd640 10 monclient: dump: 2024-09-17T16:46:40.564 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.565 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 10 monclient: shutdown 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 -- 172.21.15.105:0/618653719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301396a0 msgr2=0x7f9830139b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301396a0 0x7f9830139b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f983006d2f0 tx=0x7f982801c3c0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 -- 172.21.15.105:0/618653719 shutdown_connections 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98300a6cc0 0x7f98300a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 --2- 172.21.15.105:0/618653719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301396a0 0x7f9830139b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 --2- 172.21.15.105:0/618653719 >> v2:172.21.15.160:3300/0 conn(0x7f98300a68e0 0x7f98300a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 -- 172.21.15.105:0/618653719 >> 172.21.15.105:0/618653719 conn(0x7f9830068ba0 msgr2=0x7f9830068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 -- 172.21.15.105:0/618653719 shutdown_connections 2024-09-17T16:46:40.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 -- 172.21.15.105:0/618653719 wait complete. 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 10 monclient: monmap: 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.567 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 librados: starting msgr at 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.546+0000 7f983680c640 1 librados: starting objecter 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 Processor -- start 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- start start 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 librados: setting wanted keys 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 librados: calling monclient init 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: init 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _add_conns ranks=[7,0,2,6,5,4,3,8,1] 2024-09-17T16:46:40.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98300a68e0 0x7f98301d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): picked mon.noname-i con 0x7f98300a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 0x7f98301d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): picked mon.noname-a con 0x7f98300a6cc0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98301396a0 0x7f98300a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): picked mon.noname-b con 0x7f98301396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98300a7640 con 0x7f98300a6cc0 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f983006e6a0 con 0x7f98301396a0 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f9830143390 con 0x7f98300a68e0 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient(hunting): authenticate will time out at 2083.816719s 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 0x7f98301d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 0x7f98301d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37770/0 (socket says 172.21.15.105:37770) 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 -- 172.21.15.105:0/297466469 learned_addr learned my addr 172.21.15.105:0/297466469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): get_auth_request con 0x7f98300a6cc0 auth_method 0 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98301396a0 0x7f98300a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): get_auth_request con 0x7f98301396a0 auth_method 0 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98300a68e0 0x7f98301d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): get_auth_request con 0x7f98300a68e0 auth_method 0 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient(hunting): handle_auth_done global_id 104265 payload 824 2024-09-17T16:46:40.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 -- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98300a68e0 msgr2=0x7f98301d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f98300a68e0 0x7f98301d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 -- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98301396a0 msgr2=0x7f98300a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98301396a0 0x7f98300a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 monclient: found mon.noname-a 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98301239a0 con 0x7f98300a6cc0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982f7fe640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 0x7f98301d1a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f982400d7d0 tx=0x7f982400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.?.objecter ms_handle_connect 0x7f98300a6cc0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f982400bb30 con 0x7f98300a6cc0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:40.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: dump: 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.573 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982400bcf0 con 0x7f98300a6cc0 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 5 monclient: authenticate success, global_id 104265 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _renew_subs 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98300a22f0 con 0x7f98300a6cc0 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98240148a0 con 0x7f98300a6cc0 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 client.104265.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982e7fc640 4 set_mon_vals no callback set 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: dump: 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.575 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _renew_subs 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9830069110 con 0x7f98300a6cc0 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 librados: init done 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 client.104265.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 15 client.104265.objecter _session_command_op_assign -1 1 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 20 client.104265.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 client.104265.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f983013fb90 con 0x7f98300a6cc0 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f982401b030 con 0x7f98300a6cc0 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f981c03e550 0x7f981c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f9824019020 con 0x7f98300a6cc0 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.104265.objecter ms_dispatch 0x7f9830000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 3 client.104265.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 3 client.104265.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.104265.objecter checking command 1 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f981c045d10 0x7f981c0481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter _get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 15 client.104265.objecter _session_command_op_remove -1 1 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter _get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 15 client.104265.objecter _session_command_op_assign 2 1 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 4 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.104265.objecter _send_command 1 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98240044c0 con 0x7f981c045d10 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 20 client.104265.objecter dump_active .. 0 homeless 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7f98240542c0 con 0x7f98300a6cc0 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f981c045d10 0x7f981c0481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 10 monclient: get_auth_request con 0x7f981c045d10 auth_method 0 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f981c03e550 0x7f981c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982ffff640 10 monclient: get_auth_request con 0x7f981c03e550 auth_method 0 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f9834da9640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f981c045d10 0x7f981c0481b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.550+0000 7f982cff9640 10 client.104265.objecter ms_handle_connect 0x7f981c045d10 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 10 client.104265.objecter ms_handle_connect 0x7f981c03e550 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982ffff640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f981c03e550 0x7f981c040a10 secure :-1 s=READY pgs=3684 cs=0 l=1 rev1=1 crypto rx=0x7f9820003b90 tx=0x7f98200098d0 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== osd.2 v2:172.21.15.105:6808/1006010180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98240044c0 con 0x7f981c045d10 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 10 client.104265.objecter ms_dispatch 0x7f9830000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 10 client.104265.objecter _finish_command 1 = system:0 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 4 2024-09-17T16:46:40.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.554+0000 7f982cff9640 15 client.104265.objecter _session_command_op_remove 2 1 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 client.104265.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 15 client.104265.objecter _session_command_op_assign -1 2 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter _get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter _get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 15 client.104265.objecter _session_command_op_remove -1 2 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter get_session s=0x7f981c045640 osd=2 3 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 15 client.104265.objecter _session_command_op_assign 2 2 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 4 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 client.104265.objecter _send_command 2 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 --> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98300a7100 con 0x7f981c045d10 2024-09-17T16:46:40.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f982cff9640 1 -- 172.21.15.105:0/297466469 <== osd.2 v2:172.21.15.105:6808/1006010180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98300a7100 con 0x7f981c045d10 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f982cff9640 10 client.104265.objecter ms_dispatch 0x7f9830000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f982cff9640 10 client.104265.objecter _finish_command 2 = system:0 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f982cff9640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 4 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f982cff9640 15 client.104265.objecter _session_command_op_remove 2 2 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:40.581 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:40.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 librados: watch_flush enter 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 librados: watch_flush exit 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 client.104265.objecter close_session for osd.2 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f981c045d10 msgr2=0x7f981c0481b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:6808/1006010180,v1:172.21.15.105:6809/1006010180] conn(0x7f981c045d10 0x7f981c0481b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter put_session s=0x7f981c045640 osd=2 1 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 client.104265.objecter shutdown clearing up homeless session... 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 client.104265.objecter successfully canceled tick 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f981c03e550 msgr2=0x7f981c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f981c03e550 0x7f981c040a10 secure :-1 s=READY pgs=3684 cs=0 l=1 rev1=1 crypto rx=0x7f9820003b90 tx=0x7f98200098d0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 10 monclient: shutdown 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 msgr2=0x7f98301d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 --2- 172.21.15.105:0/297466469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98300a6cc0 0x7f98301d1a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f982400d7d0 tx=0x7f982400dc90 comp rx=0 tx=0).stop 2024-09-17T16:46:40.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f9834da9640 1 -- 172.21.15.105:0/297466469 reap_dead start 2024-09-17T16:46:40.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 shutdown_connections 2024-09-17T16:46:40.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 >> 172.21.15.105:0/297466469 conn(0x7f9830068ba0 msgr2=0x7f98300a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 shutdown_connections 2024-09-17T16:46:40.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 -- 172.21.15.105:0/297466469 wait complete. 2024-09-17T16:46:40.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.566+0000 7f983680c640 1 librados: shutdown 2024-09-17T16:46:40.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: get_monmap_and_config 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: monmap: 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:40.700726+0000 2024-09-17T16:46:40.704 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:40.700726+0000 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:40.705 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:40.706 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 Processor -- start 2024-09-17T16:46:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 -- start start 2024-09-17T16:46:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: init 2024-09-17T16:46:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient: _add_conns ranks=[8,9,10,6,2,4,5,3,1,0,7] 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 0x7f88d0137610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-e con 0x7f88d00a4e00 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-a con 0x7f88d0137b50 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f88d00a51e0 0x7f88d00a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-f con 0x7f88d00a51e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00a1c30 con 0x7f88d0137b50 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00a18f0 con 0x7f88d00a51e0 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f88d00a2c40 con 0x7f88d00a4e00 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d66eb640 10 monclient(hunting): authenticate will time out at 2083.967236s 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 0x7f88d0137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37780/0 (socket says 172.21.15.105:37780) 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 1 -- 172.21.15.105:0/2826687974 learned_addr learned my addr 172.21.15.105:0/2826687974 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): get_auth_request con 0x7f88d0137b50 auth_method 0 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): get_auth_request con 0x7f88d00a4e00 auth_method 0 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d4c88640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.160:3300/0 conn(0x7f88d00a51e0 0x7f88d00a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d4c88640 10 monclient(hunting): get_auth_request con 0x7f88d00a51e0 auth_method 0 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d4c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d4c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88d4c88640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient(hunting): handle_auth_done global_id 55232 payload 274 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 -- 172.21.15.105:0/2826687974 >> v2:172.21.15.160:3300/0 conn(0x7f88d00a51e0 msgr2=0x7f88d00a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.160:3300/0 conn(0x7f88d00a51e0 0x7f88d00a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 -- 172.21.15.105:0/2826687974 >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 msgr2=0x7f88d0137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 monclient: found mon.noname-e 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cffff640 1 -- 172.21.15.105:0/2826687974 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00a2f20 con 0x7f88d00a4e00 2024-09-17T16:46:40.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.698+0000 7f88cf7fe640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88cffff640 1 --2- 172.21.15.105:0/2826687974 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 0x7f88d0137610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88d00a6880 tx=0x7f88c001c340 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8d43c665841fa649 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 1 -- 172.21.15.105:0/2826687974 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88c00b0070 con 0x7f88d00a4e00 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: dump: 2024-09-17T16:46:40.717 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.718 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 1 -- 172.21.15.105:0/2826687974 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00aa810 con 0x7f88d00a4e00 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 5 monclient: authenticate success, global_id 55232 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 1 -- 172.21.15.105:0/2826687974 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88c00a9300 con 0x7f88d00a4e00 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 4 set_mon_vals no callback set 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:40.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88ceffd640 10 monclient: dump: 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.720 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 10 monclient: shutdown 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- 172.21.15.105:0/2826687974 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 msgr2=0x7f88d0137610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 --2- 172.21.15.105:0/2826687974 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 0x7f88d0137610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88d00a6880 tx=0x7f88c001c340 comp rx=0 tx=0).stop 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- 172.21.15.105:0/2826687974 shutdown_connections 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.160:3300/0 conn(0x7f88d00a51e0 0x7f88d00a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 --2- 172.21.15.105:0/2826687974 >> v2:172.21.15.105:3300/0 conn(0x7f88d0137b50 0x7f88d0137f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 --2- 172.21.15.105:0/2826687974 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f88d00a4e00 0x7f88d0137610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- 172.21.15.105:0/2826687974 >> 172.21.15.105:0/2826687974 conn(0x7f88d00aa250 msgr2=0x7f88d00aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- 172.21.15.105:0/2826687974 shutdown_connections 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- 172.21.15.105:0/2826687974 wait complete. 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 10 monclient: build_initial_monmap 2024-09-17T16:46:40.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 10 monclient: monmap: 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:40.722 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 librados: starting msgr at 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 librados: starting objecter 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 Processor -- start 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 -- start start 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 librados: setting wanted keys 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 1 librados: calling monclient init 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.702+0000 7f88d66eb640 10 monclient: init 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _add_conns ranks=[1,3,7,0,8,2,6,5,4] 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 0x7f88d01cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-f con 0x7f88d00a4e00 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 0x7f88d01cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-g con 0x7f88d00a51e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 0x7f88d01cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): picked mon.noname-i con 0x7f88d0137b50 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0141200 con 0x7f88d00a4e00 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f88d0141080 con 0x7f88d00a51e0 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f88d0141380 con 0x7f88d0137b50 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient(hunting): authenticate will time out at 2083.972415s 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 0x7f88d01cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 0x7f88d01cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 0x7f88d01cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 0x7f88d01cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3303/0 says I am v2:172.21.15.105:35136/0 (socket says 172.21.15.105:35136) 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 1 -- 172.21.15.105:0/3867283273 learned_addr learned my addr 172.21.15.105:0/3867283273 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 0x7f88d01cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.105:57330/0 (socket says 172.21.15.105:57330) 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): get_auth_request con 0x7f88d0137b50 auth_method 0 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): get_auth_request con 0x7f88d00a51e0 auth_method 0 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): get_auth_request con 0x7f88d00a4e00 auth_method 0 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:40.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d4c88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient(hunting): handle_auth_done global_id 104185 payload 824 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 -- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 msgr2=0x7f88d01cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 0x7f88d01cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 -- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 msgr2=0x7f88d01cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 0x7f88d01cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient: _finish_hunting 0 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 monclient: found mon.noname-f 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.160:3300/0 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 -- 172.21.15.105:0/3867283273 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0137420 con 0x7f88d00a4e00 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cffff640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 0x7f88d01cf2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88c00ab000 tx=0x7f88c00190e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 client.?.objecter ms_handle_connect 0x7f88d00a4e00 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88c00b0040 con 0x7f88d00a4e00 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: dump: 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.727 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: _finish_auth 0 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: _check_auth_tickets 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 5 monclient: authenticate success, global_id 104185 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c00044b0 con 0x7f88d00a4e00 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _renew_subs 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d013d830 con 0x7f88d00a4e00 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88c00aac90 con 0x7f88d00a4e00 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 monclient: dump: 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:40.729 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:40.730 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ce7fc640 4 set_mon_vals no callback set 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 client.104185.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _renew_subs 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01211c0 con 0x7f88d00a4e00 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 librados: init done 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.160:3300/0 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d01375f0 con 0x7f88d00a4e00 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f88c0004040 con 0x7f88d00a4e00 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 0x7f88bc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f88c00ea390 con 0x7f88d00a4e00 2024-09-17T16:46:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 10 client.104185.objecter ms_dispatch 0x7f88d0000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 3 client.104185.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 3 client.104185.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88ccff9640 20 client.104185.objecter dump_active .. 0 homeless 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.706+0000 7f88cf7fe640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 0x7f88bc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.710+0000 7f88cf7fe640 10 monclient: get_auth_request con 0x7f88bc03e550 auth_method 0 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.710+0000 7f88ccff9640 10 client.104185.objecter ms_handle_connect 0x7f88bc03e550 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.710+0000 7f88cf7fe640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 0x7f88bc040a10 secure :-1 s=READY pgs=3685 cs=0 l=1 rev1=1 crypto rx=0x7f88c40096d0 tx=0x7f88c4009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.718+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f88c0016050 con 0x7f88d00a4e00 2024-09-17T16:46:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.718+0000 7f88ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:40.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.718+0000 7f88ccff9640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:40.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.882+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f88d013d390 con 0x7f88bc03e550 2024-09-17T16:46:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.882+0000 7f88ccff9640 1 -- 172.21.15.105:0/3867283273 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f88d013d390 con 0x7f88bc03e550 2024-09-17T16:46:40.891 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 10 librados: watch_flush enter 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 10 librados: watch_flush exit 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 20 client.104185.objecter shutdown clearing up homeless session... 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 10 client.104185.objecter successfully canceled tick 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 msgr2=0x7f88bc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 0x7f88bc040a10 secure :-1 s=READY pgs=3685 cs=0 l=1 rev1=1 crypto rx=0x7f88c40096d0 tx=0x7f88c4009240 comp rx=0 tx=0).stop 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 10 monclient: shutdown 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 msgr2=0x7f88d01cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 0x7f88d01cf2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88c00ab000 tx=0x7f88c00190e0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 shutdown_connections 2024-09-17T16:46:40.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f88bc03e550 0x7f88bc040a10 unknown :-1 s=CLOSED pgs=3685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88d00a51e0 0x7f88d01cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f88d0137b50 0x7f88d01cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 --2- 172.21.15.105:0/3867283273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88d00a4e00 0x7f88d01cf2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 >> 172.21.15.105:0/3867283273 conn(0x7f88d00aa250 msgr2=0x7f88d00a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 shutdown_connections 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 -- 172.21.15.105:0/3867283273 wait complete. 2024-09-17T16:46:40.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:40.886+0000 7f88d66eb640 1 librados: shutdown 2024-09-17T16:46:40.908 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T16:46:40.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: get_monmap_and_config 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: monmap: 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:41.032608+0000 2024-09-17T16:46:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:41.032608+0000 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:41.037 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 Processor -- start 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 -- start start 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: init 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient: _add_conns ranks=[7,1,9,4,2,5,8,0,10,6,3] 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faca40a68e0 0x7faca40a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): picked mon.noname-i con 0x7faca40a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7faca40a6cc0 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faca413eb60 0x7faca4140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): picked mon.noname-a con 0x7faca413eb60 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca40a7bf0 con 0x7faca40a6cc0 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7faca407d3b0 con 0x7faca413eb60 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7faca40a87d0 con 0x7faca40a68e0 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7facab62c640 10 monclient(hunting): authenticate will time out at 2084.299134s 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faca413eb60 0x7faca4140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faca413eb60 0x7faca4140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37788/0 (socket says 172.21.15.105:37788) 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 -- 172.21.15.105:0/1099104138 learned_addr learned my addr 172.21.15.105:0/1099104138 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): get_auth_request con 0x7faca413eb60 auth_method 0 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca93c8640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faca40a68e0 0x7faca40a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca93c8640 10 monclient(hunting): get_auth_request con 0x7faca40a68e0 auth_method 0 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca93c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca93c8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca93c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient(hunting): handle_auth_done global_id 104283 payload 274 2024-09-17T16:46:41.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 -- 172.21.15.105:0/1099104138 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faca40a68e0 msgr2=0x7faca40a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faca40a68e0 0x7faca40a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 -- 172.21.15.105:0/1099104138 >> v1:172.21.15.160:6789/0 conn(0x7faca40a6cc0 legacy=0x7faca40a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 monclient: found mon.noname-a 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.030+0000 7faca9bc9640 1 -- 172.21.15.105:0/1099104138 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca406ec50 con 0x7faca413eb60 2024-09-17T16:46:41.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7faca9bc9640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca413eb60 0x7faca4140f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009e50 tx=0x7fac9c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca170a79350851c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 1 -- 172.21.15.105:0/1099104138 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac9c0090d0 con 0x7faca413eb60 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: dump: 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.050 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 1 -- 172.21.15.105:0/1099104138 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c0a5a90 con 0x7faca413eb60 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 1 -- 172.21.15.105:0/1099104138 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac9c0a5c80 con 0x7faca413eb60 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 5 monclient: authenticate success, global_id 104283 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 4 set_mon_vals no callback set 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7fac9bfff640 10 monclient: dump: 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.052 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 10 monclient: shutdown 2024-09-17T16:46:41.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 -- 172.21.15.105:0/1099104138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca413eb60 msgr2=0x7faca4140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca413eb60 0x7faca4140f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009e50 tx=0x7fac9c01c8f0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 -- 172.21.15.105:0/1099104138 shutdown_connections 2024-09-17T16:46:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca413eb60 0x7faca4140f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 --2- 172.21.15.105:0/1099104138 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7faca40a68e0 0x7faca40a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 -- 172.21.15.105:0/1099104138 >> 172.21.15.105:0/1099104138 conn(0x7faca4068ba0 msgr2=0x7faca4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 -- 172.21.15.105:0/1099104138 shutdown_connections 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 -- 172.21.15.105:0/1099104138 wait complete. 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 10 monclient: monmap: 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.056 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 librados: starting msgr at 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.034+0000 7facab62c640 1 librados: starting objecter 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 Processor -- start 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- start start 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 librados: setting wanted keys 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 librados: calling monclient init 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: init 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _add_conns ranks=[0,6,4,7,3,8,5,2,1] 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 0x7faca4123310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): picked mon.noname-a con 0x7faca40a68e0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faca40a6cc0 0x7faca4123850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): picked mon.noname-d con 0x7faca40a6cc0 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faca413eb60 0x7faca4123d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): picked mon.noname-c con 0x7faca413eb60 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faca4143e30 con 0x7faca40a68e0 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7faca4143cb0 con 0x7faca413eb60 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7faca4143fb0 con 0x7faca40a6cc0 2024-09-17T16:46:41.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient(hunting): authenticate will time out at 2084.304686s 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 0x7faca4123310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 0x7faca4123310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37804/0 (socket says 172.21.15.105:37804) 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 -- 172.21.15.105:0/513323855 learned_addr learned my addr 172.21.15.105:0/513323855 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): get_auth_request con 0x7faca40a68e0 auth_method 0 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faca413eb60 0x7faca4123d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): get_auth_request con 0x7faca413eb60 auth_method 0 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faca40a6cc0 0x7faca4123850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 10 monclient(hunting): get_auth_request con 0x7faca40a6cc0 auth_method 0 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient(hunting): handle_auth_done global_id 75010 payload 824 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 -- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faca40a6cc0 msgr2=0x7faca4123850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faca40a6cc0 0x7faca4123850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 -- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 msgr2=0x7faca4123310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 0x7faca4123310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 monclient: found mon.noname-c 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41262c0 con 0x7faca413eb60 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faca40a68e0 0x7faca4123310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7faca40a6cc0 0x7faca4123850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca9bc9640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faca413eb60 0x7faca4123d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0ab000 tx=0x7fac9c019690 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.?.objecter ms_handle_connect 0x7faca413eb60 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac9c0bc020 con 0x7faca413eb60 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:41.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: dump: 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.062 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c003bb0 con 0x7faca413eb60 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 5 monclient: authenticate success, global_id 75010 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _renew_subs 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca4124dd0 con 0x7faca413eb60 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac9c003d60 con 0x7faca413eb60 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: dump: 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.064 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac9b7fe640 4 set_mon_vals no callback set 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 client.75010.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _renew_subs 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7faca413dae0 con 0x7faca413eb60 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 librados: init done 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 client.75010.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 15 client.75010.objecter _session_command_op_assign -1 1 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 20 client.75010.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 client.75010.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faca406da90 con 0x7faca413eb60 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fac9c0bf020 con 0x7faca413eb60 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fac8803e540 0x7fac88040a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fac9c0b3040 con 0x7faca413eb60 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.75010.objecter ms_dispatch 0x7faca4000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 3 client.75010.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 3 client.75010.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.75010.objecter checking command 1 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fac88045d00 0x7fac880481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter _get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 15 client.75010.objecter _session_command_op_remove -1 1 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter _get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 15 client.75010.objecter _session_command_op_assign 3 1 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 4 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.75010.objecter _send_command 1 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac88048870 con 0x7fac88045d00 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 20 client.75010.objecter dump_active .. 0 homeless 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7fac9c0ef2c0 con 0x7faca413eb60 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fac8803e540 0x7fac88040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 10 monclient: get_auth_request con 0x7fac8803e540 auth_method 0 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fac88045d00 0x7fac880481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca8bc7640 10 monclient: get_auth_request con 0x7fac88045d00 auth_method 0 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7fac99ffb640 10 client.75010.objecter ms_handle_connect 0x7fac8803e540 2024-09-17T16:46:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.038+0000 7faca93c8640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fac8803e540 0x7fac88040a00 secure :-1 s=READY pgs=3686 cs=0 l=1 rev1=1 crypto rx=0x7faca0007f00 tx=0x7faca0008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 10 client.75010.objecter ms_handle_connect 0x7fac88045d00 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7faca8bc7640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fac88045d00 0x7fac880481a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== osd.3 v2:172.21.15.160:6808/3218035673 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac88048870 con 0x7fac88045d00 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 10 client.75010.objecter ms_dispatch 0x7faca4000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 10 client.75010.objecter _finish_command 1 = system:0 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 4 2024-09-17T16:46:41.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.042+0000 7fac99ffb640 15 client.75010.objecter _session_command_op_remove 3 1 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 10 client.75010.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 15 client.75010.objecter _session_command_op_assign -1 2 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 20 client.75010.objecter _get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 20 client.75010.objecter _get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 15 client.75010.objecter _session_command_op_remove -1 2 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 20 client.75010.objecter get_session s=0x7fac88045630 osd=3 3 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 15 client.75010.objecter _session_command_op_assign 3 2 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 4 2024-09-17T16:46:41.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 10 client.75010.objecter _send_command 2 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 --> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faca4127630 con 0x7fac88045d00 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7fac99ffb640 1 -- 172.21.15.105:0/513323855 <== osd.3 v2:172.21.15.160:6808/3218035673 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faca4127630 con 0x7fac88045d00 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7fac99ffb640 10 client.75010.objecter ms_dispatch 0x7faca4000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7fac99ffb640 10 client.75010.objecter _finish_command 2 = system:0 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7fac99ffb640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 4 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.054+0000 7fac99ffb640 15 client.75010.objecter _session_command_op_remove 3 2 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:41.070 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 10 librados: watch_flush enter 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 10 librados: watch_flush exit 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 10 client.75010.objecter close_session for osd.3 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fac88045d00 msgr2=0x7fac880481a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6808/3218035673,v1:172.21.15.160:6809/3218035673] conn(0x7fac88045d00 0x7fac880481a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 20 client.75010.objecter put_session s=0x7fac88045630 osd=3 1 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 20 client.75010.objecter shutdown clearing up homeless session... 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 10 client.75010.objecter successfully canceled tick 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fac8803e540 msgr2=0x7fac88040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fac8803e540 0x7fac88040a00 secure :-1 s=READY pgs=3686 cs=0 l=1 rev1=1 crypto rx=0x7faca0007f00 tx=0x7faca0008040 comp rx=0 tx=0).stop 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 10 monclient: shutdown 2024-09-17T16:46:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faca413eb60 msgr2=0x7faca4123d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 --2- 172.21.15.105:0/513323855 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7faca413eb60 0x7faca4123d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0ab000 tx=0x7fac9c019690 comp rx=0 tx=0).stop 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7faca9bc9640 1 -- 172.21.15.105:0/513323855 reap_dead start 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 shutdown_connections 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 >> 172.21.15.105:0/513323855 conn(0x7faca4068ba0 msgr2=0x7faca40a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 shutdown_connections 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 -- 172.21.15.105:0/513323855 wait complete. 2024-09-17T16:46:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.058+0000 7facab62c640 1 librados: shutdown 2024-09-17T16:46:41.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T16:46:41.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: get_monmap_and_config 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: monmap: 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:41.184227+0000 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:41.184227+0000 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:41.189 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 Processor -- start 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 -- start start 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: init 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient: _add_conns ranks=[2,10,3,0,8,4,7,5,6,1,9] 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-b con 0x7fe7dc0a2620 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7dc0a2f40 0x7fe7dc137400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-f con 0x7fe7dc0a2f40 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7dc137940 0x7fe7dc137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-g con 0x7fe7dc137940 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe7dc06e580 con 0x7fe7dc0a2f40 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe7dc06f2e0 con 0x7fe7dc0a2620 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7fe7dc0a15e0 con 0x7fe7dc137940 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e3e27640 10 monclient(hunting): authenticate will time out at 2084.449874s 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38634/0 (socket says 172.21.15.105:38634) 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 -- 172.21.15.105:0/233169590 learned_addr learned my addr 172.21.15.105:0/233169590 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): get_auth_request con 0x7fe7dc0a2620 auth_method 0 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_done global_id 84710 payload 274 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 -- 172.21.15.105:0/233169590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7dc137940 msgr2=0x7fe7dc137df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/233169590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7dc137940 0x7fe7dc137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e13c2640 1 --2- 172.21.15.105:0/233169590 >> v2:172.21.15.160:3300/0 conn(0x7fe7dc0a2f40 0x7fe7dc137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 -- 172.21.15.105:0/233169590 >> v2:172.21.15.160:3300/0 conn(0x7fe7dc0a2f40 msgr2=0x7fe7dc137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/233169590 >> v2:172.21.15.160:3300/0 conn(0x7fe7dc0a2f40 0x7fe7dc137400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 monclient: found mon.noname-b 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:41.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.182+0000 7fe7e1bc3640 1 -- 172.21.15.105:0/233169590 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc0a3c30 con 0x7fe7dc0a2620 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/233169590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826e59f12beea3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 1 -- 172.21.15.105:0/233169590 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7d80b0070 con 0x7fe7dc0a2620 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: dump: 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.200 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.201 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 1 -- 172.21.15.105:0/233169590 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d80aa810 con 0x7fe7dc0a2620 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 5 monclient: authenticate success, global_id 84710 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 1 -- 172.21.15.105:0/233169590 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7d80a9300 con 0x7fe7dc0a2620 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 4 set_mon_vals no callback set 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e0bc1640 10 monclient: dump: 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: shutdown 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/233169590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 msgr2=0x7fe7dc0a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/233169590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d801c420 comp rx=0 tx=0).stop 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/233169590 shutdown_connections 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/233169590 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7dc137940 0x7fe7dc137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/233169590 >> v2:172.21.15.160:3300/0 conn(0x7fe7dc0a2f40 0x7fe7dc137400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/233169590 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7dc0a2620 0x7fe7dc0a2a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/233169590 >> 172.21.15.105:0/233169590 conn(0x7fe7dc068ba0 msgr2=0x7fe7dc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/233169590 shutdown_connections 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/233169590 wait complete. 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: monmap: 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.205 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 librados: starting msgr at 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 librados: starting objecter 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 Processor -- start 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- start start 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 librados: setting wanted keys 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 librados: calling monclient init 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: init 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _add_conns ranks=[4,7,8,0,5,6,3,2,1] 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-c con 0x7fe7dc0a2620 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7dc0a2f40 0x7fe7dc1cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-i con 0x7fe7dc0a2f40 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:41.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): picked mon.noname-e con 0x7fe7dc137940 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fe7dc0a1f50 con 0x7fe7dc0a2620 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fe7dc0a1dd0 con 0x7fe7dc0a2f40 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fe7dc141500 con 0x7fe7dc137940 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient(hunting): authenticate will time out at 2084.454727s 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41296/0 (socket says 172.21.15.105:41296) 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 -- 172.21.15.105:0/1431991820 learned_addr learned my addr 172.21.15.105:0/1431991820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:42840/0 (socket says 172.21.15.105:42840) 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): get_auth_request con 0x7fe7dc137940 auth_method 0 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): get_auth_request con 0x7fe7dc0a2620 auth_method 0 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e13c2640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7dc0a2f40 0x7fe7dc1cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e13c2640 10 monclient(hunting): get_auth_request con 0x7fe7dc0a2f40 auth_method 0 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e13c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e13c2640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e13c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient(hunting): handle_auth_done global_id 55241 payload 824 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 -- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7dc0a2f40 msgr2=0x7fe7dc1cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7dc0a2f40 0x7fe7dc1cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 -- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 msgr2=0x7fe7dc1cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 monclient: found mon.noname-e 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 -- 172.21.15.105:0/1431991820 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc11f400 con 0x7fe7dc137940 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e23c4640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b970 tx=0x7fe7d400be30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 client.?.objecter ms_handle_connect 0x7fe7dc137940 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7d4019070 con 0x7fe7dc137940 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: dump: 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d400cdb0 con 0x7fe7dc137940 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 5 monclient: authenticate success, global_id 55241 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7d40077e0 con 0x7fe7dc137940 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _renew_subs 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc11f9c0 con 0x7fe7dc137940 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d3fff640 4 set_mon_vals no callback set 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7d27fc640 10 monclient: dump: 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.212 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 client.55241.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _renew_subs 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc0a6d10 con 0x7fe7dc137940 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.186+0000 7fe7e3e27640 1 librados: init done 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e3e27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e3e27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e3e27640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc11f7a0 con 0x7fe7dc137940 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fe7d4007370 con 0x7fe7dc137940 2024-09-17T16:46:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 0x7fe7c0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fe7d404f880 con 0x7fe7dc137940 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 10 client.55241.objecter ms_dispatch 0x7fe7dc000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 3 client.55241.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 3 client.55241.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 20 client.55241.objecter dump_active .. 0 homeless 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 0x7fe7c0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e1bc3640 10 monclient: get_auth_request con 0x7fe7c003e550 auth_method 0 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7e1bc3640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 0x7fe7c0040a10 secure :-1 s=READY pgs=3687 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ad000 tx=0x7fe7d80a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.190+0000 7fe7d27fc640 10 client.55241.objecter ms_handle_connect 0x7fe7c003e550 2024-09-17T16:46:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.194+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7d401c360 con 0x7fe7dc137940 2024-09-17T16:46:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.194+0000 7fe7d27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.194+0000 7fe7d27fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.362+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe7dc0a2a00 con 0x7fe7c003e550 2024-09-17T16:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7d27fc640 1 -- 172.21.15.105:0/1431991820 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe7dc0a2a00 con 0x7fe7c003e550 2024-09-17T16:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 10 librados: watch_flush enter 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 10 librados: watch_flush exit 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 20 client.55241.objecter shutdown clearing up homeless session... 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 10 client.55241.objecter successfully canceled tick 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 msgr2=0x7fe7c0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 0x7fe7c0040a10 secure :-1 s=READY pgs=3687 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ad000 tx=0x7fe7d80a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 10 monclient: shutdown 2024-09-17T16:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 msgr2=0x7fe7dc13b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b970 tx=0x7fe7d400be30 comp rx=0 tx=0).stop 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 shutdown_connections 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fe7c003e550 0x7fe7c0040a10 unknown :-1 s=CLOSED pgs=3687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fe7dc0a2f40 0x7fe7dc1cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fe7dc137940 0x7fe7dc13b3b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 --2- 172.21.15.105:0/1431991820 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fe7dc0a2620 0x7fe7dc1cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 >> 172.21.15.105:0/1431991820 conn(0x7fe7dc068ba0 msgr2=0x7fe7dc06f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 shutdown_connections 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 -- 172.21.15.105:0/1431991820 wait complete. 2024-09-17T16:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.366+0000 7fe7e3e27640 1 librados: shutdown 2024-09-17T16:46:41.390 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T16:46:41.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: get_monmap_and_config 2024-09-17T16:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: monmap: 2024-09-17T16:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:41.508854+0000 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:41.508854+0000 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.513 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:41.514 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:41.514 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 Processor -- start 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 -- start start 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: init 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient: _add_conns ranks=[6,9,5,7,10,0,4,1,2,8,3] 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-d con 0x7f7a180a6a30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-a con 0x7f7a180a6e10 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7a180a77b0 0x7f7a181397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-h con 0x7f7a180a77b0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7a1807d3b0 con 0x7f7a180a6e10 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f7a180a3e30 con 0x7f7a180a77b0 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f7a180a4080 con 0x7f7a180a6a30 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1cebc640 10 monclient(hunting): authenticate will time out at 2084.774598s 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37814/0 (socket says 172.21.15.105:37814) 2024-09-17T16:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:45738/0 (socket says 172.21.15.105:45738) 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 1 -- 172.21.15.105:0/1440976839 learned_addr learned my addr 172.21.15.105:0/1440976839 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): get_auth_request con 0x7f7a180a6e10 auth_method 0 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): get_auth_request con 0x7f7a180a6a30 auth_method 0 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a16d9d640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7a180a77b0 0x7f7a181397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a16d9d640 10 monclient(hunting): get_auth_request con 0x7f7a180a77b0 auth_method 0 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a16d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient(hunting): handle_auth_done global_id 94497 payload 274 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 -- 172.21.15.105:0/1440976839 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7a180a77b0 msgr2=0x7f7a181397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7a180a77b0 0x7f7a181397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 -- 172.21.15.105:0/1440976839 >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 msgr2=0x7f7a180a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 --2- 172.21.15.105:0/1440976839 >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 monclient: found mon.noname-d 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a1659c640 1 -- 172.21.15.105:0/1440976839 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a180a3030 con 0x7f7a180a6a30 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.506+0000 7f7a15d9b640 1 --2- 172.21.15.105:0/1440976839 >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a080097c0 tx=0x7f7a0801c0a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7b5fa026989a0712 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 1 -- 172.21.15.105:0/1440976839 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a080b0070 con 0x7f7a180a6a30 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: dump: 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 1 -- 172.21.15.105:0/1440976839 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a080aa810 con 0x7f7a180a6a30 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 1 -- 172.21.15.105:0/1440976839 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a080a9300 con 0x7f7a180a6a30 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 5 monclient: authenticate success, global_id 94497 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 4 set_mon_vals no callback set 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1559a640 10 monclient: dump: 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: shutdown 2024-09-17T16:46:41.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- 172.21.15.105:0/1440976839 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 msgr2=0x7f7a180a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a080097c0 tx=0x7f7a0801c0a0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- 172.21.15.105:0/1440976839 shutdown_connections 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f7a180a77b0 0x7f7a181397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/1440976839 >> v2:172.21.15.105:3300/0 conn(0x7f7a180a6e10 0x7f7a180a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/1440976839 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f7a180a6a30 0x7f7a180a4a60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- 172.21.15.105:0/1440976839 >> 172.21.15.105:0/1440976839 conn(0x7f7a180a8b30 msgr2=0x7f7a180a8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- 172.21.15.105:0/1440976839 shutdown_connections 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- 172.21.15.105:0/1440976839 wait complete. 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: monmap: 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.531 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.532 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 librados: starting msgr at 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 librados: starting objecter 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 Processor -- start 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- start start 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 librados: setting wanted keys 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 librados: calling monclient init 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: init 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient: _add_conns ranks=[2,1,4,7,8,3,6,0,5] 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 0x7f7a181d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-b con 0x7f7a180a6a30 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a180a6e10 0x7f7a18138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-f con 0x7f7a180a6e10 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7a180a77b0 0x7f7a1813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): picked mon.noname-c con 0x7f7a180a77b0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18142ff0 con 0x7f7a180a6e10 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7a18142e70 con 0x7f7a180a6a30 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f7a18143170 con 0x7f7a180a77b0 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1cebc640 10 monclient(hunting): authenticate will time out at 2084.779770s 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 0x7f7a181d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 0x7f7a181d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38636/0 (socket says 172.21.15.105:38636) 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a16d9d640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7a180a77b0 0x7f7a1813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 1 -- 172.21.15.105:0/2522313163 learned_addr learned my addr 172.21.15.105:0/2522313163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 10 monclient(hunting): get_auth_request con 0x7f7a180a6a30 auth_method 0 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a1659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a16d9d640 10 monclient(hunting): get_auth_request con 0x7f7a180a77b0 auth_method 0 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a16d9d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.510+0000 7f7a16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a180a6e10 0x7f7a18138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 10 monclient(hunting): get_auth_request con 0x7f7a180a6e10 auth_method 0 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient(hunting): handle_auth_done global_id 75028 payload 824 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 -- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 msgr2=0x7f7a181d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 0x7f7a181d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 -- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a180a6e10 msgr2=0x7f7a18138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a180a6e10 0x7f7a18138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 monclient: found mon.noname-c 2024-09-17T16:46:41.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a18124380 con 0x7f7a180a77b0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7a180a6a30 0x7f7a181d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a16d9d640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7a180a77b0 0x7f7a1813df00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d4c0 tx=0x7f7a0000d980 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.?.objecter ms_handle_connect 0x7f7a180a77b0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a00012a60 con 0x7f7a180a77b0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: dump: 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.537 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a00012c20 con 0x7f7a180a77b0 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 5 monclient: authenticate success, global_id 75028 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 monclient: _renew_subs 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a180a2270 con 0x7f7a180a77b0 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a00012dd0 con 0x7f7a180a77b0 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: dump: 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 client.75028.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a14d99640 4 set_mon_vals no callback set 2024-09-17T16:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 monclient: _renew_subs 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a18122e60 con 0x7f7a180a77b0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 1 librados: init done 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 client.75028.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 15 client.75028.objecter _session_command_op_assign -1 1 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 20 client.75028.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 client.75028.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a181227a0 con 0x7f7a180a77b0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f7a0001b030 con 0x7f7a180a77b0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f79fc03e550 0x7f79fc040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f7a00019020 con 0x7f7a180a77b0 2024-09-17T16:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.75028.objecter ms_dispatch 0x7f7a18000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 3 client.75028.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 3 client.75028.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.75028.objecter checking command 1 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f79fc045d10 0x7f79fc0481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter _get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 15 client.75028.objecter _session_command_op_remove -1 1 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter _get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 15 client.75028.objecter _session_command_op_assign 4 1 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 4 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.75028.objecter _send_command 1 2024-09-17T16:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79fc048880 con 0x7f79fc045d10 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 20 client.75028.objecter dump_active .. 0 homeless 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7f7a00056050 con 0x7f7a180a77b0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f79fc03e550 0x7f79fc040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 10 monclient: get_auth_request con 0x7f79fc03e550 auth_method 0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f79fc045d10 0x7f79fc0481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 10 monclient: get_auth_request con 0x7f79fc045d10 auth_method 0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.75028.objecter ms_handle_connect 0x7f79fc03e550 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a1659c640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f79fc03e550 0x7f79fc040a10 secure :-1 s=READY pgs=3688 cs=0 l=1 rev1=1 crypto rx=0x7f7a0801bed0 tx=0x7f7a080a6000 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a0effd640 10 client.75028.objecter ms_handle_connect 0x7f79fc045d10 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.514+0000 7f7a15d9b640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f79fc045d10 0x7f79fc0481b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.518+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== osd.4 v2:172.21.15.160:6816/639753971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79fc048880 con 0x7f79fc045d10 2024-09-17T16:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.518+0000 7f7a0effd640 10 client.75028.objecter ms_dispatch 0x7f7a18000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:41.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.518+0000 7f7a0effd640 10 client.75028.objecter _finish_command 1 = system:0 2024-09-17T16:46:41.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.518+0000 7f7a0effd640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 4 2024-09-17T16:46:41.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.518+0000 7f7a0effd640 15 client.75028.objecter _session_command_op_remove 4 1 2024-09-17T16:46:41.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 client.75028.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 15 client.75028.objecter _session_command_op_assign -1 2 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter _get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter _get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 15 client.75028.objecter _session_command_op_remove -1 2 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter get_session s=0x7f79fc045640 osd=4 3 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 15 client.75028.objecter _session_command_op_assign 4 2 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 4 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 client.75028.objecter _send_command 2 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 --> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a180a7270 con 0x7f79fc045d10 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a0effd640 1 -- 172.21.15.105:0/2522313163 <== osd.4 v2:172.21.15.160:6816/639753971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a180a7270 con 0x7f79fc045d10 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a0effd640 10 client.75028.objecter ms_dispatch 0x7f7a18000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a0effd640 10 client.75028.objecter _finish_command 2 = system:0 2024-09-17T16:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a0effd640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 4 2024-09-17T16:46:41.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a0effd640 15 client.75028.objecter _session_command_op_remove 4 2 2024-09-17T16:46:41.546 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:41.546 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:41.546 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:41.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 librados: watch_flush enter 2024-09-17T16:46:41.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 librados: watch_flush exit 2024-09-17T16:46:41.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 client.75028.objecter close_session for osd.4 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f79fc045d10 msgr2=0x7f79fc0481b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6816/639753971,v1:172.21.15.160:6817/639753971] conn(0x7f79fc045d10 0x7f79fc0481b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter put_session s=0x7f79fc045640 osd=4 1 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 client.75028.objecter shutdown clearing up homeless session... 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 client.75028.objecter successfully canceled tick 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f79fc03e550 msgr2=0x7f79fc040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f79fc03e550 0x7f79fc040a10 secure :-1 s=READY pgs=3688 cs=0 l=1 rev1=1 crypto rx=0x7f7a0801bed0 tx=0x7f7a080a6000 comp rx=0 tx=0).stop 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 10 monclient: shutdown 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7a180a77b0 msgr2=0x7f7a1813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 --2- 172.21.15.105:0/2522313163 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f7a180a77b0 0x7f7a1813df00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d4c0 tx=0x7f7a0000d980 comp rx=0 tx=0).stop 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a16d9d640 1 -- 172.21.15.105:0/2522313163 reap_dead start 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 shutdown_connections 2024-09-17T16:46:41.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 >> 172.21.15.105:0/2522313163 conn(0x7f7a180a8b30 msgr2=0x7f7a180a2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 shutdown_connections 2024-09-17T16:46:41.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 -- 172.21.15.105:0/2522313163 wait complete. 2024-09-17T16:46:41.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.530+0000 7f7a1cebc640 1 librados: shutdown 2024-09-17T16:46:41.554 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.658+0000 7f2b7fb3b640 10 monclient: get_monmap_and_config 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.658+0000 7f2b7fb3b640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.658+0000 7f2b7fb3b640 10 monclient: monmap: 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:41.663646+0000 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:41.663646+0000 2024-09-17T16:46:41.667 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:41.668 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 Processor -- start 2024-09-17T16:46:41.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 -- start start 2024-09-17T16:46:41.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient: init 2024-09-17T16:46:41.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient: _add_conns ranks=[4,6,9,3,8,10,7,5,0,2,1] 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 0x7f2b781375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-c con 0x7f2b780a4640 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-d con 0x7f2b78137b30 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b7806c110 0x7f2b7806c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-a con 0x7f2b7806c110 addr v2:172.21.15.105:3300/0 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2b780a05f0 con 0x7f2b7806c110 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2b780a4de0 con 0x7f2b780a4640 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f2b780a4a20 con 0x7f2b78137b30 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7fb3b640 10 monclient(hunting): authenticate will time out at 2084.929493s 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b7806c110 0x7f2b7806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 0x7f2b781375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:45742/0 (socket says 172.21.15.105:45742) 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2b7806c110 0x7f2b7806c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37818/0 (socket says 172.21.15.105:37818) 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 1 -- 172.21.15.105:0/2050615409 learned_addr learned my addr 172.21.15.105:0/2050615409 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): get_auth_request con 0x7f2b78137b30 auth_method 0 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): get_auth_request con 0x7f2b7806c110 auth_method 0 2024-09-17T16:46:41.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): get_auth_request con 0x7f2b780a4640 auth_method 0 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient(hunting): handle_auth_done global_id 104310 payload 274 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 -- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 msgr2=0x7f2b78137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 -- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 msgr2=0x7f2b781375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 0x7f2b781375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 monclient: found mon.noname-a 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7e0d8640 1 -- 172.21.15.105:0/2050615409 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780a0350 con 0x7f2b7806c110 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d0d6640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.662+0000 7f2b7d8d7640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 0x7f2b781375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7e0d8640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b7806c110 0x7f2b7806c580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000f370 tx=0x7f2b70020c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70c3f850e0e23fc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 1 -- 172.21.15.105:0/2050615409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b7000d0d0 con 0x7f2b7806c110 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: dump: 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.677 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.678 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 1 -- 172.21.15.105:0/2050615409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7000d290 con 0x7f2b7806c110 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 5 monclient: authenticate success, global_id 104310 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 1 -- 172.21.15.105:0/2050615409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b7000d480 con 0x7f2b7806c110 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 4 set_mon_vals no callback set 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7c8d5640 10 monclient: dump: 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.680 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.681 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: shutdown 2024-09-17T16:46:41.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b7806c110 msgr2=0x7f2b7806c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b7806c110 0x7f2b7806c580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000f370 tx=0x7f2b70020c30 comp rx=0 tx=0).stop 2024-09-17T16:46:41.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2050615409 shutdown_connections 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b7806c110 0x7f2b7806c580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f2b78137b30 0x7f2b78137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2050615409 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b780a4640 0x7f2b781375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2050615409 >> 172.21.15.105:0/2050615409 conn(0x7f2b78068dd0 msgr2=0x7f2b780691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2050615409 shutdown_connections 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2050615409 wait complete. 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: monmap: 2024-09-17T16:46:41.687 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.688 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 librados: starting msgr at 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 librados: starting objecter 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 Processor -- start 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- start start 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 librados: setting wanted keys 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 librados: calling monclient init 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: init 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _add_conns ranks=[4,5,1,7,0,6,2,3,8] 2024-09-17T16:46:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-c con 0x7f2b7806c110 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b780a4640 0x7f2b7813dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-h con 0x7f2b780a4640 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b78137b30 0x7f2b78072b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): picked mon.noname-f con 0x7f2b78137b30 addr [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7813efb0 con 0x7f2b78137b30 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f2b7806ced0 con 0x7f2b7806c110 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f2b7813f130 con 0x7f2b780a4640 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient(hunting): authenticate will time out at 2084.934738s 2024-09-17T16:46:41.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:42868/0 (socket says 172.21.15.105:42868) 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 -- 172.21.15.105:0/2752887018 learned_addr learned my addr 172.21.15.105:0/2752887018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): get_auth_request con 0x7f2b7806c110 auth_method 0 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient(hunting): handle_auth_done global_id 75046 payload 824 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 -- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b780a4640 msgr2=0x7f2b7813dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b780a4640 0x7f2b7813dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 -- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b78137b30 msgr2=0x7f2b78072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b78137b30 0x7f2b78072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 monclient: found mon.noname-c 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 -- 172.21.15.105:0/2752887018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7813a750 con 0x7f2b7806c110 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7d8d7640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ec10 tx=0x7f2b7400c520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 client.?.objecter ms_handle_connect 0x7f2b7806c110 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b74017070 con 0x7f2b7806c110 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: dump: 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.692 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.693 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7400cc30 con 0x7f2b7806c110 2024-09-17T16:46:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 5 monclient: authenticate success, global_id 75046 2024-09-17T16:46:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _renew_subs 2024-09-17T16:46:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7813aa30 con 0x7f2b7806c110 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b7400cde0 con 0x7f2b7806c110 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6e7fc640 10 monclient: dump: 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.696 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b6ffff640 4 set_mon_vals no callback set 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 client.75046.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _renew_subs 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b780695e0 con 0x7f2b7806c110 2024-09-17T16:46:41.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.666+0000 7f2b7fb3b640 1 librados: init done 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7fb3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7fb3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7fb3b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b781375f0 con 0x7f2b7806c110 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f2b7401a070 con 0x7f2b7806c110 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 0x7f2b5c040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f2b7400f040 con 0x7f2b7806c110 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 10 client.75046.objecter ms_dispatch 0x7f2b78000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 3 client.75046.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 3 client.75046.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 20 client.75046.objecter dump_active .. 0 homeless 2024-09-17T16:46:41.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7d0d6640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 0x7f2b5c040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7d0d6640 10 monclient: get_auth_request con 0x7f2b5c03e550 auth_method 0 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b6e7fc640 10 client.75046.objecter ms_handle_connect 0x7f2b5c03e550 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.670+0000 7f2b7d0d6640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 0x7f2b5c040a10 secure :-1 s=READY pgs=3689 cs=0 l=1 rev1=1 crypto rx=0x7f2b78138d40 tx=0x7f2b68009470 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.674+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b74021070 con 0x7f2b7806c110 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.674+0000 7f2b6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:41.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.674+0000 7f2b6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:41.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.834+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f2b78139bc0 con 0x7f2b5c03e550 2024-09-17T16:46:41.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.834+0000 7f2b6e7fc640 1 -- 172.21.15.105:0/2752887018 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b78139bc0 con 0x7f2b5c03e550 2024-09-17T16:46:41.841 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 10 librados: watch_flush enter 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 10 librados: watch_flush exit 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 20 client.75046.objecter shutdown clearing up homeless session... 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 10 client.75046.objecter successfully canceled tick 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 msgr2=0x7f2b5c040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 0x7f2b5c040a10 secure :-1 s=READY pgs=3689 cs=0 l=1 rev1=1 crypto rx=0x7f2b78138d40 tx=0x7f2b68009470 comp rx=0 tx=0).stop 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 10 monclient: shutdown 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:41.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 msgr2=0x7f2b7813da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400ec10 tx=0x7f2b7400c520 comp rx=0 tx=0).stop 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 shutdown_connections 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f2b5c03e550 0x7f2b5c040a10 unknown :-1 s=CLOSED pgs=3689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b78137b30 0x7f2b78072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f2b780a4640 0x7f2b7813dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 --2- 172.21.15.105:0/2752887018 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f2b7806c110 0x7f2b7813da60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 >> 172.21.15.105:0/2752887018 conn(0x7f2b78068dd0 msgr2=0x7f2b780a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 shutdown_connections 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 -- 172.21.15.105:0/2752887018 wait complete. 2024-09-17T16:46:41.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.838+0000 7f2b7fb3b640 1 librados: shutdown 2024-09-17T16:46:41.859 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T16:46:41.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: get_monmap_and_config 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: build_initial_monmap 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: monmap: 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:41.977313+0000 2024-09-17T16:46:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:41.977313+0000 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:41.983 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:41.983 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:41.983 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 Processor -- start 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 -- start start 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: init 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient: _add_conns ranks=[0,8,1,2,3,5,10,7,4,9,6] 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc7a80a35a0 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-e con 0x7fc7a80a1840 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc7a80a4090 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a80a0ed0 con 0x7fc7a80a35a0 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a80a0180 con 0x7fc7a80a4090 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc7a807d3b0 con 0x7fc7a80a1840 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7ad609640 10 monclient(hunting): authenticate will time out at 2085.243007s 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41302/0 (socket says 172.21.15.105:41302) 2024-09-17T16:46:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a6ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc7a80a35a0 0x7fc7a80a3980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44360/0 (socket says 172.21.15.105:44360) 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 -- 172.21.15.105:0/2715371002 learned_addr learned my addr 172.21.15.105:0/2715371002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): get_auth_request con 0x7fc7a80a1840 auth_method 0 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_done global_id 55250 payload 274 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a5ffb640 1 -- 172.21.15.105:0/2715371002 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787629455 0 0) 0x7fc7a80a0ed0 con 0x7fc7a80a35a0 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 -- 172.21.15.105:0/2715371002 >> v1:172.21.15.160:6789/0 conn(0x7fc7a80a4090 legacy=0x7fc7a813dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 -- 172.21.15.105:0/2715371002 >> v1:172.21.15.105:6789/0 conn(0x7fc7a80a35a0 legacy=0x7fc7a80a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 monclient: found mon.noname-e 2024-09-17T16:46:41.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:41.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:41.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a67fc640 1 -- 172.21.15.105:0/2715371002 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a80a0bf0 con 0x7fc7a80a1840 2024-09-17T16:46:41.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.974+0000 7fc7a5ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a67fc640 1 --2- 172.21.15.105:0/2715371002 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00bb40 tx=0x7fc79c01eb10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5585d56634e48174 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 1 -- 172.21.15.105:0/2715371002 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c0b4070 con 0x7fc7a80a1840 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: dump: 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.997 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:41.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 1 -- 172.21.15.105:0/2715371002 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c0a7b50 con 0x7fc7a80a1840 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 5 monclient: authenticate success, global_id 55250 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 4 set_mon_vals no callback set 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 1 -- 172.21.15.105:0/2715371002 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c0a7d00 con 0x7fc7a80a1840 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7a5ffb640 10 monclient: dump: 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:41.999 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.000 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 10 monclient: shutdown 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 -- 172.21.15.105:0/2715371002 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 msgr2=0x7fc7a813b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 --2- 172.21.15.105:0/2715371002 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00bb40 tx=0x7fc79c01eb10 comp rx=0 tx=0).stop 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 -- 172.21.15.105:0/2715371002 shutdown_connections 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 --2- 172.21.15.105:0/2715371002 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a1840 0x7fc7a813b7b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 -- 172.21.15.105:0/2715371002 >> 172.21.15.105:0/2715371002 conn(0x7fc7a80689d0 msgr2=0x7fc7a8068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 -- 172.21.15.105:0/2715371002 shutdown_connections 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.978+0000 7fc7ad609640 1 -- 172.21.15.105:0/2715371002 wait complete. 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: build_initial_monmap 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: monmap: 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:42.001 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:42.002 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 librados: starting msgr at 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 librados: starting objecter 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 Processor -- start 2024-09-17T16:46:42.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- start start 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 librados: setting wanted keys 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 librados: calling monclient init 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: init 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _add_conns ranks=[2,8,5,1,6,7,0,3,4] 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 0x7fc7a813de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-b con 0x7fc7a80a1840 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a35a0 0x7fc7a813e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-e con 0x7fc7a80a35a0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc7a80a4090 0x7fc7a81c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): picked mon.noname-h con 0x7fc7a80a4090 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a809e5a0 con 0x7fc7a80a1840 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fc7a809e420 con 0x7fc7a80a4090 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7fc7a809e720 con 0x7fc7a80a35a0 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient(hunting): authenticate will time out at 2085.249187s 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 0x7fc7a813de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 0x7fc7a813de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:38648/0 (socket says 172.21.15.105:38648) 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a35a0 0x7fc7a813e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 1 -- 172.21.15.105:0/1194050220 learned_addr learned my addr 172.21.15.105:0/1194050220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): get_auth_request con 0x7fc7a80a1840 auth_method 0 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): get_auth_request con 0x7fc7a80a35a0 auth_method 0 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient(hunting): handle_auth_done global_id 55259 payload 824 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a77fe640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc7a80a4090 0x7fc7a81c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 -- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc7a80a4090 msgr2=0x7fc7a81c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fc7a80a4090 0x7fc7a81c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 -- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 msgr2=0x7fc7a813de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 0x7fc7a813de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient: _finish_hunting 0 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 monclient: found mon.noname-e 2024-09-17T16:46:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.105:3304/0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a81345a0 con 0x7fc7a80a35a0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7a80a1840 0x7fc7a813de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a67fc640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a35a0 0x7fc7a813e3b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae000 tx=0x7fc79c01b8b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 client.?.objecter ms_handle_connect 0x7fc7a80a35a0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c0bf070 con 0x7fc7a80a35a0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: dump: 2024-09-17T16:46:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: _finish_auth 0 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 5 monclient: authenticate success, global_id 55259 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c004460 con 0x7fc7a80a35a0 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:42.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _renew_subs 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc79c0b4040 con 0x7fc7a80a35a0 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a813eda0 con 0x7fc7a80a35a0 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a57fa640 4 set_mon_vals no callback set 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 client.55259.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: dump: 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _renew_subs 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a8136060 con 0x7fc7a80a35a0 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 librados: init done 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 client.55259.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 15 client.55259.objecter _session_command_op_assign -1 1 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 20 client.55259.objecter _get_session osd=-1 returning homeless 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 client.55259.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.105:3304/0 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7a80a2fe0 con 0x7fc7a80a35a0 2024-09-17T16:46:42.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fc79c003620 con 0x7fc7a80a35a0 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc79403e550 0x7fc794040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7fc79c0ec180 con 0x7fc7a80a35a0 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 client.55259.objecter ms_dispatch 0x7fc7a8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 3 client.55259.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 3 client.55259.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 client.55259.objecter checking command 1 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7fc794045d10 0x7fc7940481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter _get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 15 client.55259.objecter _session_command_op_remove -1 1 2024-09-17T16:46:42.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter _get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 15 client.55259.objecter _session_command_op_assign 5 1 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 4 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 client.55259.objecter _send_command 1 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc79c0b1ec0 con 0x7fc794045d10 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 20 client.55259.objecter dump_active .. 0 homeless 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== mon.8 v2:172.21.15.105:3304/0 6 ==== mon_get_version_reply(handle=1 version=707) ==== 24+0+0 (secure 0 0 0) 0x7fc79c0f3050 con 0x7fc7a80a35a0 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc78f7fe640 10 monclient: handle_get_version_reply finishing 1 version 707 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc79403e550 0x7fc794040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a6ffd640 10 monclient: get_auth_request con 0x7fc79403e550 auth_method 0 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a77fe640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7fc794045d10 0x7fc7940481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.982+0000 7fc7a77fe640 10 monclient: get_auth_request con 0x7fc794045d10 auth_method 0 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 10 client.55259.objecter ms_handle_connect 0x7fc794045d10 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc7a77fe640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7fc794045d10 0x7fc7940481b0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 10 client.55259.objecter ms_handle_connect 0x7fc79403e550 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc7a6ffd640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc79403e550 0x7fc794040a10 secure :-1 s=READY pgs=3690 cs=0 l=1 rev1=1 crypto rx=0x7fc798000f70 tx=0x7fc798007240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== osd.5 v2:172.21.15.160:6800/2641095577 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc79c0b1ec0 con 0x7fc794045d10 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 10 client.55259.objecter ms_dispatch 0x7fc7a8000ba0 command_reply(tid 1: 0 ) 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 10 client.55259.objecter _finish_command 1 = system:0 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 4 2024-09-17T16:46:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.986+0000 7fc78f7fe640 15 client.55259.objecter _session_command_op_remove 5 1 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 10 client.55259.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 15 client.55259.objecter _session_command_op_assign -1 2 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 20 client.55259.objecter _get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 20 client.55259.objecter _get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 15 client.55259.objecter _session_command_op_remove -1 2 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 20 client.55259.objecter get_session s=0x7fc794045640 osd=5 3 2024-09-17T16:46:42.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 15 client.55259.objecter _session_command_op_assign 5 2 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 4 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 10 client.55259.objecter _send_command 2 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:41.998+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 --> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc7a813e8f0 con 0x7fc794045d10 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc78f7fe640 1 -- 172.21.15.105:0/1194050220 <== osd.5 v2:172.21.15.160:6800/2641095577 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc7a813e8f0 con 0x7fc794045d10 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc78f7fe640 10 client.55259.objecter ms_dispatch 0x7fc7a8000ba0 command_reply(tid 2: 0 ) 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc78f7fe640 10 client.55259.objecter _finish_command 2 = system:0 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc78f7fe640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 4 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc78f7fe640 15 client.55259.objecter _session_command_op_remove 5 2 2024-09-17T16:46:42.018 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-17T16:46:42.019 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T16:46:42.019 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-17T16:46:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 10 librados: watch_flush enter 2024-09-17T16:46:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 10 librados: watch_flush exit 2024-09-17T16:46:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 10 client.55259.objecter close_session for osd.5 2024-09-17T16:46:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7fc794045d10 msgr2=0x7fc7940481b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6800/2641095577,v1:172.21.15.160:6801/2641095577] conn(0x7fc794045d10 0x7fc7940481b0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 20 client.55259.objecter put_session s=0x7fc794045640 osd=5 1 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 20 client.55259.objecter shutdown clearing up homeless session... 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 10 client.55259.objecter successfully canceled tick 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc79403e550 msgr2=0x7fc794040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fc79403e550 0x7fc794040a10 secure :-1 s=READY pgs=3690 cs=0 l=1 rev1=1 crypto rx=0x7fc798000f70 tx=0x7fc798007240 comp rx=0 tx=0).stop 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 10 monclient: shutdown 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a35a0 msgr2=0x7fc7a813e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 --2- 172.21.15.105:0/1194050220 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7fc7a80a35a0 0x7fc7a813e3b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0ae000 tx=0x7fc79c01b8b0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7a77fe640 1 -- 172.21.15.105:0/1194050220 reap_dead start 2024-09-17T16:46:42.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 shutdown_connections 2024-09-17T16:46:42.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 >> 172.21.15.105:0/1194050220 conn(0x7fc7a80689d0 msgr2=0x7fc7a81318d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 shutdown_connections 2024-09-17T16:46:42.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 -- 172.21.15.105:0/1194050220 wait complete. 2024-09-17T16:46:42.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.002+0000 7fc7ad609640 1 librados: shutdown 2024-09-17T16:46:42.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T16:46:42.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: get_monmap_and_config 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: build_initial_monmap 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: monmap: 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:42.128413+0000 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:42.128413+0000 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:42.133 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 Processor -- start 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 -- start start 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: init 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient: _add_conns ranks=[10,3,1,4,0,7,8,9,6,5,2] 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): picked mon.noname-f con 0x7f870c0a5860 addr v2:172.21.15.160:3300/0 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 0x7f870c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): picked mon.noname-g con 0x7f870c0a18f0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f870c0a4070 addr v1:172.21.15.160:6789/0 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870c0a1d70 con 0x7f870c0a4070 2024-09-17T16:46:42.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f870c0a66c0 con 0x7f870c0a5860 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f870c0a33d0 con 0x7f870c0a18f0 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f8713303640 10 monclient(hunting): authenticate will time out at 2085.393997s 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 0x7f870c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.105:54536/0 (socket says 172.21.15.105:54536) 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f87118a0640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f870c0a4070 0x7f870c137610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.105:34398/0 (socket says 172.21.15.105:34398) 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 1 -- 172.21.15.105:0/2980251134 learned_addr learned my addr 172.21.15.105:0/2980251134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): get_auth_request con 0x7f870c0a5860 auth_method 0 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): get_auth_request con 0x7f870c0a18f0 auth_method 0 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient(hunting): handle_auth_done global_id 55164 payload 274 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 -- 172.21.15.105:0/2980251134 >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 msgr2=0x7f870c0a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 --2- 172.21.15.105:0/2980251134 >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 -- 172.21.15.105:0/2980251134 >> v1:172.21.15.160:6789/0 conn(0x7f870c0a4070 legacy=0x7f870c137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient: _finish_hunting 0 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 monclient: found mon.noname-g 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.160:3301/0 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871089e640 1 -- 172.21.15.105:0/2980251134 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c0a36b0 con 0x7f870c0a18f0 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.126+0000 7f871109f640 1 --2- 172.21.15.105:0/2980251134 >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:42.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871089e640 1 --2- 172.21.15.105:0/2980251134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 0x7f870c0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0097c0 tx=0x7f86fc01c3f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2eb05069b820c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 1 -- 172.21.15.105:0/2980251134 <== mon.3 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86fc0b0070 con 0x7f870c0a18f0 2024-09-17T16:46:42.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: dump: 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.145 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: _finish_auth 0 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: _check_auth_tickets 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 1 -- 172.21.15.105:0/2980251134 <== mon.3 v2:172.21.15.160:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86fc0aa810 con 0x7f870c0a18f0 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 1 -- 172.21.15.105:0/2980251134 <== mon.3 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86fc0aaa40 con 0x7f870c0a18f0 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-17T16:46:42.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8703fff640 10 monclient: dump: 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.147 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 5 monclient: authenticate success, global_id 55164 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 4 set_mon_vals no callback set 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: shutdown 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- 172.21.15.105:0/2980251134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 msgr2=0x7f870c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- 172.21.15.105:0/2980251134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 0x7f870c0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0097c0 tx=0x7f86fc01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- 172.21.15.105:0/2980251134 shutdown_connections 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- 172.21.15.105:0/2980251134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f870c0a18f0 0x7f870c0a6180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- 172.21.15.105:0/2980251134 >> v2:172.21.15.160:3300/0 conn(0x7f870c0a5860 0x7f870c0a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- 172.21.15.105:0/2980251134 >> 172.21.15.105:0/2980251134 conn(0x7f870c0aa450 msgr2=0x7f870c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- 172.21.15.105:0/2980251134 shutdown_connections 2024-09-17T16:46:42.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- 172.21.15.105:0/2980251134 wait complete. 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: build_initial_monmap 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: monmap: 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:42.149 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 librados: starting msgr at 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 librados: starting objecter 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 Processor -- start 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- start start 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 librados: setting wanted keys 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 librados: calling monclient init 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: init 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient: _add_conns ranks=[8,7,2,3,6,0,4,5,1] 2024-09-17T16:46:42.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 0x7f870c1204a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): picked mon.noname-e con 0x7f870c0a18f0 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 0x7f870c1209e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): picked mon.noname-i con 0x7f870c0a4070 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 0x7f870c120f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): picked mon.noname-b con 0x7f870c0a5860 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f870c142490 con 0x7f870c0a5860 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f870c142310 con 0x7f870c0a4070 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f870c142610 con 0x7f870c0a18f0 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f8713303640 10 monclient(hunting): authenticate will time out at 2085.399723s 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 0x7f870c120f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 0x7f870c1204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 0x7f870c1204a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3304/0 says I am v2:172.21.15.105:41316/0 (socket says 172.21.15.105:41316) 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 1 -- 172.21.15.105:0/1559704248 learned_addr learned my addr 172.21.15.105:0/1559704248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): get_auth_request con 0x7f870c0a5860 auth_method 0 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): get_auth_request con 0x7f870c0a18f0 auth_method 0 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f87118a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.130+0000 7f871109f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871089e640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 0x7f870c1209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 10 monclient(hunting): handle_auth_done global_id 84737 payload 824 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 -- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 msgr2=0x7f870c1204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 0x7f870c1204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 -- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 msgr2=0x7f870c1209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 0x7f870c1209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 10 monclient: _finish_hunting 0 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 monclient: found mon.noname-b 2024-09-17T16:46:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 -- 172.21.15.105:0/1559704248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c0a60d0 con 0x7f870c0a5860 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871089e640 10 monclient: get_auth_request con 0x7f870c0a4070 auth_method 0 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871089e640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 0x7f870c1209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87118a0640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 0x7f870c120f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f870400c770 tx=0x7f870400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 client.?.objecter ms_handle_connect 0x7f870c0a5860 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8704017070 con 0x7f870c0a5860 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: dump: 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: _finish_auth 0 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: _check_auth_tickets 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704005db0 con 0x7f870c0a5860 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 5 monclient: authenticate success, global_id 84737 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _renew_subs 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c122d60 con 0x7f870c0a5860 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8704007070 con 0x7f870c0a5860 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f87037fe640 4 set_mon_vals no callback set 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 monclient: dump: 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 client.84737.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _renew_subs 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c06f0d0 con 0x7f870c0a5860 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 1 librados: init done 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.105:3301/0 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c137610 con 0x7f870c0a5860 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f870400a640 con 0x7f870c0a5860 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 0x7f86f0040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f870404f010 con 0x7f870c0a5860 2024-09-17T16:46:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 client.84737.objecter ms_dispatch 0x7f870c000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 3 client.84737.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 3 client.84737.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 20 client.84737.objecter dump_active .. 0 homeless 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871109f640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 0x7f86f0040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871109f640 10 monclient: get_auth_request con 0x7f86f003e550 auth_method 0 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f8701ffb640 10 client.84737.objecter ms_handle_connect 0x7f86f003e550 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.134+0000 7f871109f640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 0x7f86f0040a10 secure :-1 s=READY pgs=3691 cs=0 l=1 rev1=1 crypto rx=0x7f8708009630 tx=0x7f8708009240 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.138+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f870c0a60d0 con 0x7f870c0a5860 2024-09-17T16:46:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.138+0000 7f8701ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.138+0000 7f8701ffb640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:42.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.302+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f870c122200 con 0x7f86f003e550 2024-09-17T16:46:42.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.302+0000 7f8701ffb640 1 -- 172.21.15.105:0/1559704248 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f870c122200 con 0x7f86f003e550 2024-09-17T16:46:42.310 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 10 librados: watch_flush enter 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 10 librados: watch_flush exit 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 20 client.84737.objecter shutdown clearing up homeless session... 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 10 client.84737.objecter successfully canceled tick 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 msgr2=0x7f86f0040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 0x7f86f0040a10 secure :-1 s=READY pgs=3691 cs=0 l=1 rev1=1 crypto rx=0x7f8708009630 tx=0x7f8708009240 comp rx=0 tx=0).stop 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 10 monclient: shutdown 2024-09-17T16:46:42.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 msgr2=0x7f870c120f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 0x7f870c120f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f870400c770 tx=0x7f870400cc30 comp rx=0 tx=0).stop 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 shutdown_connections 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f86f003e550 0x7f86f0040a10 unknown :-1 s=CLOSED pgs=3691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f870c0a5860 0x7f870c120f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f870c0a4070 0x7f870c1209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 --2- 172.21.15.105:0/1559704248 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f870c0a18f0 0x7f870c1204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 >> 172.21.15.105:0/1559704248 conn(0x7f870c0aa450 msgr2=0x7f870c06d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 shutdown_connections 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 -- 172.21.15.105:0/1559704248 wait complete. 2024-09-17T16:46:42.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.306+0000 7f8713303640 1 librados: shutdown 2024-09-17T16:46:42.328 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: get_monmap_and_config 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: build_initial_monmap 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: monmap: 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:46:42.434385+0000 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:46:42.434385+0000 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:46:42.438 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:46:42.439 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 1 Processor -- start 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 1 -- start start 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: init 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 10 monclient: _add_conns ranks=[4,0,5,6,3,7,10,9,1,2,8] 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.430+0000 7f23cd70e640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 0x7f23c80a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-c con 0x7f23c80a68e0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:46:42.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f23c80a4910 addr v1:172.21.15.105:6789/0 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 0x7f23c8139db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-h con 0x7f23c80a7200 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c806ebc0 con 0x7f23c80a4910 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7f23c80a3fc0 con 0x7f23c80a68e0 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f23c80a63c0 con 0x7f23c80a7200 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient(hunting): authenticate will time out at 2085.700099s 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23bffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f23c80a4910 0x7f23c81396a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:44370/0 (socket says 172.21.15.105:44370) 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23bffff640 1 -- 172.21.15.105:0/2295365225 learned_addr learned my addr 172.21.15.105:0/2295365225 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 0x7f23c80a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): get_auth_request con 0x7f23c80a68e0 auth_method 0 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 1 -- 172.21.15.105:0/2295365225 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309561431 0 0) 0x7f23c806ebc0 con 0x7f23c80a4910 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c77fe640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 0x7f23c8139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient(hunting): _negotiate 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient(hunting): my global_id is 104328 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 1 -- 172.21.15.105:0/2295365225 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b4003530 con 0x7f23c80a4910 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_done global_id 75055 payload 274 2024-09-17T16:46:42.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 -- 172.21.15.105:0/2295365225 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 msgr2=0x7f23c8139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 0x7f23c8139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 -- 172.21.15.105:0/2295365225 >> v1:172.21.15.105:6789/0 conn(0x7f23c80a4910 legacy=0x7f23c81396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 monclient: found mon.noname-c 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 -- 172.21.15.105:0/2295365225 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c80a3690 con 0x7f23c80a68e0 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c77fe640 10 monclient: get_auth_request con 0x7f23c80a7200 auth_method 0 2024-09-17T16:46:42.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c77fe640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 0x7f23c8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 0x7f23c80a6cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009dc0 tx=0x7f23b801c820 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bb3e9b7fa9ae4042 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 1 -- 172.21.15.105:0/2295365225 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23b80b2070 con 0x7f23c80a68e0 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: dump: 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.450 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: _finish_auth 0 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: _check_auth_tickets 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 5 monclient: authenticate success, global_id 75055 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 1 -- 172.21.15.105:0/2295365225 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80a5630 con 0x7f23c80a68e0 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 1 -- 172.21.15.105:0/2295365225 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23b80a5c50 con 0x7f23c80a68e0 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient: get_monmap_and_config success 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 4 set_mon_vals no callback set 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23c67fc640 10 monclient: dump: 2024-09-17T16:46:42.452 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.453 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 10 monclient: shutdown 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- 172.21.15.105:0/2295365225 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 msgr2=0x7f23c80a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 0x7f23c80a6cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009dc0 tx=0x7f23b801c820 comp rx=0 tx=0).stop 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- 172.21.15.105:0/2295365225 shutdown_connections 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a7200 0x7f23c8139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 --2- 172.21.15.105:0/2295365225 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7f23c80a68e0 0x7f23c80a6cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- 172.21.15.105:0/2295365225 >> 172.21.15.105:0/2295365225 conn(0x7f23c8068ba0 msgr2=0x7f23c8068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- 172.21.15.105:0/2295365225 shutdown_connections 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.434+0000 7f23cd70e640 1 -- 172.21.15.105:0/2295365225 wait complete. 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: build_initial_monmap 2024-09-17T16:46:42.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: monmap: 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:46:42.455 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 librados: starting msgr at 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 librados: starting objecter 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 Processor -- start 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- start start 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 librados: setting wanted keys 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 librados: calling monclient init 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: init 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _reopen_session rank -1 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _add_conns ranks=[6,5,8,4,7,0,2,1,3] 2024-09-17T16:46:42.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-d con 0x7f23c80a4910 addr [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 0x7f23c813e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-h con 0x7f23c80a68e0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 0x7f23c813e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): picked mon.noname-e con 0x7f23c80a7200 addr [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f23c813a2f0 con 0x7f23c80a68e0 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_getmap magic: 0 -- 0x7f23c80a7610 con 0x7f23c80a4910 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): start opening mon connection 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- --> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] -- mon_getmap magic: 0 -- 0x7f23c813a470 con 0x7f23c80a7200 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): _renew_subs 2024-09-17T16:46:42.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient(hunting): authenticate will time out at 2085.705018s 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 1 --2- >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 0x7f23c813e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 --2- >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3303/0 says I am v2:172.21.15.105:45768/0 (socket says 172.21.15.105:45768) 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 -- 172.21.15.105:0/267542042 learned_addr learned my addr 172.21.15.105:0/267542042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): get_auth_request con 0x7f23c80a7200 auth_method 0 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): get_auth_request con 0x7f23c80a4910 auth_method 0 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:46:42.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bffff640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 0x7f23c813e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient(hunting): handle_auth_done global_id 94515 payload 824 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 -- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 msgr2=0x7f23c813e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 0x7f23c813e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 -- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 msgr2=0x7f23c813e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 0x7f23c813e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient: _finish_hunting 0 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 monclient: found mon.noname-d 2024-09-17T16:46:42.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.105:3303/0 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 -- 172.21.15.105:0/267542042 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c813f900 con 0x7f23c80a4910 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c77fe640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 0x7f23c813e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bffff640 10 monclient: get_auth_request con 0x7f23c80a68e0 auth_method 0 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bffff640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 0x7f23c813e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c6ffd640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23b801aed0 tx=0x7f23b80195c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 client.?.objecter ms_handle_connect 0x7f23c80a4910 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 client.?.objecter resend_mon_ops 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23b80b2070 con 0x7f23c80a4910 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: dump: 2024-09-17T16:46:42.460 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.461 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: _finish_auth 0 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: _check_auth_tickets 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80bc070 con 0x7f23c80a4910 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 5 monclient: authenticate success, global_id 94515 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _renew_subs 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c8123ae0 con 0x7f23c80a4910 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f23b80a5e60 con 0x7f23c80a4910 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-17T16:46:42.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23c5ffb640 4 set_mon_vals no callback set 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 monclient: dump: 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:46:42.463 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 client.94515.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _renew_subs 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c81236c0 con 0x7f23c80a4910 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 librados: init done 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.105:3303/0 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 --> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c806dd90 con 0x7f23c80a4910 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f23b80a59f0 con 0x7f23c80a4910 2024-09-17T16:46:42.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 0x7f23a8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 5 ==== osd_map(707..707 src has 542..707) ==== 5529+0+0 (secure 0 0 0) 0x7f23b80ea550 con 0x7f23c80a4910 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 10 client.94515.objecter ms_dispatch 0x7f23c8000ba0 osd_map(707..707 src has 542..707) 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 3 client.94515.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 3 client.94515.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bf7fe640 20 client.94515.objecter dump_active .. 0 homeless 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bffff640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 0x7f23a8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.438+0000 7f23bffff640 10 monclient: get_auth_request con 0x7f23a803e550 auth_method 0 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.442+0000 7f23bf7fe640 10 client.94515.objecter ms_handle_connect 0x7f23a803e550 2024-09-17T16:46:42.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.442+0000 7f23bffff640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 0x7f23a8040a10 secure :-1 s=READY pgs=3692 cs=0 l=1 rev1=1 crypto rx=0x7f23b0001230 tx=0x7f23b0009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:46:42.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.442+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mon.6 v2:172.21.15.105:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23c8123ae0 con 0x7f23c80a4910 2024-09-17T16:46:42.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.442+0000 7f23bf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:46:42.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.442+0000 7f23bf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:46:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.598+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23c80a1e50 con 0x7f23a803e550 2024-09-17T16:46:42.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23bf7fe640 1 -- 172.21.15.105:0/267542042 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+114046 (secure 0 0 0) 0x7f23c80a1e50 con 0x7f23a803e550 2024-09-17T16:46:42.612 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:46:42.613 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:46:42.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 10 librados: watch_flush enter 2024-09-17T16:46:42.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 10 librados: watch_flush exit 2024-09-17T16:46:42.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 20 client.94515.objecter shutdown clearing up homeless session... 2024-09-17T16:46:42.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 10 client.94515.objecter successfully canceled tick 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 msgr2=0x7f23a8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 0x7f23a8040a10 secure :-1 s=READY pgs=3692 cs=0 l=1 rev1=1 crypto rx=0x7f23b0001230 tx=0x7f23b0009040 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 10 monclient: shutdown 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 msgr2=0x7f23c813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23b801aed0 tx=0x7f23b80195c0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 shutdown_connections 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f23a803e550 0x7f23a8040a10 unknown :-1 s=CLOSED pgs=3692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] conn(0x7f23c80a7200 0x7f23c813e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f23c80a68e0 0x7f23c813e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 --2- 172.21.15.105:0/267542042 >> [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] conn(0x7f23c80a4910 0x7f23c813df00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 >> 172.21.15.105:0/267542042 conn(0x7f23c8068ba0 msgr2=0x7f23c809fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:46:42.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 shutdown_connections 2024-09-17T16:46:42.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.606+0000 7f23cd70e640 1 -- 172.21.15.105:0/267542042 wait complete. 2024-09-17T16:46:42.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:46:42.610+0000 7f23cd70e640 1 librados: shutdown 2024-09-17T16:46:42.630 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":1267,"stamp":"2024-09-17T16:46:42.195113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":61,"num_write":110,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470648,"kb_used_data":62808,"kb_used_omap":9,"kb_used_meta":407734,"kb_avail":565760392,"statfs":{"total":579820584960,"available":579338641408,"internally_reserved":0,"allocated":64315392,"data_stored":36273584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":417520288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004046"},"pg_stats":[{"pgid":"6.16","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356126+0000","last_change":"2024-09-17T16:29:02.018311+0000","last_active":"2024-09-17T16:46:00.356126+0000","last_peered":"2024-09-17T16:46:00.356126+0000","last_clean":"2024-09-17T16:46:00.356126+0000","last_became_active":"2024-09-17T16:29:02.017756+0000","last_became_peered":"2024-09-17T16:29:02.017756+0000","last_unstale":"2024-09-17T16:46:00.356126+0000","last_undegraded":"2024-09-17T16:46:00.356126+0000","last_fullsized":"2024-09-17T16:46:00.356126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:23:14.467205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360583+0000","last_change":"2024-09-17T16:29:02.054931+0000","last_active":"2024-09-17T16:46:00.360583+0000","last_peered":"2024-09-17T16:46:00.360583+0000","last_clean":"2024-09-17T16:46:00.360583+0000","last_became_active":"2024-09-17T16:29:02.054559+0000","last_became_peered":"2024-09-17T16:29:02.054559+0000","last_unstale":"2024-09-17T16:46:00.360583+0000","last_undegraded":"2024-09-17T16:46:00.360583+0000","last_fullsized":"2024-09-17T16:46:00.360583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:37.538511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239713+0000","last_change":"2024-09-17T16:29:02.079133+0000","last_active":"2024-09-17T16:46:00.239713+0000","last_peered":"2024-09-17T16:46:00.239713+0000","last_clean":"2024-09-17T16:46:00.239713+0000","last_became_active":"2024-09-17T16:29:02.076410+0000","last_became_peered":"2024-09-17T16:29:02.076410+0000","last_unstale":"2024-09-17T16:46:00.239713+0000","last_undegraded":"2024-09-17T16:46:00.239713+0000","last_fullsized":"2024-09-17T16:46:00.239713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:04.096556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243332+0000","last_change":"2024-09-17T16:29:02.123528+0000","last_active":"2024-09-17T16:46:00.243332+0000","last_peered":"2024-09-17T16:46:00.243332+0000","last_clean":"2024-09-17T16:46:00.243332+0000","last_became_active":"2024-09-17T16:29:02.122292+0000","last_became_peered":"2024-09-17T16:29:02.122292+0000","last_unstale":"2024-09-17T16:46:00.243332+0000","last_undegraded":"2024-09-17T16:46:00.243332+0000","last_fullsized":"2024-09-17T16:46:00.243332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:48.555269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360740+0000","last_change":"2024-09-17T16:29:02.064488+0000","last_active":"2024-09-17T16:46:00.360740+0000","last_peered":"2024-09-17T16:46:00.360740+0000","last_clean":"2024-09-17T16:46:00.360740+0000","last_became_active":"2024-09-17T16:29:02.064232+0000","last_became_peered":"2024-09-17T16:29:02.064232+0000","last_unstale":"2024-09-17T16:46:00.360740+0000","last_undegraded":"2024-09-17T16:46:00.360740+0000","last_fullsized":"2024-09-17T16:46:00.360740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:14.884902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354984+0000","last_change":"2024-09-17T16:29:02.121725+0000","last_active":"2024-09-17T16:46:00.354984+0000","last_peered":"2024-09-17T16:46:00.354984+0000","last_clean":"2024-09-17T16:46:00.354984+0000","last_became_active":"2024-09-17T16:29:02.116801+0000","last_became_peered":"2024-09-17T16:29:02.116801+0000","last_unstale":"2024-09-17T16:46:00.354984+0000","last_undegraded":"2024-09-17T16:46:00.354984+0000","last_fullsized":"2024-09-17T16:46:00.354984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:23.905864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242889+0000","last_change":"2024-09-17T16:29:02.131256+0000","last_active":"2024-09-17T16:46:00.242889+0000","last_peered":"2024-09-17T16:46:00.242889+0000","last_clean":"2024-09-17T16:46:00.242889+0000","last_became_active":"2024-09-17T16:29:02.129701+0000","last_became_peered":"2024-09-17T16:29:02.129701+0000","last_unstale":"2024-09-17T16:46:00.242889+0000","last_undegraded":"2024-09-17T16:46:00.242889+0000","last_fullsized":"2024-09-17T16:46:00.242889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:34:10.375884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356246+0000","last_change":"2024-09-17T16:29:02.079372+0000","last_active":"2024-09-17T16:46:00.356246+0000","last_peered":"2024-09-17T16:46:00.356246+0000","last_clean":"2024-09-17T16:46:00.356246+0000","last_became_active":"2024-09-17T16:29:02.078833+0000","last_became_peered":"2024-09-17T16:29:02.078833+0000","last_unstale":"2024-09-17T16:46:00.356246+0000","last_undegraded":"2024-09-17T16:46:00.356246+0000","last_fullsized":"2024-09-17T16:46:00.356246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:50.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360568+0000","last_change":"2024-09-17T16:29:02.185295+0000","last_active":"2024-09-17T16:46:00.360568+0000","last_peered":"2024-09-17T16:46:00.360568+0000","last_clean":"2024-09-17T16:46:00.360568+0000","last_became_active":"2024-09-17T16:29:02.179921+0000","last_became_peered":"2024-09-17T16:29:02.179921+0000","last_unstale":"2024-09-17T16:46:00.360568+0000","last_undegraded":"2024-09-17T16:46:00.360568+0000","last_fullsized":"2024-09-17T16:46:00.360568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:18.876647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.240108+0000","last_change":"2024-09-17T16:29:02.165149+0000","last_active":"2024-09-17T16:46:00.240108+0000","last_peered":"2024-09-17T16:46:00.240108+0000","last_clean":"2024-09-17T16:46:00.240108+0000","last_became_active":"2024-09-17T16:29:02.164714+0000","last_became_peered":"2024-09-17T16:29:02.164714+0000","last_unstale":"2024-09-17T16:46:00.240108+0000","last_undegraded":"2024-09-17T16:46:00.240108+0000","last_fullsized":"2024-09-17T16:46:00.240108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:52:19.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355098+0000","last_change":"2024-09-17T16:29:02.177944+0000","last_active":"2024-09-17T16:46:00.355098+0000","last_peered":"2024-09-17T16:46:00.355098+0000","last_clean":"2024-09-17T16:46:00.355098+0000","last_became_active":"2024-09-17T16:29:02.177674+0000","last_became_peered":"2024-09-17T16:29:02.177674+0000","last_unstale":"2024-09-17T16:46:00.355098+0000","last_undegraded":"2024-09-17T16:46:00.355098+0000","last_fullsized":"2024-09-17T16:46:00.355098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:42.365472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360691+0000","last_change":"2024-09-17T16:29:02.185360+0000","last_active":"2024-09-17T16:46:00.360691+0000","last_peered":"2024-09-17T16:46:00.360691+0000","last_clean":"2024-09-17T16:46:00.360691+0000","last_became_active":"2024-09-17T16:29:02.184301+0000","last_became_peered":"2024-09-17T16:29:02.184301+0000","last_unstale":"2024-09-17T16:46:00.360691+0000","last_undegraded":"2024-09-17T16:46:00.360691+0000","last_fullsized":"2024-09-17T16:46:00.360691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:12.678748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361203+0000","last_change":"2024-09-17T16:29:02.050147+0000","last_active":"2024-09-17T16:46:00.361203+0000","last_peered":"2024-09-17T16:46:00.361203+0000","last_clean":"2024-09-17T16:46:00.361203+0000","last_became_active":"2024-09-17T16:29:02.044488+0000","last_became_peered":"2024-09-17T16:29:02.044488+0000","last_unstale":"2024-09-17T16:46:00.361203+0000","last_undegraded":"2024-09-17T16:46:00.361203+0000","last_fullsized":"2024-09-17T16:46:00.361203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:42:16.232335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":1490,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243057+0000","last_change":"2024-09-17T16:27:54.924373+0000","last_active":"2024-09-17T16:46:00.243057+0000","last_peered":"2024-09-17T16:46:00.243057+0000","last_clean":"2024-09-17T16:46:00.243057+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:46:00.243057+0000","last_undegraded":"2024-09-17T16:46:00.243057+0000","last_fullsized":"2024-09-17T16:46:00.243057+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:27:53.893285+0000","last_clean_scrub_stamp":"2024-09-17T16:27:53.893285+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:10:46.234547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360719+0000","last_change":"2024-09-17T16:28:03.457371+0000","last_active":"2024-09-17T16:46:00.360719+0000","last_peered":"2024-09-17T16:46:00.360719+0000","last_clean":"2024-09-17T16:46:00.360719+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:46:00.360719+0000","last_undegraded":"2024-09-17T16:46:00.360719+0000","last_fullsized":"2024-09-17T16:46:00.360719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:11.419424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355189+0000","last_change":"2024-09-17T16:29:02.189168+0000","last_active":"2024-09-17T16:46:00.355189+0000","last_peered":"2024-09-17T16:46:00.355189+0000","last_clean":"2024-09-17T16:46:00.355189+0000","last_became_active":"2024-09-17T16:29:02.186575+0000","last_became_peered":"2024-09-17T16:29:02.186575+0000","last_unstale":"2024-09-17T16:46:00.355189+0000","last_undegraded":"2024-09-17T16:46:00.355189+0000","last_fullsized":"2024-09-17T16:46:00.355189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:10.592381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1399,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.242887+0000","last_change":"2024-09-17T16:28:03.452726+0000","last_active":"2024-09-17T16:46:00.242887+0000","last_peered":"2024-09-17T16:46:00.242887+0000","last_clean":"2024-09-17T16:46:00.242887+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:46:00.242887+0000","last_undegraded":"2024-09-17T16:46:00.242887+0000","last_fullsized":"2024-09-17T16:46:00.242887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:24:04.043723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.361012+0000","last_change":"2024-09-17T16:29:02.070104+0000","last_active":"2024-09-17T16:46:00.361012+0000","last_peered":"2024-09-17T16:46:00.361012+0000","last_clean":"2024-09-17T16:46:00.361012+0000","last_became_active":"2024-09-17T16:29:02.067560+0000","last_became_peered":"2024-09-17T16:29:02.067560+0000","last_unstale":"2024-09-17T16:46:00.361012+0000","last_undegraded":"2024-09-17T16:46:00.361012+0000","last_fullsized":"2024-09-17T16:46:00.361012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:51.122320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1397,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239633+0000","last_change":"2024-09-17T16:28:03.447500+0000","last_active":"2024-09-17T16:46:00.239633+0000","last_peered":"2024-09-17T16:46:00.239633+0000","last_clean":"2024-09-17T16:46:00.239633+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:46:00.239633+0000","last_undegraded":"2024-09-17T16:46:00.239633+0000","last_fullsized":"2024-09-17T16:46:00.239633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:31.347220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1372,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355522+0000","last_change":"2024-09-17T16:29:02.175917+0000","last_active":"2024-09-17T16:46:00.355522+0000","last_peered":"2024-09-17T16:46:00.355522+0000","last_clean":"2024-09-17T16:46:00.355522+0000","last_became_active":"2024-09-17T16:29:02.175702+0000","last_became_peered":"2024-09-17T16:29:02.175702+0000","last_unstale":"2024-09-17T16:46:00.355522+0000","last_undegraded":"2024-09-17T16:46:00.355522+0000","last_fullsized":"2024-09-17T16:46:00.355522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:10.503343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1388,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:28:03.461846+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:05:53.036432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355872+0000","last_change":"2024-09-17T16:29:02.160011+0000","last_active":"2024-09-17T16:46:00.355872+0000","last_peered":"2024-09-17T16:46:00.355872+0000","last_clean":"2024-09-17T16:46:00.355872+0000","last_became_active":"2024-09-17T16:29:02.158196+0000","last_became_peered":"2024-09-17T16:29:02.158196+0000","last_unstale":"2024-09-17T16:46:00.355872+0000","last_undegraded":"2024-09-17T16:46:00.355872+0000","last_fullsized":"2024-09-17T16:46:00.355872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:11.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1387,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355364+0000","last_change":"2024-09-17T16:28:03.463511+0000","last_active":"2024-09-17T16:46:00.355364+0000","last_peered":"2024-09-17T16:46:00.355364+0000","last_clean":"2024-09-17T16:46:00.355364+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:46:00.355364+0000","last_undegraded":"2024-09-17T16:46:00.355364+0000","last_fullsized":"2024-09-17T16:46:00.355364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:19:27.721355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"22'7","reported_seq":1385,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.354809+0000","last_change":"2024-09-17T16:29:02.118963+0000","last_active":"2024-09-17T16:46:00.354809+0000","last_peered":"2024-09-17T16:46:00.354809+0000","last_clean":"2024-09-17T16:46:00.354809+0000","last_became_active":"2024-09-17T16:29:02.116933+0000","last_became_peered":"2024-09-17T16:29:02.116933+0000","last_unstale":"2024-09-17T16:46:00.354809+0000","last_undegraded":"2024-09-17T16:46:00.354809+0000","last_fullsized":"2024-09-17T16:46:00.354809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:02.801705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243451+0000","last_change":"2024-09-17T16:28:03.451997+0000","last_active":"2024-09-17T16:46:00.243451+0000","last_peered":"2024-09-17T16:46:00.243451+0000","last_clean":"2024-09-17T16:46:00.243451+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:46:00.243451+0000","last_undegraded":"2024-09-17T16:46:00.243451+0000","last_fullsized":"2024-09-17T16:46:00.243451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:15.353100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243022+0000","last_change":"2024-09-17T16:29:02.137515+0000","last_active":"2024-09-17T16:46:00.243022+0000","last_peered":"2024-09-17T16:46:00.243022+0000","last_clean":"2024-09-17T16:46:00.243022+0000","last_became_active":"2024-09-17T16:29:02.136808+0000","last_became_peered":"2024-09-17T16:29:02.136808+0000","last_unstale":"2024-09-17T16:46:00.243022+0000","last_undegraded":"2024-09-17T16:46:00.243022+0000","last_fullsized":"2024-09-17T16:46:00.243022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:04:14.068157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1395,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.324898+0000","last_change":"2024-09-17T16:46:41.324898+0000","last_active":"2024-09-17T16:46:41.324898+0000","last_peered":"2024-09-17T16:46:41.324898+0000","last_clean":"2024-09-17T16:46:41.324898+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:46:41.324898+0000","last_undegraded":"2024-09-17T16:46:41.324898+0000","last_fullsized":"2024-09-17T16:46:41.324898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:41.324741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:41.324741+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.324741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:30:51.362327+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1369,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360439+0000","last_change":"2024-09-17T16:29:02.049672+0000","last_active":"2024-09-17T16:46:00.360439+0000","last_peered":"2024-09-17T16:46:00.360439+0000","last_clean":"2024-09-17T16:46:00.360439+0000","last_became_active":"2024-09-17T16:29:02.048114+0000","last_became_peered":"2024-09-17T16:29:02.048114+0000","last_unstale":"2024-09-17T16:46:00.360439+0000","last_undegraded":"2024-09-17T16:46:00.360439+0000","last_fullsized":"2024-09-17T16:46:00.360439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:00:48.297807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243579+0000","last_change":"2024-09-17T16:28:03.452937+0000","last_active":"2024-09-17T16:46:00.243579+0000","last_peered":"2024-09-17T16:46:00.243579+0000","last_clean":"2024-09-17T16:46:00.243579+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:46:00.243579+0000","last_undegraded":"2024-09-17T16:46:00.243579+0000","last_fullsized":"2024-09-17T16:46:00.243579+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:28:00.422834+0000","last_clean_scrub_stamp":"2024-09-17T16:28:00.422834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:01.469218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1381,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239579+0000","last_change":"2024-09-17T16:29:02.135845+0000","last_active":"2024-09-17T16:46:00.239579+0000","last_peered":"2024-09-17T16:46:00.239579+0000","last_clean":"2024-09-17T16:46:00.239579+0000","last_became_active":"2024-09-17T16:29:02.132790+0000","last_became_peered":"2024-09-17T16:29:02.132790+0000","last_unstale":"2024-09-17T16:46:00.239579+0000","last_undegraded":"2024-09-17T16:46:00.239579+0000","last_fullsized":"2024-09-17T16:46:00.239579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:19.264101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355839+0000","last_change":"2024-09-17T16:29:02.096285+0000","last_active":"2024-09-17T16:46:00.355839+0000","last_peered":"2024-09-17T16:46:00.355839+0000","last_clean":"2024-09-17T16:46:00.355839+0000","last_became_active":"2024-09-17T16:29:02.096072+0000","last_became_peered":"2024-09-17T16:29:02.096072+0000","last_unstale":"2024-09-17T16:46:00.355839+0000","last_undegraded":"2024-09-17T16:46:00.355839+0000","last_fullsized":"2024-09-17T16:46:00.355839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:56.215507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1389,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.286750+0000","last_change":"2024-09-17T16:46:41.286750+0000","last_active":"2024-09-17T16:46:41.286750+0000","last_peered":"2024-09-17T16:46:41.286750+0000","last_clean":"2024-09-17T16:46:41.286750+0000","last_became_active":"2024-09-17T16:29:02.165109+0000","last_became_peered":"2024-09-17T16:29:02.165109+0000","last_unstale":"2024-09-17T16:46:41.286750+0000","last_undegraded":"2024-09-17T16:46:41.286750+0000","last_fullsized":"2024-09-17T16:46:41.286750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:41.286652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:41.286652+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.286652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:30:53.371295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360820+0000","last_change":"2024-09-17T16:29:02.052616+0000","last_active":"2024-09-17T16:46:00.360820+0000","last_peered":"2024-09-17T16:46:00.360820+0000","last_clean":"2024-09-17T16:46:00.360820+0000","last_became_active":"2024-09-17T16:29:02.050385+0000","last_became_peered":"2024-09-17T16:29:02.050385+0000","last_unstale":"2024-09-17T16:46:00.360820+0000","last_undegraded":"2024-09-17T16:46:00.360820+0000","last_fullsized":"2024-09-17T16:46:00.360820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:14.785603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1371,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.355442+0000","last_change":"2024-09-17T16:29:02.192467+0000","last_active":"2024-09-17T16:46:00.355442+0000","last_peered":"2024-09-17T16:46:00.355442+0000","last_clean":"2024-09-17T16:46:00.355442+0000","last_became_active":"2024-09-17T16:29:02.192181+0000","last_became_peered":"2024-09-17T16:29:02.192181+0000","last_unstale":"2024-09-17T16:46:00.355442+0000","last_undegraded":"2024-09-17T16:46:00.355442+0000","last_fullsized":"2024-09-17T16:46:00.355442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:14.890573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243241+0000","last_change":"2024-09-17T16:29:02.123632+0000","last_active":"2024-09-17T16:46:00.243241+0000","last_peered":"2024-09-17T16:46:00.243241+0000","last_clean":"2024-09-17T16:46:00.243241+0000","last_became_active":"2024-09-17T16:29:02.119724+0000","last_became_peered":"2024-09-17T16:29:02.119724+0000","last_unstale":"2024-09-17T16:46:00.243241+0000","last_undegraded":"2024-09-17T16:46:00.243241+0000","last_fullsized":"2024-09-17T16:46:00.243241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:59.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1382,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.239848+0000","last_change":"2024-09-17T16:29:02.075907+0000","last_active":"2024-09-17T16:46:00.239848+0000","last_peered":"2024-09-17T16:46:00.239848+0000","last_clean":"2024-09-17T16:46:00.239848+0000","last_became_active":"2024-09-17T16:29:02.075599+0000","last_became_peered":"2024-09-17T16:29:02.075599+0000","last_unstale":"2024-09-17T16:46:00.239848+0000","last_undegraded":"2024-09-17T16:46:00.239848+0000","last_fullsized":"2024-09-17T16:46:00.239848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:34:21.469780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.243362+0000","last_change":"2024-09-17T16:29:02.074165+0000","last_active":"2024-09-17T16:46:00.243362+0000","last_peered":"2024-09-17T16:46:00.243362+0000","last_clean":"2024-09-17T16:46:00.243362+0000","last_became_active":"2024-09-17T16:29:02.073908+0000","last_became_peered":"2024-09-17T16:29:02.073908+0000","last_unstale":"2024-09-17T16:46:00.243362+0000","last_undegraded":"2024-09-17T16:46:00.243362+0000","last_fullsized":"2024-09-17T16:46:00.243362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:09:53.379585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1370,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360319+0000","last_change":"2024-09-17T16:29:02.151691+0000","last_active":"2024-09-17T16:46:00.360319+0000","last_peered":"2024-09-17T16:46:00.360319+0000","last_clean":"2024-09-17T16:46:00.360319+0000","last_became_active":"2024-09-17T16:29:02.151449+0000","last_became_peered":"2024-09-17T16:29:02.151449+0000","last_unstale":"2024-09-17T16:46:00.360319+0000","last_undegraded":"2024-09-17T16:46:00.360319+0000","last_fullsized":"2024-09-17T16:46:00.360319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:00.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356257+0000","last_change":"2024-09-17T16:29:02.159400+0000","last_active":"2024-09-17T16:46:00.356257+0000","last_peered":"2024-09-17T16:46:00.356257+0000","last_clean":"2024-09-17T16:46:00.356257+0000","last_became_active":"2024-09-17T16:29:02.157013+0000","last_became_peered":"2024-09-17T16:29:02.157013+0000","last_unstale":"2024-09-17T16:46:00.356257+0000","last_undegraded":"2024-09-17T16:46:00.356257+0000","last_fullsized":"2024-09-17T16:46:00.356257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.554975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1375,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.356342+0000","last_change":"2024-09-17T16:29:02.146313+0000","last_active":"2024-09-17T16:46:00.356342+0000","last_peered":"2024-09-17T16:46:00.356342+0000","last_clean":"2024-09-17T16:46:00.356342+0000","last_became_active":"2024-09-17T16:29:02.144760+0000","last_became_peered":"2024-09-17T16:29:02.144760+0000","last_unstale":"2024-09-17T16:46:00.356342+0000","last_undegraded":"2024-09-17T16:46:00.356342+0000","last_fullsized":"2024-09-17T16:46:00.356342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:19.598819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1368,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:00.360880+0000","last_change":"2024-09-17T16:29:02.050143+0000","last_active":"2024-09-17T16:46:00.360880+0000","last_peered":"2024-09-17T16:46:00.360880+0000","last_clean":"2024-09-17T16:46:00.360880+0000","last_became_active":"2024-09-17T16:29:02.049439+0000","last_became_peered":"2024-09-17T16:29:02.049439+0000","last_unstale":"2024-09-17T16:46:00.360880+0000","last_undegraded":"2024-09-17T16:46:00.360880+0000","last_fullsized":"2024-09-17T16:46:00.360880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:29:00.894997+0000","last_clean_scrub_stamp":"2024-09-17T16:29:00.894997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:44.935606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607780,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607781,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10856,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11116544,"data_stored":6439173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607781,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78132,"kb_used_data":10852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293708,"statfs":{"total":96636764160,"available":96556756992,"internally_reserved":0,"allocated":11112448,"data_stored":6439168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640485,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81716,"kb_used_data":10276,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290124,"statfs":{"total":96636764160,"available":96553086976,"internally_reserved":0,"allocated":10522624,"data_stored":5848819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640485,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640485,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77560,"kb_used_data":10280,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294280,"statfs":{"total":96636764160,"available":96557342720,"internally_reserved":0,"allocated":10526720,"data_stored":5848824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:46:42.635 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.635 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.635 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.635 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.636 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T16:27:53.893285+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=27, tm_sec=53, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.637 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.638 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T16:28:00.422834+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=28, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.639 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.640 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.640 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.640 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-17T16:29:00.894997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=29, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=16, tm_min=46, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T16:46:42.640 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T16:47:02.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T16:47:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.762+0000 7fb38a254640 10 monclient: get_monmap_and_config 2024-09-17T16:47:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.762+0000 7fb38a254640 10 monclient: build_initial_monmap 2024-09-17T16:47:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.762+0000 7fb38a254640 10 monclient: monmap: 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:47:02.766970+0000 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:47:02.766970+0000 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:47:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:47:02.772 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:47:02.772 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:47:02.772 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:47:02.772 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 Processor -- start 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 -- start start 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient: init 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient: _reopen_session rank -1 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient: _add_conns ranks=[10,4,2,3,1,0,8,7,5,9,6] 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 0x7fb3840a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-f con 0x7fb3840a68e0 addr v2:172.21.15.160:3300/0 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-c con 0x7fb3840a4910 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 0x7fb384139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-b con 0x7fb3841396a0 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb3840a3fc0 con 0x7fb3840a68e0 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb3840a63c0 con 0x7fb3841396a0 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb3840a7740 con 0x7fb3840a4910 2024-09-17T16:47:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): _renew_subs 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient(hunting): authenticate will time out at 2106.032732s 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 0x7fb384139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:44004/0 (socket says 172.21.15.105:44004) 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 1 -- 172.21.15.105:0/1798419856 learned_addr learned my addr 172.21.15.105:0/1798419856 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): get_auth_request con 0x7fb3840a4910 auth_method 0 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): get_auth_request con 0x7fb3841396a0 auth_method 0 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:02.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3837fe640 1 --2- 172.21.15.105:0/1798419856 >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 0x7fb3840a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient(hunting): handle_auth_done global_id 84746 payload 274 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 -- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 msgr2=0x7fb3840a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 -- 172.21.15.105:0/1798419856 >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 msgr2=0x7fb3840a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:47:02.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 --2- 172.21.15.105:0/1798419856 >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 0x7fb3840a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient: _finish_hunting 0 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 monclient: found mon.noname-b 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.105:3301/0 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 -- 172.21.15.105:0/1798419856 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38406ebc0 con 0x7fb3841396a0 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3837fe640 10 monclient: get_auth_request con 0x7fb3840a68e0 auth_method 0 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb382ffd640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:47:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3837fe640 1 --2- 172.21.15.105:0/1798419856 >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 0x7fb3840a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb383fff640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 0x7fb384139b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009d90 tx=0x7fb37c01c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb7db03c8386b47f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 1 -- 172.21.15.105:0/1798419856 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb37c0b2070 con 0x7fb3841396a0 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: dump: 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: _finish_auth 0 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: _check_auth_tickets 2024-09-17T16:47:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 1 -- 172.21.15.105:0/1798419856 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37c0a5ad0 con 0x7fb3841396a0 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 5 monclient: authenticate success, global_id 84746 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: handle_config config(0 keys) 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 10 monclient: get_monmap_and_config success 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 1 -- 172.21.15.105:0/1798419856 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb37c0a5d00 con 0x7fb3841396a0 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb38a254640 4 set_mon_vals no callback set 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.766+0000 7fb3827fc640 10 monclient: dump: 2024-09-17T16:47:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: shutdown 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 msgr2=0x7fb384139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 0x7fb384139b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009d90 tx=0x7fb37c01c800 comp rx=0 tx=0).stop 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1798419856 shutdown_connections 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3841396a0 0x7fb384139b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- 172.21.15.105:0/1798419856 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3840a4910 0x7fb3840a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- 172.21.15.105:0/1798419856 >> v2:172.21.15.160:3300/0 conn(0x7fb3840a68e0 0x7fb3840a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1798419856 >> 172.21.15.105:0/1798419856 conn(0x7fb384068ba0 msgr2=0x7fb384068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1798419856 shutdown_connections 2024-09-17T16:47:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1798419856 wait complete. 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: build_initial_monmap 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: monmap: 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:47:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 librados: starting msgr at 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 librados: starting objecter 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 Processor -- start 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- start start 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 librados: setting wanted keys 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 librados: calling monclient init 2024-09-17T16:47:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: init 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _reopen_session rank -1 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _add_conns ranks=[5,7,4,8,3,6,2,1,0] 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb3840a4910 0x7fb3841d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-h con 0x7fb3840a4910 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 0x7fb3841d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-i con 0x7fb3840a68e0 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): picked mon.noname-c con 0x7fb3841396a0 addr [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_getmap magic: 0 -- 0x7fb38406e630 con 0x7fb3841396a0 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7fb3840a7af0 con 0x7fb3840a4910 2024-09-17T16:47:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7fb384143320 con 0x7fb3840a68e0 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): _renew_subs 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient(hunting): authenticate will time out at 2106.037808s 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 --2- >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3302/0 says I am v2:172.21.15.105:44008/0 (socket says 172.21.15.105:44008) 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 -- 172.21.15.105:0/1133595573 learned_addr learned my addr 172.21.15.105:0/1133595573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): get_auth_request con 0x7fb3841396a0 auth_method 0 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3837fe640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb3840a4910 0x7fb3841d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb382ffd640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 0x7fb3841d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3837fe640 10 monclient(hunting): get_auth_request con 0x7fb3840a4910 auth_method 0 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3837fe640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient(hunting): handle_auth_done global_id 75073 payload 824 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 -- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 msgr2=0x7fb3841d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 0x7fb3841d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 -- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb3840a4910 msgr2=0x7fb3841d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb3840a4910 0x7fb3841d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient: _finish_hunting 0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 monclient: found mon.noname-c 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.105:3302/0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 -- 172.21.15.105:0/1133595573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb384123ba0 con 0x7fb3841396a0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb382ffd640 10 monclient: get_auth_request con 0x7fb3840a68e0 auth_method 0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb382ffd640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 0x7fb3841d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb383fff640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0b0040 tx=0x7fb37c0195a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 client.?.objecter ms_handle_connect 0x7fb3841396a0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 client.?.objecter resend_mon_ops 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb37c0b2050 con 0x7fb3841396a0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: dump: 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: _finish_auth 0 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: _check_auth_tickets 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 5 monclient: authenticate success, global_id 75073 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37c0be070 con 0x7fb3841396a0 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: handle_config config(0 keys) 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb37c003e10 con 0x7fb3841396a0 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb3817fa640 4 set_mon_vals no callback set 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-17T16:47:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb36bfff640 10 monclient: dump: 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _renew_subs 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3840ab9a0 con 0x7fb3841396a0 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 client.75073.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _renew_subs 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841238c0 con 0x7fb3841396a0 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.770+0000 7fb38a254640 1 librados: init done 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb38a254640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb38a254640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb38a254640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.105:3302/0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7fb37c0039a0 con 0x7fb3841396a0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 --> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb384139b50 con 0x7fb3841396a0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 0x7fb370040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 5 ==== osd_map(707..707 src has 561..707) ==== 5529+0+0 (secure 0 0 0) 0x7fb37c0eaca0 con 0x7fb3841396a0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 10 client.75073.objecter ms_dispatch 0x7fb384000ba0 osd_map(707..707 src has 561..707) 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 3 client.75073.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 3 client.75073.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 20 client.75073.objecter dump_active .. 0 homeless 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb3837fe640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 0x7fb370040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb3837fe640 10 monclient: get_auth_request con 0x7fb37003e550 auth_method 0 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb36bfff640 10 client.75073.objecter ms_handle_connect 0x7fb37003e550 2024-09-17T16:47:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.774+0000 7fb3837fe640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 0x7fb370040a10 secure :-1 s=READY pgs=3693 cs=0 l=1 rev1=1 crypto rx=0x7fb37800a9a0 tx=0x7fb378008040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.778+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mon.4 v2:172.21.15.105:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb37c016540 con 0x7fb3841396a0 2024-09-17T16:47:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.778+0000 7fb36bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.778+0000 7fb36bfff640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:47:02.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.942+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 --> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb38413f2c0 con 0x7fb37003e550 2024-09-17T16:47:02.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.950+0000 7fb36bfff640 1 -- 172.21.15.105:0/1133595573 <== mgr.4118 v2:172.21.15.160:6824/3499101759 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+114069 (secure 0 0 0) 0x7fb38413f2c0 con 0x7fb37003e550 2024-09-17T16:47:02.958 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-17T16:47:02.959 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 10 librados: watch_flush enter 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 10 librados: watch_flush exit 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 20 client.75073.objecter shutdown clearing up homeless session... 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 10 client.75073.objecter successfully canceled tick 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 msgr2=0x7fb370040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:02.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 0x7fb370040a10 secure :-1 s=READY pgs=3693 cs=0 l=1 rev1=1 crypto rx=0x7fb37800a9a0 tx=0x7fb378008040 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 10 monclient: shutdown 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 msgr2=0x7fb38413d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0b0040 tx=0x7fb37c0195a0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 shutdown_connections 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7fb37003e550 0x7fb370040a10 unknown :-1 s=CLOSED pgs=3693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] conn(0x7fb3841396a0 0x7fb38413d470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7fb3840a68e0 0x7fb3841d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 --2- 172.21.15.105:0/1133595573 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7fb3840a4910 0x7fb3841d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 >> 172.21.15.105:0/1133595573 conn(0x7fb384068ba0 msgr2=0x7fb3840a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 shutdown_connections 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 -- 172.21.15.105:0/1133595573 wait complete. 2024-09-17T16:47:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:02.954+0000 7fb38a254640 1 librados: shutdown 2024-09-17T16:47:02.975 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":1277,"stamp":"2024-09-17T16:47:02.201710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":72,"num_read_kb":61,"num_write":110,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470648,"kb_used_data":62808,"kb_used_omap":9,"kb_used_meta":407734,"kb_avail":565760392,"statfs":{"total":579820584960,"available":579338641408,"internally_reserved":0,"allocated":64315392,"data_stored":36273584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":417520288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003978"},"pg_stats":[{"pgid":"6.16","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:42.456008+0000","last_change":"2024-09-17T16:46:42.456008+0000","last_active":"2024-09-17T16:46:42.456008+0000","last_peered":"2024-09-17T16:46:42.456008+0000","last_clean":"2024-09-17T16:46:42.456008+0000","last_became_active":"2024-09-17T16:29:02.017756+0000","last_became_peered":"2024-09-17T16:29:02.017756+0000","last_unstale":"2024-09-17T16:46:42.456008+0000","last_undegraded":"2024-09-17T16:46:42.456008+0000","last_fullsized":"2024-09-17T16:46:42.456008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:42.455945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:42.455945+0000","last_clean_scrub_stamp":"2024-09-17T16:46:42.455945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:56.921906+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.011307+0000","last_change":"2024-09-17T16:46:44.011307+0000","last_active":"2024-09-17T16:46:44.011307+0000","last_peered":"2024-09-17T16:46:44.011307+0000","last_clean":"2024-09-17T16:46:44.011307+0000","last_became_active":"2024-09-17T16:29:02.054559+0000","last_became_peered":"2024-09-17T16:29:02.054559+0000","last_unstale":"2024-09-17T16:46:44.011307+0000","last_undegraded":"2024-09-17T16:46:44.011307+0000","last_fullsized":"2024-09-17T16:46:44.011307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.011151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.011151+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.011151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:21:16.078789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1389,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.295223+0000","last_change":"2024-09-17T16:46:45.295223+0000","last_active":"2024-09-17T16:46:45.295223+0000","last_peered":"2024-09-17T16:46:45.295223+0000","last_clean":"2024-09-17T16:46:45.295223+0000","last_became_active":"2024-09-17T16:29:02.076410+0000","last_became_peered":"2024-09-17T16:29:02.076410+0000","last_unstale":"2024-09-17T16:46:45.295223+0000","last_undegraded":"2024-09-17T16:46:45.295223+0000","last_fullsized":"2024-09-17T16:46:45.295223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.295066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.295066+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.295066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:34:00.129349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1390,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:49.944218+0000","last_change":"2024-09-17T16:46:49.944218+0000","last_active":"2024-09-17T16:46:49.944218+0000","last_peered":"2024-09-17T16:46:49.944218+0000","last_clean":"2024-09-17T16:46:49.944218+0000","last_became_active":"2024-09-17T16:29:02.122292+0000","last_became_peered":"2024-09-17T16:29:02.122292+0000","last_unstale":"2024-09-17T16:46:49.944218+0000","last_undegraded":"2024-09-17T16:46:49.944218+0000","last_fullsized":"2024-09-17T16:46:49.944218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:49.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:49.944055+0000","last_clean_scrub_stamp":"2024-09-17T16:46:49.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:47.155858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1379,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:46.304919+0000","last_change":"2024-09-17T16:46:46.304919+0000","last_active":"2024-09-17T16:46:46.304919+0000","last_peered":"2024-09-17T16:46:46.304919+0000","last_clean":"2024-09-17T16:46:46.304919+0000","last_became_active":"2024-09-17T16:29:02.064232+0000","last_became_peered":"2024-09-17T16:29:02.064232+0000","last_unstale":"2024-09-17T16:46:46.304919+0000","last_undegraded":"2024-09-17T16:46:46.304919+0000","last_fullsized":"2024-09-17T16:46:46.304919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:46.304753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:46.304753+0000","last_clean_scrub_stamp":"2024-09-17T16:46:46.304753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:53.698134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1380,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:48.396543+0000","last_change":"2024-09-17T16:46:48.396464+0000","last_active":"2024-09-17T16:46:48.396543+0000","last_peered":"2024-09-17T16:46:48.396543+0000","last_clean":"2024-09-17T16:46:48.396543+0000","last_became_active":"2024-09-17T16:29:02.116801+0000","last_became_peered":"2024-09-17T16:29:02.116801+0000","last_unstale":"2024-09-17T16:46:48.396543+0000","last_undegraded":"2024-09-17T16:46:48.396543+0000","last_fullsized":"2024-09-17T16:46:48.396543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:48.396376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:48.396376+0000","last_clean_scrub_stamp":"2024-09-17T16:46:48.396376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:10.526943+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1391,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:47.044734+0000","last_change":"2024-09-17T16:46:47.044734+0000","last_active":"2024-09-17T16:46:47.044734+0000","last_peered":"2024-09-17T16:46:47.044734+0000","last_clean":"2024-09-17T16:46:47.044734+0000","last_became_active":"2024-09-17T16:29:02.129701+0000","last_became_peered":"2024-09-17T16:29:02.129701+0000","last_unstale":"2024-09-17T16:46:47.044734+0000","last_undegraded":"2024-09-17T16:46:47.044734+0000","last_fullsized":"2024-09-17T16:46:47.044734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:47.044575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:47.044575+0000","last_clean_scrub_stamp":"2024-09-17T16:46:47.044575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:52.325684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1384,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.505083+0000","last_change":"2024-09-17T16:46:41.505083+0000","last_active":"2024-09-17T16:46:41.505083+0000","last_peered":"2024-09-17T16:46:41.505083+0000","last_clean":"2024-09-17T16:46:41.505083+0000","last_became_active":"2024-09-17T16:29:02.078833+0000","last_became_peered":"2024-09-17T16:29:02.078833+0000","last_unstale":"2024-09-17T16:46:41.505083+0000","last_undegraded":"2024-09-17T16:46:41.505083+0000","last_fullsized":"2024-09-17T16:46:41.505083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:41.505018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:41.505018+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.505018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:47.446010+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.037696+0000","last_change":"2024-09-17T16:46:45.037696+0000","last_active":"2024-09-17T16:46:45.037696+0000","last_peered":"2024-09-17T16:46:45.037696+0000","last_clean":"2024-09-17T16:46:45.037696+0000","last_became_active":"2024-09-17T16:29:02.179921+0000","last_became_peered":"2024-09-17T16:29:02.179921+0000","last_unstale":"2024-09-17T16:46:45.037696+0000","last_undegraded":"2024-09-17T16:46:45.037696+0000","last_fullsized":"2024-09-17T16:46:45.037696+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.037518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.037518+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.037518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:18.616474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1390,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:42.317836+0000","last_change":"2024-09-17T16:46:42.317770+0000","last_active":"2024-09-17T16:46:42.317836+0000","last_peered":"2024-09-17T16:46:42.317836+0000","last_clean":"2024-09-17T16:46:42.317836+0000","last_became_active":"2024-09-17T16:29:02.164714+0000","last_became_peered":"2024-09-17T16:29:02.164714+0000","last_unstale":"2024-09-17T16:46:42.317836+0000","last_undegraded":"2024-09-17T16:46:42.317836+0000","last_fullsized":"2024-09-17T16:46:42.317836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:42.317685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:42.317685+0000","last_clean_scrub_stamp":"2024-09-17T16:46:42.317685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:36:35.352594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1380,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.423839+0000","last_change":"2024-09-17T16:46:44.423673+0000","last_active":"2024-09-17T16:46:44.423839+0000","last_peered":"2024-09-17T16:46:44.423839+0000","last_clean":"2024-09-17T16:46:44.423839+0000","last_became_active":"2024-09-17T16:29:02.177674+0000","last_became_peered":"2024-09-17T16:29:02.177674+0000","last_unstale":"2024-09-17T16:46:44.423839+0000","last_undegraded":"2024-09-17T16:46:44.423839+0000","last_fullsized":"2024-09-17T16:46:44.423839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.423516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.423516+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.423516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:57:29.656702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1379,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:48.281998+0000","last_change":"2024-09-17T16:46:48.281839+0000","last_active":"2024-09-17T16:46:48.281998+0000","last_peered":"2024-09-17T16:46:48.281998+0000","last_clean":"2024-09-17T16:46:48.281998+0000","last_became_active":"2024-09-17T16:29:02.184301+0000","last_became_peered":"2024-09-17T16:29:02.184301+0000","last_unstale":"2024-09-17T16:46:48.281998+0000","last_undegraded":"2024-09-17T16:46:48.281998+0000","last_fullsized":"2024-09-17T16:46:48.281998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:48.281650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:48.281650+0000","last_clean_scrub_stamp":"2024-09-17T16:46:48.281650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:05:45.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1379,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.314147+0000","last_change":"2024-09-17T16:46:45.314147+0000","last_active":"2024-09-17T16:46:45.314147+0000","last_peered":"2024-09-17T16:46:45.314147+0000","last_clean":"2024-09-17T16:46:45.314147+0000","last_became_active":"2024-09-17T16:29:02.044488+0000","last_became_peered":"2024-09-17T16:29:02.044488+0000","last_unstale":"2024-09-17T16:46:45.314147+0000","last_undegraded":"2024-09-17T16:46:45.314147+0000","last_fullsized":"2024-09-17T16:46:45.314147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.313987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.313987+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.313987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:53:21.042363+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"707'74","reported_seq":1500,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.976696+0000","last_change":"2024-09-17T16:46:41.976696+0000","last_active":"2024-09-17T16:46:41.976696+0000","last_peered":"2024-09-17T16:46:41.976696+0000","last_clean":"2024-09-17T16:46:41.976696+0000","last_became_active":"2024-09-17T16:27:54.923583+0000","last_became_peered":"2024-09-17T16:27:54.923583+0000","last_unstale":"2024-09-17T16:46:41.976696+0000","last_undegraded":"2024-09-17T16:46:41.976696+0000","last_fullsized":"2024-09-17T16:46:41.976696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'74","last_scrub_stamp":"2024-09-17T16:46:41.976521+0000","last_deep_scrub":"707'74","last_deep_scrub_stamp":"2024-09-17T16:46:41.976521+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.976521+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:29:46.753084+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"707'2","reported_seq":1397,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:42.340196+0000","last_change":"2024-09-17T16:46:42.340053+0000","last_active":"2024-09-17T16:46:42.340196+0000","last_peered":"2024-09-17T16:46:42.340196+0000","last_clean":"2024-09-17T16:46:42.340196+0000","last_became_active":"2024-09-17T16:28:01.455670+0000","last_became_peered":"2024-09-17T16:28:01.455670+0000","last_unstale":"2024-09-17T16:46:42.340196+0000","last_undegraded":"2024-09-17T16:46:42.340196+0000","last_fullsized":"2024-09-17T16:46:42.340196+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'2","last_scrub_stamp":"2024-09-17T16:46:42.339883+0000","last_deep_scrub":"707'2","last_deep_scrub_stamp":"2024-09-17T16:46:42.339883+0000","last_clean_scrub_stamp":"2024-09-17T16:46:42.339883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:22:56.608807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010408419999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1380,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.401959+0000","last_change":"2024-09-17T16:46:43.401959+0000","last_active":"2024-09-17T16:46:43.401959+0000","last_peered":"2024-09-17T16:46:43.401959+0000","last_clean":"2024-09-17T16:46:43.401959+0000","last_became_active":"2024-09-17T16:29:02.186575+0000","last_became_peered":"2024-09-17T16:29:02.186575+0000","last_unstale":"2024-09-17T16:46:43.401959+0000","last_undegraded":"2024-09-17T16:46:43.401959+0000","last_fullsized":"2024-09-17T16:46:43.401959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:43.401811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:43.401811+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.401811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:08.831209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1407,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.011584+0000","last_change":"2024-09-17T16:46:43.011584+0000","last_active":"2024-09-17T16:46:43.011584+0000","last_peered":"2024-09-17T16:46:43.011584+0000","last_clean":"2024-09-17T16:46:43.011584+0000","last_became_active":"2024-09-17T16:28:01.451690+0000","last_became_peered":"2024-09-17T16:28:01.451690+0000","last_unstale":"2024-09-17T16:46:43.011584+0000","last_undegraded":"2024-09-17T16:46:43.011584+0000","last_fullsized":"2024-09-17T16:46:43.011584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:43.011421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:43.011421+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.011421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:31:10.879942+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021251,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1379,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.292496+0000","last_change":"2024-09-17T16:46:44.292496+0000","last_active":"2024-09-17T16:46:44.292496+0000","last_peered":"2024-09-17T16:46:44.292496+0000","last_clean":"2024-09-17T16:46:44.292496+0000","last_became_active":"2024-09-17T16:29:02.067560+0000","last_became_peered":"2024-09-17T16:29:02.067560+0000","last_unstale":"2024-09-17T16:46:44.292496+0000","last_undegraded":"2024-09-17T16:46:44.292496+0000","last_fullsized":"2024-09-17T16:46:44.292496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.292339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.292339+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.292339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:35:31.443622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1406,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.308301+0000","last_change":"2024-09-17T16:46:43.308178+0000","last_active":"2024-09-17T16:46:43.308301+0000","last_peered":"2024-09-17T16:46:43.308301+0000","last_clean":"2024-09-17T16:46:43.308301+0000","last_became_active":"2024-09-17T16:28:01.455448+0000","last_became_peered":"2024-09-17T16:28:01.455448+0000","last_unstale":"2024-09-17T16:46:43.308301+0000","last_undegraded":"2024-09-17T16:46:43.308301+0000","last_fullsized":"2024-09-17T16:46:43.308301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:43.308018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:43.308018+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.308018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:18:21.215535+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0011083130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1380,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:46.346814+0000","last_change":"2024-09-17T16:46:46.346814+0000","last_active":"2024-09-17T16:46:46.346814+0000","last_peered":"2024-09-17T16:46:46.346814+0000","last_clean":"2024-09-17T16:46:46.346814+0000","last_became_active":"2024-09-17T16:29:02.175702+0000","last_became_peered":"2024-09-17T16:29:02.175702+0000","last_unstale":"2024-09-17T16:46:46.346814+0000","last_undegraded":"2024-09-17T16:46:46.346814+0000","last_fullsized":"2024-09-17T16:46:46.346814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:46.346659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:46.346659+0000","last_clean_scrub_stamp":"2024-09-17T16:46:46.346659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:19.984831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"707'3","reported_seq":1398,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.377315+0000","last_change":"2024-09-17T16:46:43.377069+0000","last_active":"2024-09-17T16:46:43.377315+0000","last_peered":"2024-09-17T16:46:43.377315+0000","last_clean":"2024-09-17T16:46:43.377315+0000","last_became_active":"2024-09-17T16:28:01.454558+0000","last_became_peered":"2024-09-17T16:28:01.454558+0000","last_unstale":"2024-09-17T16:46:43.377315+0000","last_undegraded":"2024-09-17T16:46:43.377315+0000","last_fullsized":"2024-09-17T16:46:43.377315+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'3","last_scrub_stamp":"2024-09-17T16:46:43.376841+0000","last_deep_scrub":"707'3","last_deep_scrub_stamp":"2024-09-17T16:46:43.376841+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.376841+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:47.046474+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0055389840000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1384,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:40.458928+0000","last_change":"2024-09-17T16:46:40.458928+0000","last_active":"2024-09-17T16:46:40.458928+0000","last_peered":"2024-09-17T16:46:40.458928+0000","last_clean":"2024-09-17T16:46:40.458928+0000","last_became_active":"2024-09-17T16:29:02.158196+0000","last_became_peered":"2024-09-17T16:29:02.158196+0000","last_unstale":"2024-09-17T16:46:40.458928+0000","last_undegraded":"2024-09-17T16:46:40.458928+0000","last_fullsized":"2024-09-17T16:46:40.458928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:40.458861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:40.458861+0000","last_clean_scrub_stamp":"2024-09-17T16:46:40.458861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:42.291038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1396,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:47.369207+0000","last_change":"2024-09-17T16:46:47.369207+0000","last_active":"2024-09-17T16:46:47.369207+0000","last_peered":"2024-09-17T16:46:47.369207+0000","last_clean":"2024-09-17T16:46:47.369207+0000","last_became_active":"2024-09-17T16:28:01.448270+0000","last_became_peered":"2024-09-17T16:28:01.448270+0000","last_unstale":"2024-09-17T16:46:47.369207+0000","last_undegraded":"2024-09-17T16:46:47.369207+0000","last_fullsized":"2024-09-17T16:46:47.369207+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:47.368958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:47.368958+0000","last_clean_scrub_stamp":"2024-09-17T16:46:47.368958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:35:39.202646+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0012208309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"707'8","reported_seq":1394,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:42.375731+0000","last_change":"2024-09-17T16:46:42.375731+0000","last_active":"2024-09-17T16:46:42.375731+0000","last_peered":"2024-09-17T16:46:42.375731+0000","last_clean":"2024-09-17T16:46:42.375731+0000","last_became_active":"2024-09-17T16:29:02.116933+0000","last_became_peered":"2024-09-17T16:29:02.116933+0000","last_unstale":"2024-09-17T16:46:42.375731+0000","last_undegraded":"2024-09-17T16:46:42.375731+0000","last_fullsized":"2024-09-17T16:46:42.375731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"707'8","last_scrub_stamp":"2024-09-17T16:46:42.375643+0000","last_deep_scrub":"707'8","last_deep_scrub_stamp":"2024-09-17T16:46:42.375643+0000","last_clean_scrub_stamp":"2024-09-17T16:46:42.375643+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:00:56.913914+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1407,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.020005+0000","last_change":"2024-09-17T16:46:45.019875+0000","last_active":"2024-09-17T16:46:45.020005+0000","last_peered":"2024-09-17T16:46:45.020005+0000","last_clean":"2024-09-17T16:46:45.020005+0000","last_became_active":"2024-09-17T16:28:01.451973+0000","last_became_peered":"2024-09-17T16:28:01.451973+0000","last_unstale":"2024-09-17T16:46:45.020005+0000","last_undegraded":"2024-09-17T16:46:45.020005+0000","last_fullsized":"2024-09-17T16:46:45.020005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.019689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.019689+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.019689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:40:52.595734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0012435739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1391,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.011231+0000","last_change":"2024-09-17T16:46:44.011231+0000","last_active":"2024-09-17T16:46:44.011231+0000","last_peered":"2024-09-17T16:46:44.011231+0000","last_clean":"2024-09-17T16:46:44.011231+0000","last_became_active":"2024-09-17T16:29:02.136808+0000","last_became_peered":"2024-09-17T16:29:02.136808+0000","last_unstale":"2024-09-17T16:46:44.011231+0000","last_undegraded":"2024-09-17T16:46:44.011231+0000","last_fullsized":"2024-09-17T16:46:44.011231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.011061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.011061+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.011061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:40:42.183858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1395,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.324898+0000","last_change":"2024-09-17T16:46:41.324898+0000","last_active":"2024-09-17T16:46:41.324898+0000","last_peered":"2024-09-17T16:46:41.324898+0000","last_clean":"2024-09-17T16:46:41.324898+0000","last_became_active":"2024-09-17T16:28:01.447614+0000","last_became_peered":"2024-09-17T16:28:01.447614+0000","last_unstale":"2024-09-17T16:46:41.324898+0000","last_undegraded":"2024-09-17T16:46:41.324898+0000","last_fullsized":"2024-09-17T16:46:41.324898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:41.324741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:41.324741+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.324741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:30:51.362327+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001173299,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1377,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:42.055523+0000","last_change":"2024-09-17T16:46:42.055523+0000","last_active":"2024-09-17T16:46:42.055523+0000","last_peered":"2024-09-17T16:46:42.055523+0000","last_clean":"2024-09-17T16:46:42.055523+0000","last_became_active":"2024-09-17T16:29:02.048114+0000","last_became_peered":"2024-09-17T16:29:02.048114+0000","last_unstale":"2024-09-17T16:46:42.055523+0000","last_undegraded":"2024-09-17T16:46:42.055523+0000","last_fullsized":"2024-09-17T16:46:42.055523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:42.055360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:42.055360+0000","last_clean_scrub_stamp":"2024-09-17T16:46:42.055360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:31:11.671133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1406,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.998092+0000","last_change":"2024-09-17T16:46:45.998092+0000","last_active":"2024-09-17T16:46:45.998092+0000","last_peered":"2024-09-17T16:46:45.998092+0000","last_clean":"2024-09-17T16:46:45.998092+0000","last_became_active":"2024-09-17T16:28:01.450595+0000","last_became_peered":"2024-09-17T16:28:01.450595+0000","last_unstale":"2024-09-17T16:46:45.998092+0000","last_undegraded":"2024-09-17T16:46:45.998092+0000","last_fullsized":"2024-09-17T16:46:45.998092+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.997903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.997903+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.997903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:37:56.778029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001275553,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1389,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:46.326323+0000","last_change":"2024-09-17T16:46:46.326323+0000","last_active":"2024-09-17T16:46:46.326323+0000","last_peered":"2024-09-17T16:46:46.326323+0000","last_clean":"2024-09-17T16:46:46.326323+0000","last_became_active":"2024-09-17T16:29:02.132790+0000","last_became_peered":"2024-09-17T16:29:02.132790+0000","last_unstale":"2024-09-17T16:46:46.326323+0000","last_undegraded":"2024-09-17T16:46:46.326323+0000","last_fullsized":"2024-09-17T16:46:46.326323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:46.326159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:46.326159+0000","last_clean_scrub_stamp":"2024-09-17T16:46:46.326159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:21:05.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.474465+0000","last_change":"2024-09-17T16:46:44.474465+0000","last_active":"2024-09-17T16:46:44.474465+0000","last_peered":"2024-09-17T16:46:44.474465+0000","last_clean":"2024-09-17T16:46:44.474465+0000","last_became_active":"2024-09-17T16:29:02.096072+0000","last_became_peered":"2024-09-17T16:29:02.096072+0000","last_unstale":"2024-09-17T16:46:44.474465+0000","last_undegraded":"2024-09-17T16:46:44.474465+0000","last_fullsized":"2024-09-17T16:46:44.474465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.474307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.474307+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.474307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:10.896453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1389,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:41.286750+0000","last_change":"2024-09-17T16:46:41.286750+0000","last_active":"2024-09-17T16:46:41.286750+0000","last_peered":"2024-09-17T16:46:41.286750+0000","last_clean":"2024-09-17T16:46:41.286750+0000","last_became_active":"2024-09-17T16:29:02.165109+0000","last_became_peered":"2024-09-17T16:29:02.165109+0000","last_unstale":"2024-09-17T16:46:41.286750+0000","last_undegraded":"2024-09-17T16:46:41.286750+0000","last_fullsized":"2024-09-17T16:46:41.286750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:41.286652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:41.286652+0000","last_clean_scrub_stamp":"2024-09-17T16:46:41.286652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:30:53.371295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1378,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:47.289921+0000","last_change":"2024-09-17T16:46:47.289921+0000","last_active":"2024-09-17T16:46:47.289921+0000","last_peered":"2024-09-17T16:46:47.289921+0000","last_clean":"2024-09-17T16:46:47.289921+0000","last_became_active":"2024-09-17T16:29:02.050385+0000","last_became_peered":"2024-09-17T16:29:02.050385+0000","last_unstale":"2024-09-17T16:46:47.289921+0000","last_undegraded":"2024-09-17T16:46:47.289921+0000","last_fullsized":"2024-09-17T16:46:47.289921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:47.289768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:47.289768+0000","last_clean_scrub_stamp":"2024-09-17T16:46:47.289768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:44:05.916027+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1379,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.387860+0000","last_change":"2024-09-17T16:46:45.387860+0000","last_active":"2024-09-17T16:46:45.387860+0000","last_peered":"2024-09-17T16:46:45.387860+0000","last_clean":"2024-09-17T16:46:45.387860+0000","last_became_active":"2024-09-17T16:29:02.192181+0000","last_became_peered":"2024-09-17T16:29:02.192181+0000","last_unstale":"2024-09-17T16:46:45.387860+0000","last_undegraded":"2024-09-17T16:46:45.387860+0000","last_fullsized":"2024-09-17T16:46:45.387860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.387681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.387681+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.387681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:02:05.217499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1390,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:48.031787+0000","last_change":"2024-09-17T16:46:48.031787+0000","last_active":"2024-09-17T16:46:48.031787+0000","last_peered":"2024-09-17T16:46:48.031787+0000","last_clean":"2024-09-17T16:46:48.031787+0000","last_became_active":"2024-09-17T16:29:02.119724+0000","last_became_peered":"2024-09-17T16:29:02.119724+0000","last_unstale":"2024-09-17T16:46:48.031787+0000","last_undegraded":"2024-09-17T16:46:48.031787+0000","last_fullsized":"2024-09-17T16:46:48.031787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:48.031591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:48.031591+0000","last_clean_scrub_stamp":"2024-09-17T16:46:48.031591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:39:43.975893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1390,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:44.313317+0000","last_change":"2024-09-17T16:46:44.313317+0000","last_active":"2024-09-17T16:46:44.313317+0000","last_peered":"2024-09-17T16:46:44.313317+0000","last_clean":"2024-09-17T16:46:44.313317+0000","last_became_active":"2024-09-17T16:29:02.075599+0000","last_became_peered":"2024-09-17T16:29:02.075599+0000","last_unstale":"2024-09-17T16:46:44.313317+0000","last_undegraded":"2024-09-17T16:46:44.313317+0000","last_fullsized":"2024-09-17T16:46:44.313317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:44.313154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:44.313154+0000","last_clean_scrub_stamp":"2024-09-17T16:46:44.313154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:10:47.187502+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1391,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:48.991352+0000","last_change":"2024-09-17T16:46:48.991352+0000","last_active":"2024-09-17T16:46:48.991352+0000","last_peered":"2024-09-17T16:46:48.991352+0000","last_clean":"2024-09-17T16:46:48.991352+0000","last_became_active":"2024-09-17T16:29:02.073908+0000","last_became_peered":"2024-09-17T16:29:02.073908+0000","last_unstale":"2024-09-17T16:46:48.991352+0000","last_undegraded":"2024-09-17T16:46:48.991352+0000","last_fullsized":"2024-09-17T16:46:48.991352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:48.991180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:48.991180+0000","last_clean_scrub_stamp":"2024-09-17T16:46:48.991180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:26:20.812059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1378,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:49.247323+0000","last_change":"2024-09-17T16:46:49.247323+0000","last_active":"2024-09-17T16:46:49.247323+0000","last_peered":"2024-09-17T16:46:49.247323+0000","last_clean":"2024-09-17T16:46:49.247323+0000","last_became_active":"2024-09-17T16:29:02.151449+0000","last_became_peered":"2024-09-17T16:29:02.151449+0000","last_unstale":"2024-09-17T16:46:49.247323+0000","last_undegraded":"2024-09-17T16:46:49.247323+0000","last_fullsized":"2024-09-17T16:46:49.247323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:49.247164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:49.247164+0000","last_clean_scrub_stamp":"2024-09-17T16:46:49.247164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:25:41.268495+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.441739+0000","last_change":"2024-09-17T16:46:43.441739+0000","last_active":"2024-09-17T16:46:43.441739+0000","last_peered":"2024-09-17T16:46:43.441739+0000","last_clean":"2024-09-17T16:46:43.441739+0000","last_became_active":"2024-09-17T16:29:02.157013+0000","last_became_peered":"2024-09-17T16:29:02.157013+0000","last_unstale":"2024-09-17T16:46:43.441739+0000","last_undegraded":"2024-09-17T16:46:43.441739+0000","last_fullsized":"2024-09-17T16:46:43.441739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:43.441633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:43.441633+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.441633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T20:10:07.930540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1383,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:45.494921+0000","last_change":"2024-09-17T16:46:45.494921+0000","last_active":"2024-09-17T16:46:45.494921+0000","last_peered":"2024-09-17T16:46:45.494921+0000","last_clean":"2024-09-17T16:46:45.494921+0000","last_became_active":"2024-09-17T16:29:02.144760+0000","last_became_peered":"2024-09-17T16:29:02.144760+0000","last_unstale":"2024-09-17T16:46:45.494921+0000","last_undegraded":"2024-09-17T16:46:45.494921+0000","last_fullsized":"2024-09-17T16:46:45.494921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:45.494775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:45.494775+0000","last_clean_scrub_stamp":"2024-09-17T16:46:45.494775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:56.480291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1376,"reported_epoch":707,"state":"active+clean","last_fresh":"2024-09-17T16:46:43.048437+0000","last_change":"2024-09-17T16:46:43.048437+0000","last_active":"2024-09-17T16:46:43.048437+0000","last_peered":"2024-09-17T16:46:43.048437+0000","last_clean":"2024-09-17T16:46:43.048437+0000","last_became_active":"2024-09-17T16:29:02.049439+0000","last_became_peered":"2024-09-17T16:29:02.049439+0000","last_unstale":"2024-09-17T16:46:43.048437+0000","last_undegraded":"2024-09-17T16:46:43.048437+0000","last_fullsized":"2024-09-17T16:46:43.048437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T16:46:43.048282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T16:46:43.048282+0000","last_clean_scrub_stamp":"2024-09-17T16:46:43.048282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:03.783503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607784,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607785,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10856,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11116544,"data_stored":6439173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607785,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78132,"kb_used_data":10852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293708,"statfs":{"total":96636764160,"available":96556756992,"internally_reserved":0,"allocated":11112448,"data_stored":6439168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81716,"kb_used_data":10276,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94290124,"statfs":{"total":96636764160,"available":96553086976,"internally_reserved":0,"allocated":10522624,"data_stored":5848819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77552,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294288,"statfs":{"total":96636764160,"available":96557350912,"internally_reserved":0,"allocated":10518528,"data_stored":5848800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640489,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77560,"kb_used_data":10280,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94294280,"statfs":{"total":96636764160,"available":96557342720,"internally_reserved":0,"allocated":10526720,"data_stored":5848824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T16:47:02.978 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: get_monmap_and_config 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: build_initial_monmap 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: monmap: 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:47:03.085659+0000 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:47:03.085659+0000 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:47:03.089 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:0: v1:172.21.15.105:6789/0 mon.noname-a-legacy 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:1: v1:172.21.15.160:6789/0 mon.noname-f-legacy 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:9: v2:172.21.15.105:3300/0 mon.noname-a 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr:10: v2:172.21.15.160:3300/0 mon.noname-f 2024-09-17T16:47:03.090 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 Processor -- start 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 -- start start 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: init 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: _reopen_session rank -1 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient: _add_conns ranks=[9,7,5,3,6,8,2,10,0,4,1] 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-a con 0x7f90dc0a68e0 addr v2:172.21.15.105:3300/0 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 --2- >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 0x7f90dc0a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-i con 0x7f90dc0a4910 addr [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 --2- >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f90dc1396a0 0x7f90dc139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-h con 0x7f90dc1396a0 addr [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] 2024-09-17T16:47:03.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0a3fc0 con 0x7f90dc0a68e0 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 -- --> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] -- mon_getmap magic: 0 -- 0x7f90dc0a63c0 con 0x7f90dc1396a0 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 1 -- --> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] -- mon_getmap magic: 0 -- 0x7f90dc0a7740 con 0x7f90dc0a4910 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): _renew_subs 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e2f90640 10 monclient(hunting): authenticate will time out at 2106.351394s 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38896/0 (socket says 172.21.15.105:38896) 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 -- 172.21.15.105:0/4150252131 learned_addr learned my addr 172.21.15.105:0/4150252131 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): get_auth_request con 0x7f90dc0a68e0 auth_method 0 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:03.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e152d640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f90dc1396a0 0x7f90dc139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90d3fff640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 0x7f90dc0a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e152d640 10 monclient(hunting): get_auth_request con 0x7f90dc1396a0 auth_method 0 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e152d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e152d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e152d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_done global_id 104337 payload 274 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 -- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 msgr2=0x7f90dc0a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:47:03.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 0x7f90dc0a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 -- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f90dc1396a0 msgr2=0x7f90dc139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f90dc1396a0 0x7f90dc139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient: _finish_hunting 0 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 monclient: found mon.noname-a 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.082+0000 7f90e0d2c640 1 -- 172.21.15.105:0/4150252131 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc06ebc0 con 0x7f90dc0a68e0 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d3fff640 10 monclient: get_auth_request con 0x7f90dc0a4910 auth_method 0 2024-09-17T16:47:03.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d3fff640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 0x7f90dc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:47:03.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009870 tx=0x7f90cc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b821ab22769e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 1 -- 172.21.15.105:0/4150252131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90cc0b2070 con 0x7f90dc0a68e0 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: dump: 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:03.102 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: _finish_auth 0 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: _check_auth_tickets 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 1 -- 172.21.15.105:0/4150252131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc0ab7b0 con 0x7f90dc0a68e0 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 5 monclient: authenticate success, global_id 104337 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-17T16:47:03.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: handle_config config(0 keys) 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 10 monclient: get_monmap_and_config success 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 1 -- 172.21.15.105:0/4150252131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90cc0abe50 con 0x7f90dc0a68e0 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 4 set_mon_vals no callback set 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90d37fe640 10 monclient: dump: 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:03.104 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 10 monclient: shutdown 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 -- 172.21.15.105:0/4150252131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc0a68e0 msgr2=0x7f90dc0a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009870 tx=0x7f90cc01c3f0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 -- 172.21.15.105:0/4150252131 shutdown_connections 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] conn(0x7f90dc1396a0 0x7f90dc139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] conn(0x7f90dc0a4910 0x7f90dc0a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 --2- 172.21.15.105:0/4150252131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc0a68e0 0x7f90dc0a6cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 -- 172.21.15.105:0/4150252131 >> 172.21.15.105:0/4150252131 conn(0x7f90dc068ba0 msgr2=0x7f90dc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 -- 172.21.15.105:0/4150252131 shutdown_connections 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.086+0000 7f90e2f90640 1 -- 172.21.15.105:0/4150252131 wait complete. 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: build_initial_monmap 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: monmap: 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 0 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 0.000000 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:created 0.000000 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 0 (unknown) 2024-09-17T16:47:03.106 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 1 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.noname-a 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.noname-f 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.noname-b 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.noname-g 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.noname-c 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.noname-h 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.noname-d 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.noname-i 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.noname-e 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 librados: starting msgr at 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 librados: starting objecter 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 Processor -- start 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- start start 2024-09-17T16:47:03.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 librados: setting wanted keys 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 librados: calling monclient init 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: init 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _reopen_session rank -1 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _add_conns ranks=[2,3,0,6,5,8,7,4,1] 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-b con 0x7f90dc0a4910 addr [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 0x7f90dc1d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-g con 0x7f90dc0a68e0 addr [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 0x7f90dc1d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): picked mon.noname-a con 0x7f90dc1396a0 addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc06e630 con 0x7f90dc1396a0 2024-09-17T16:47:03.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0a7af0 con 0x7f90dc0a4910 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): start opening mon connection 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc143320 con 0x7f90dc0a68e0 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): _renew_subs 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient(hunting): authenticate will time out at 2106.356999s 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 0x7f90dc1d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:37232/0 (socket says 172.21.15.105:37232) 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 -- 172.21.15.105:0/645959689 learned_addr learned my addr 172.21.15.105:0/645959689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): get_auth_request con 0x7f90dc1396a0 auth_method 0 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:03.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): get_auth_request con 0x7f90dc0a4910 auth_method 0 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): _init_auth method 2 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): _init_auth creating new auth 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d3fff640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 0x7f90dc1d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient(hunting): handle_auth_done global_id 104346 payload 824 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 -- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 msgr2=0x7f90dc1d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:03.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 0x7f90dc1d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 -- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 msgr2=0x7f90dc122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient: _finish_hunting 0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 monclient: found mon.noname-a 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 -- 172.21.15.105:0/645959689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc123a50 con 0x7f90dc1396a0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d3fff640 10 monclient: get_auth_request con 0x7f90dc0a68e0 auth_method 0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d3fff640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 0x7f90dc1d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e152d640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 0x7f90dc1d1c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007cf0 tx=0x7f90d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 client.?.objecter ms_handle_connect 0x7f90dc1396a0 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 client.?.objecter resend_mon_ops 2024-09-17T16:47:03.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90d40098b0 con 0x7f90dc1396a0 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: dump: 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:03.112 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: _finish_auth 0 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: _check_auth_tickets 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d400edd0 con 0x7f90dc1396a0 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 5 monclient: authenticate success, global_id 104346 2024-09-17T16:47:03.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _renew_subs 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc123d30 con 0x7f90dc1396a0 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: handle_config config(0 keys) 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90d4014d00 con 0x7f90dc1396a0 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d2ffd640 4 set_mon_vals no callback set 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 monclient: dump: 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:epoch 1 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:fsid 4403e27b-1d4f-4d49-a333-3b7d5cfcb76b 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:last_changed 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:created 2024-09-17T16:27:05.094380+0000 2024-09-17T16:47:03.114 INFO:teuthology.orchestra.run.smithi105.stderr:min_mon_release 19 (squid) 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:election_strategy: 3 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.f 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.b 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:3: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.g 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:4: [v2:172.21.15.105:3302/0,v1:172.21.15.105:6791/0] mon.c 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:5: [v2:172.21.15.160:3302/0,v1:172.21.15.160:6791/0] mon.h 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:6: [v2:172.21.15.105:3303/0,v1:172.21.15.105:6792/0] mon.d 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:7: [v2:172.21.15.160:3303/0,v1:172.21.15.160:6792/0] mon.i 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:8: [v2:172.21.15.105:3304/0,v1:172.21.15.105:6793/0] mon.e 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 client.104346.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _renew_subs 2024-09-17T16:47:03.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc0a9f20 con 0x7f90dc1396a0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 librados: init done 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 48944+0+0 (secure 0 0 0) 0x7f90d401b030 con 0x7f90dc1396a0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc139b50 con 0x7f90dc1396a0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 0x7f90c8040a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(707..707 src has 561..707) ==== 5529+0+0 (secure 0 0 0) 0x7f90d4019020 con 0x7f90dc1396a0 2024-09-17T16:47:03.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 client.104346.objecter ms_dispatch 0x7f90dc000ba0 osd_map(707..707 src has 561..707) 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 3 client.104346.objecter handle_osd_map got epochs [707,707] > 0 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 3 client.104346.objecter handle_osd_map decoding full epoch 707 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 20 client.104346.objecter dump_active .. 0 homeless 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 0x7f90c8040a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 10 monclient: get_auth_request con 0x7f90c803e550 auth_method 0 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90d17fa640 10 client.104346.objecter ms_handle_connect 0x7f90c803e550 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.090+0000 7f90e0d2c640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 0x7f90c8040a10 secure :-1 s=READY pgs=3694 cs=0 l=1 rev1=1 crypto rx=0x7f90cc01c220 tx=0x7f90cc009040 comp rx=0 tx=0).ready entity=mgr.4118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T16:47:03.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.094+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90d4022060 con 0x7f90dc1396a0 2024-09-17T16:47:03.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.094+0000 7f90d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-17T16:47:03.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.094+0000 7f90d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-17T16:47:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.282+0000 7f90e2f90640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-17T16:47:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.282+0000 7f90e2f90640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-17T16:47:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.282+0000 7f90e2f90640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.105:3300/0 2024-09-17T16:47:03.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.282+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f90dc138a00 con 0x7f90dc1396a0 2024-09-17T16:47:03.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f90d4012860 con 0x7f90dc1396a0 2024-09-17T16:47:03.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d17fa640 10 monclient: handle_config config(1 keys) 2024-09-17T16:47:03.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d2ffd640 4 set_mon_vals no callback set 2024-09-17T16:47:03.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d2ffd640 10 set_mon_vals mon_health_to_clog = false 2024-09-17T16:47:03.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d17fa640 1 -- 172.21.15.105:0/645959689 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v4)=0 v4) ==== 123+0+0 (secure 0 0 0) 0x7f90d4012cf0 con 0x7f90dc1396a0 2024-09-17T16:47:03.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-17T16:47:03.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.294+0000 7f90d17fa640 10 monclient: _finish_command 2 = system:0 2024-09-17T16:47:03.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 10 librados: watch_flush enter 2024-09-17T16:47:03.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 10 librados: watch_flush exit 2024-09-17T16:47:03.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 20 client.104346.objecter shutdown clearing up homeless session... 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 10 client.104346.objecter successfully canceled tick 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 msgr2=0x7f90c8040a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 0x7f90c8040a10 secure :-1 s=READY pgs=3694 cs=0 l=1 rev1=1 crypto rx=0x7f90cc01c220 tx=0x7f90cc009040 comp rx=0 tx=0).stop 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 10 monclient: shutdown 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 msgr2=0x7f90dc1d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 0x7f90dc1d1c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90d4007cf0 tx=0x7f90d400e4b0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 shutdown_connections 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:6824/3499101759,v1:172.21.15.160:6825/3499101759] conn(0x7f90c803e550 0x7f90c8040a10 unknown :-1 s=CLOSED pgs=3694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90dc1396a0 0x7f90dc1d1c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90dc0a68e0 0x7f90dc1d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 --2- 172.21.15.105:0/645959689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90dc0a4910 0x7f90dc122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 >> 172.21.15.105:0/645959689 conn(0x7f90dc068ba0 msgr2=0x7f90dc0aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 shutdown_connections 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 -- 172.21.15.105:0/645959689 wait complete. 2024-09-17T16:47:03.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-17T16:47:03.306+0000 7f90e2f90640 1 librados: shutdown 2024-09-17T16:47:03.335 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T16:47:03.335 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T16:47:03.336 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T16:47:03.336 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:03.427 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T16:47:03.427 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T16:47:03.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:03.482 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T16:47:03.483 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T16:47:03.483 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:03.581 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T16:47:03.582 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T16:47:03.582 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:07.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-17T16:47:09.684 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T16:47:09.684 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T16:47:09.684 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:09.743 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T16:47:09.743 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T16:47:09.743 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:09.829 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T16:47:09.829 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T16:47:09.829 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T16:47:09.829 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:09.860 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T16:47:09.860 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T16:47:09.860 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T16:47:09.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:09.884 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T16:47:09.884 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T16:47:09.884 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:09.961 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T16:47:09.961 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T16:47:09.961 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.034 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T16:47:10.034 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-17T16:47:10.034 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.090 INFO:tasks.ceph.mon.d:Stopped 2024-09-17T16:47:10.091 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-17T16:47:10.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.171 INFO:tasks.ceph.mon.e:Stopped 2024-09-17T16:47:10.171 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-17T16:47:10.171 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.200 INFO:tasks.ceph.mon.f:Stopped 2024-09-17T16:47:10.200 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-17T16:47:10.200 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.282 INFO:tasks.ceph.mon.g:Stopped 2024-09-17T16:47:10.282 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-17T16:47:10.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.349 INFO:tasks.ceph.mon.h:Stopped 2024-09-17T16:47:10.349 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-17T16:47:10.349 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T16:47:10.424 INFO:tasks.ceph.mon.i:Stopped 2024-09-17T16:47:10.424 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T16:47:10.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(MON_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'slow request' | head -n 1 2024-09-17T16:47:10.472 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:47:10.472 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T16:47:10.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:47:10.739 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T16:47:10.964 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:47:10.965 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T16:47:11.138 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:47:11.139 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T16:47:11.387 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:47:11.387 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T16:47:11.580 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:47:11.580 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T16:47:11.763 INFO:tasks.ceph:Archiving mon data... 2024-09-17T16:47:11.764 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.a.tgz 2024-09-17T16:47:11.764 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:47:11.770 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.8xuAXRqxWv 2024-09-17T16:47:11.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.8xuAXRqxWv 2024-09-17T16:47:12.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.8xuAXRqxWv 2024-09-17T16:47:12.122 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.8xuAXRqxWv is 1MB 2024-09-17T16:47:12.194 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.8xuAXRqxWv 2024-09-17T16:47:12.199 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.b.tgz 2024-09-17T16:47:12.199 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:47:12.247 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.zvvz3lP2Nl 2024-09-17T16:47:12.248 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.zvvz3lP2Nl 2024-09-17T16:47:12.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-17T16:47:12.524 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.zvvz3lP2Nl 2024-09-17T16:47:12.590 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.zvvz3lP2Nl is 1MB 2024-09-17T16:47:12.661 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.zvvz3lP2Nl 2024-09-17T16:47:12.666 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.c.tgz 2024-09-17T16:47:12.666 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:47:12.711 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.T8KVXdnsEn 2024-09-17T16:47:12.712 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.T8KVXdnsEn 2024-09-17T16:47:12.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.T8KVXdnsEn 2024-09-17T16:47:13.063 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.T8KVXdnsEn is 1MB 2024-09-17T16:47:13.134 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.T8KVXdnsEn 2024-09-17T16:47:13.139 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.d.tgz 2024-09-17T16:47:13.139 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:47:13.187 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.U2NyIZqYGj 2024-09-17T16:47:13.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.U2NyIZqYGj 2024-09-17T16:47:13.474 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.U2NyIZqYGj 2024-09-17T16:47:13.539 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.U2NyIZqYGj is 1MB 2024-09-17T16:47:13.609 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.U2NyIZqYGj 2024-09-17T16:47:13.614 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.e.tgz 2024-09-17T16:47:13.615 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-17T16:47:13.663 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.Z45qnqfNRB 2024-09-17T16:47:13.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.Z45qnqfNRB 2024-09-17T16:47:13.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.Z45qnqfNRB 2024-09-17T16:47:14.015 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.Z45qnqfNRB is 1MB 2024-09-17T16:47:14.086 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.Z45qnqfNRB 2024-09-17T16:47:14.091 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.f.tgz 2024-09-17T16:47:14.091 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-17T16:47:14.097 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.8HQqosL7Uc 2024-09-17T16:47:14.097 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.8HQqosL7Uc 2024-09-17T16:47:14.381 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.8HQqosL7Uc 2024-09-17T16:47:14.444 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.8HQqosL7Uc is 1MB 2024-09-17T16:47:14.514 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.8HQqosL7Uc 2024-09-17T16:47:14.519 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.g.tgz 2024-09-17T16:47:14.519 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-17T16:47:14.568 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.9h2k7n0nbw 2024-09-17T16:47:14.568 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.9h2k7n0nbw 2024-09-17T16:47:14.854 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.9h2k7n0nbw 2024-09-17T16:47:14.921 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.9h2k7n0nbw is 1MB 2024-09-17T16:47:14.990 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.9h2k7n0nbw 2024-09-17T16:47:14.996 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.h.tgz 2024-09-17T16:47:14.996 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-17T16:47:15.045 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.VRXoAQER60 2024-09-17T16:47:15.045 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.VRXoAQER60 2024-09-17T16:47:15.312 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.VRXoAQER60 2024-09-17T16:47:15.375 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.VRXoAQER60 is 1MB 2024-09-17T16:47:15.447 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.VRXoAQER60 2024-09-17T16:47:15.452 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/data/mon.i.tgz 2024-09-17T16:47:15.452 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-09-17T16:47:15.500 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.ccnKNAdNiC 2024-09-17T16:47:15.501 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.ccnKNAdNiC 2024-09-17T16:47:15.767 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.ccnKNAdNiC 2024-09-17T16:47:15.828 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.ccnKNAdNiC is 1MB 2024-09-17T16:47:15.939 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.ccnKNAdNiC 2024-09-17T16:47:15.945 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T16:47:15.945 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T16:47:15.947 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T16:47:16.012 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T16:47:16.048 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T16:47:16.049 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T16:47:16.052 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi105/crash 2024-09-17T16:47:16.053 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T16:47:16.068 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi160/crash 2024-09-17T16:47:16.069 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T16:47:16.106 INFO:tasks.ceph:Compressing logs... 2024-09-17T16:47:16.106 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T16:47:16.112 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T16:47:16.168 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29453.log 2024-09-17T16:47:16.169 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29612.log 2024-09-17T16:47:16.169 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30793.log 2024-09-17T16:47:16.170 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29453.log: /var/log/ceph/ceph-client.admin.29612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29999.log 2024-09-17T16:47:16.170 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29871.log 2024-09-17T16:47:16.171 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29762.log 2024-09-17T16:47:16.171 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29871.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-09-17T16:47:16.173 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29518.log 2024-09-17T16:47:16.173 INFO:teuthology.orchestra.run.smithi160.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.30793.log.gz/var/log/ceph/ceph-mon.f.log: 2024-09-17T16:47:16.173 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30684.log 2024-09-17T16:47:16.173 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29518.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.29612.log.gz 2024-09-17T16:47:16.173 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29380.log 2024-09-17T16:47:16.174 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-09-17T16:47:16.175 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29380.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.29380.log.gz 2024-09-17T16:47:16.176 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30796.log 2024-09-17T16:47:16.177 INFO:teuthology.orchestra.run.smithi160.stderr: 91.4%/var/log/ceph/ceph-client.admin.29673.log: -- replaced with /var/log/ceph/ceph-client.admin.30684.log.gz 2024-09-17T16:47:16.177 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30120.log 2024-09-17T16:47:16.179 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30796.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.29999.log.gz 2024-09-17T16:47:16.179 INFO:teuthology.orchestra.run.smithi160.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.29871.log.gz 2024-09-17T16:47:16.180 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30686.log 2024-09-17T16:47:16.180 INFO:teuthology.orchestra.run.smithi160.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.29762.log.gz 2024-09-17T16:47:16.180 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31088.log 2024-09-17T16:47:16.181 INFO:teuthology.orchestra.run.smithi160.stderr: 91.6%/var/log/ceph/ceph-client.admin.30686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30737.log 2024-09-17T16:47:16.182 INFO:teuthology.orchestra.run.smithi160.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.30796.log.gz 2024-09-17T16:47:16.182 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29518.log.gz 2024-09-17T16:47:16.182 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29598.log 2024-09-17T16:47:16.182 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29504.log 2024-09-17T16:47:16.183 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29598.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-09-17T16:47:16.184 INFO:teuthology.orchestra.run.smithi160.stderr: 91.2% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.30120.log.gz 2024-09-17T16:47:16.185 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31088.log.gz 2024-09-17T16:47:16.185 INFO:teuthology.orchestra.run.smithi160.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.30686.log.gz 2024-09-17T16:47:16.185 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-09-17T16:47:16.185 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30203.log 2024-09-17T16:47:16.185 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.30737.log.gz 2024-09-17T16:47:16.186 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29356.log 2024-09-17T16:47:16.186 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25407.log 2024-09-17T16:47:16.186 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7%/var/log/ceph/ceph-client.admin.29356.log: -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz 2024-09-17T16:47:16.186 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30378.log 2024-09-17T16:47:16.187 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25383.log 2024-09-17T16:47:16.187 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25407.log: /var/log/ceph/ceph-client.admin.30378.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.25407.log.gz 2024-09-17T16:47:16.188 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29955.log 2024-09-17T16:47:16.189 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25383.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25383.log.gz 2024-09-17T16:47:16.189 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29825.log 2024-09-17T16:47:16.191 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29955.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29955.log.gz 2024-09-17T16:47:16.191 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30357.log 2024-09-17T16:47:16.194 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29825.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29598.log.gz 2024-09-17T16:47:16.195 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-09-17T16:47:16.197 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30357.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30357.log.gz 2024-09-17T16:47:16.198 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-09-17T16:47:16.201 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31004.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-09-17T16:47:16.203 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T16:47:16.204 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30591.log 2024-09-17T16:47:16.204 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30313.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.29825.log.gz 2024-09-17T16:47:16.205 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T16:47:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32240.log 2024-09-17T16:47:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35240.log 2024-09-17T16:47:16.206 INFO:teuthology.orchestra.run.smithi105.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30591.log.gz 2024-09-17T16:47:16.206 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32344.log 2024-09-17T16:47:16.220 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.3.log: 91.2% 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30203.log.gz 2024-09-17T16:47:16.221 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-09-17T16:47:16.222 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29425.log 2024-09-17T16:47:16.222 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30685.log 2024-09-17T16:47:16.225 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29425.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29425.log.gz 2024-09-17T16:47:16.226 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31003.log 2024-09-17T16:47:16.226 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30685.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.30685.log.gz 2024-09-17T16:47:16.226 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29802.log 2024-09-17T16:47:16.230 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31003.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31003.log.gz 2024-09-17T16:47:16.230 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30736.log 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29802.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.29802.log.gz 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29884.log 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30736.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30378.log.gz 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30266.log 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29884.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.30736.log.gz 2024-09-17T16:47:16.239 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30237.log 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35619.log 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32240.log.gz 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35240.log: /var/log/ceph/ceph-client.admin.32344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30998.log 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32344.log.gz 2024-09-17T16:47:16.246 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35240.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35619.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36922.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30920.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29391.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30998.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29346.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30920.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32671.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36922.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30197.log 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.30920.log.gz 2024-09-17T16:47:16.247 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33061.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32671.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29346.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31642.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33061.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32671.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31466.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30197.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33243.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33061.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32474.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31466.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31642.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30431.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30093.log 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31466.log.gz 2024-09-17T16:47:16.248 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33243.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30431.log: gzip -5 --verbose -- 89.7% /var/log/ceph/ceph-client.admin.33009.log -- replaced with /var/log/ceph/ceph-client.admin.32474.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30093.log: /var/log/ceph/ceph-client.admin.33009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32775.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30431.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33477.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30093.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32775.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33009.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36296.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29670.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32775.log.gz 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30067.log 2024-09-17T16:47:16.249 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33477.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35188.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37061.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36296.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29670.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32697.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30067.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35749.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35188.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29863.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32697.log: /var/log/ceph/ceph-client.admin.35749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30379.log 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.37061.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29863.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32697.log.gz 2024-09-17T16:47:16.250 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31980.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30379.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35749.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31746.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29863.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30487.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30379.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36948.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31980.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30487.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36948.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31746.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30972.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30487.log.gz 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31668.log 2024-09-17T16:47:16.251 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30972.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36948.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21515.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33347.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30972.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.tmp-client.admin.21515.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21515.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34796.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33347.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31668.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33560.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37790.log: /var/log/ceph/ceph-client.admin.34796.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33347.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35333.log 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2024-09-17T16:47:16.252 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33560.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34796.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29515.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37192.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33560.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29671.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5/var/log/ceph/ceph-client.admin.37192.log: --verbose -- /var/log/ceph/ceph-client.admin.32136.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29515.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29671.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.35333.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35437.log 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32136.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37192.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29671.log.gz 2024-09-17T16:47:16.253 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32801.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36740.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35437.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32136.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29321.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34484.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35437.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29513.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32801.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34484.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36740.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37712.log 88.7% 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29321.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35463.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34484.log.gz 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31336.log 2024-09-17T16:47:16.254 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29513.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35671.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35463.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37712.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31024.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35671.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35463.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35671.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr: 92.2%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.31336.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34510.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31024.log.gz 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30764.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37452.log: /var/log/ceph/ceph-client.admin.34510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31362.log 2024-09-17T16:47:16.255 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32032.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37166.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30764.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29506.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32032.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31362.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36896.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29506.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32032.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35108.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37166.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33612.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29506.log.gz 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35935.log 2024-09-17T16:47:16.256 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36896.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30868.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35082.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35935.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35108.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30868.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33612.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35082.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35935.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34926.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30868.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37348.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35082.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31284.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34926.log.gz 2024-09-17T16:47:16.257 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.31284.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31954.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31284.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34136.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31954.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29514.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz/var/log/ceph/ceph-client.admin.37244.log: 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36400.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34136.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37114.log 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-09-17T16:47:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29514.log.gz 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34162.log 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36400.log.gz 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30249.log 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37114.log.gz 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31720.log 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30249.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37764.log 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34162.log.gz 2024-09-17T16:47:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-09-17T16:47:16.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30266.log: 92.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30287.log 2024-09-17T16:47:16.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30237.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.29356.log.gz 2024-09-17T16:47:16.264 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30266.log.gz 2024-09-17T16:47:16.264 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29372.log 2024-09-17T16:47:16.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30287.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.30287.log.gz 2024-09-17T16:47:16.264 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25503.log 2024-09-17T16:47:16.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29372.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-09-17T16:47:16.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25503.log: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.29372.log.gz 2024-09-17T16:47:16.273 INFO:teuthology.orchestra.run.smithi160.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25503.log.gz 2024-09-17T16:47:16.273 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30735.log 2024-09-17T16:47:16.277 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mon.i.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.30237.log.gz 2024-09-17T16:47:16.283 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.29884.log.gz 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25479.log 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30735.log: /var/log/ceph/ceph-mon.g.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.30735.log.gz 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30058.log 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.25479.log.gz 2024-09-17T16:47:16.293 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31720.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30249.log.gz 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36039.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.31720.log.gz 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37764.log.gz 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31902.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34188.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36039.log.gz 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30353.log 2024-09-17T16:47:16.296 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30171.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31902.log.gz/var/log/ceph/ceph-client.admin.30353.log: 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36091.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34188.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30171.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30353.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30171.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33191.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31798.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31798.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36091.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33690.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36766.log 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33191.log.gz 2024-09-17T16:47:16.297 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31180.log: /var/log/ceph/ceph-client.admin.33690.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31798.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33690.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36374.log 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36688.log 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29858.log 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36532.log 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36766.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29858.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36374.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36688.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29858.log.gz 2024-09-17T16:47:16.298 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29668.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30565.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34614.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36532.log.gz 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30565.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29668.log.gz 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30565.log.gz 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36818.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31492.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34614.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33373.log 2024-09-17T16:47:16.299 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36818.log: 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31492.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31492.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.36818.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30327.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35385.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30842.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35515.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33373.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30842.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30327.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35385.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30842.log.gz 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34456.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-17T16:47:16.300 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37140.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36504.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35515.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37140.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34456.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31310.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37140.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35853.log 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36504.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31310.log: /var/log/ceph/ceph-client.admin.36584.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-09-17T16:47:16.301 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34978.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31544.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31310.log.gz 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37892.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33898.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35853.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34978.log.gz 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31544.log: /var/log/ceph/ceph-client.admin.37892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36714.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30015.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37892.log.gz 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31544.log.gz/var/log/ceph/ceph-client.admin.36714.log: 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34666.log 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30015.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.33898.log.gz 2024-09-17T16:47:16.302 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32058.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36714.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34666.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30015.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33638.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32058.log: /var/log/ceph/ceph-client.admin.33794.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34666.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz -- replaced with /var/log/ceph/ceph-client.admin.32058.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29669.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34430.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36270.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33638.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29669.log.gz 2024-09-17T16:47:16.303 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31076.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36270.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34430.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33217.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29651.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33217.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36270.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31076.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32957.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33217.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30041.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34002.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29651.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32957.log.gz 2024-09-17T16:47:16.304 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29371.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30041.log.gz/var/log/ceph/ceph-client.admin.34002.log: /var/log/ceph/ceph-client.admin.37218.log: 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32370.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29371.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34002.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32905.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29371.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31850.log 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.32370.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32905.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36870.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-09-17T16:47:16.305 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35162.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35056.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31850.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30617.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35056.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30223.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35162.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30617.log: /var/log/ceph/ceph-client.admin.30223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36322.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35056.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.1.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30617.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30223.log.gz 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35030.log 2024-09-17T16:47:16.306 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33872.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30119.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36322.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33872.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35030.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33872.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37686.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30119.log: /var/log/ceph/ceph-client.admin.37686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29869.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34588.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30119.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37686.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35359.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33820.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29989.log 2024-09-17T16:47:16.307 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35359.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29869.log.gz 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33820.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35359.log.gz 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33820.log.gz 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34028.log 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30145.log 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6%/var/log/ceph/ceph-mon.d.log: -- replaced with /var/log/ceph/ceph-client.admin.29989.log.gz 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34028.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34028.log.gz 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32266.log 2024-09-17T16:47:16.308 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25209.log 2024-09-17T16:47:16.309 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30058.log: 90.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31087.log 2024-09-17T16:47:16.314 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.30058.log.gz 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32266.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30145.log.gz 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32266.log.gz 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29443.log 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31440.log 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.25209.log: /var/log/ceph/ceph-client.admin.29443.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29443.log.gz 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.25209.log.gz 2024-09-17T16:47:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34240.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31440.log: /var/log/ceph/ceph-client.admin.33087.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31440.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31232.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34110.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34240.log: /var/log/ceph/ceph-client.admin.31232.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34240.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31232.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30894.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34110.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34110.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34352.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35305.log 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30894.log: /var/log/ceph/ceph-client.admin.34352.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30894.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34352.log.gz 2024-09-17T16:47:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34900.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35305.log: /var/log/ceph/ceph-client.admin.34900.log: 89.6% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35305.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34900.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33113.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30790.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37738.log: /var/log/ceph/ceph-client.admin.33113.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33113.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30790.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30790.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32983.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36610.log 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32983.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-09-17T16:47:16.320 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32983.log.gz 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35987.log 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31050.log 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35987.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36610.log.gz 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35987.log.gz 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35004.log 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31050.log: /var/log/ceph/ceph.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31050.log.gz 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35004.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.35004.log.gz 2024-09-17T16:47:16.321 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29247.log 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34266.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31616.log 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29247.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29247.log.gz 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35411.log 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31616.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.31616.log.gz 2024-09-17T16:47:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34214.log 2024-09-17T16:47:16.324 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T16:47:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35411.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35411.log.gz 2024-09-17T16:47:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30539.log 2024-09-17T16:47:16.326 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34214.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34214.log.gz 2024-09-17T16:47:16.326 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2024-09-17T16:47:16.327 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31087.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.31087.log.gz 2024-09-17T16:47:16.328 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30539.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30539.log.gz 2024-09-17T16:47:16.328 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-09-17T16:47:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36244.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2024-09-17T16:47:16.330 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-09-17T16:47:16.331 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35775.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-09-17T16:47:16.331 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29661.log 2024-09-17T16:47:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32162.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-09-17T16:47:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29963.log 2024-09-17T16:47:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29661.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29661.log.gz 2024-09-17T16:47:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31128.log 2024-09-17T16:47:16.336 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29963.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29963.log.gz 2024-09-17T16:47:16.336 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32188.log 2024-09-17T16:47:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31128.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31128.log.gz 2024-09-17T16:47:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30738.log 2024-09-17T16:47:16.339 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32188.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32188.log.gz 2024-09-17T16:47:16.340 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29898.log 2024-09-17T16:47:16.340 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33035.log 2024-09-17T16:47:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30738.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30738.log.gz 2024-09-17T16:47:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31258.log 2024-09-17T16:47:16.343 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33035.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33035.log.gz 2024-09-17T16:47:16.343 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35961.log 2024-09-17T16:47:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31258.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31258.log.gz 2024-09-17T16:47:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37530.log 2024-09-17T16:47:16.346 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35961.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35961.log.gz 2024-09-17T16:47:16.346 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31772.log 2024-09-17T16:47:16.348 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37530.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37530.log.gz 2024-09-17T16:47:16.348 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34874.log 2024-09-17T16:47:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31772.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31772.log.gz 2024-09-17T16:47:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30275.log 2024-09-17T16:47:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34874.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34874.log.gz 2024-09-17T16:47:16.351 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-09-17T16:47:16.352 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29707.log 2024-09-17T16:47:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30275.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30275.log.gz 2024-09-17T16:47:16.353 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29837.log 2024-09-17T16:47:16.353 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29898.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29898.log.gz 2024-09-17T16:47:16.354 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32879.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz 2024-09-17T16:47:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36218.log 2024-09-17T16:47:16.356 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29837.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29837.log.gz 2024-09-17T16:47:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T16:47:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36218.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36218.log.gz 2024-09-17T16:47:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30816.log 2024-09-17T16:47:16.360 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29931.log 2024-09-17T16:47:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32645.log 2024-09-17T16:47:16.376 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29707.log: 92.1%gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-09-17T16:47:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30816.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30816.log.gz 2024-09-17T16:47:16.379 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29931.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.29931.log.gz 2024-09-17T16:47:16.379 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29707.log.gz 2024-09-17T16:47:16.388 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30791.log 2024-09-17T16:47:16.395 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30513.log 2024-09-17T16:47:16.396 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32645.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32645.log.gz 2024-09-17T16:47:16.403 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29509.log 2024-09-17T16:47:16.404 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30513.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30513.log.gz 2024-09-17T16:47:16.404 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mon.h.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29682.log 2024-09-17T16:47:16.407 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30791.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.30791.log.gz 2024-09-17T16:47:16.411 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32500.log 2024-09-17T16:47:16.412 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29509.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29509.log.gz 2024-09-17T16:47:16.412 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29364.log 2024-09-17T16:47:16.419 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32422.log 2024-09-17T16:47:16.420 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32500.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32500.log.gz 2024-09-17T16:47:16.427 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33664.log 2024-09-17T16:47:16.428 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32422.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32422.log.gz 2024-09-17T16:47:16.428 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31006.log 2024-09-17T16:47:16.434 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29364.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.29364.log.gz 2024-09-17T16:47:16.435 INFO:teuthology.orchestra.run.smithi160.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.29682.log.gz 2024-09-17T16:47:16.435 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37660.log 2024-09-17T16:47:16.436 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33664.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33664.log.gz 2024-09-17T16:47:16.443 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30301.log 2024-09-17T16:47:16.444 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37660.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37660.log.gz 2024-09-17T16:47:16.448 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25455.log 2024-09-17T16:47:16.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32292.log 2024-09-17T16:47:16.452 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30301.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30301.log.gz 2024-09-17T16:47:16.452 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31006.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.31006.log.gz 2024-09-17T16:47:16.459 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32110.log 2024-09-17T16:47:16.460 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32292.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32292.log.gz 2024-09-17T16:47:16.465 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30138.log 2024-09-17T16:47:16.465 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25455.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.25455.log.gz 2024-09-17T16:47:16.467 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36013.log 2024-09-17T16:47:16.468 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32110.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32110.log.gz 2024-09-17T16:47:16.475 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33742.log 2024-09-17T16:47:16.476 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36013.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36013.log.gz 2024-09-17T16:47:16.476 INFO:teuthology.orchestra.run.smithi105.stderr: 90.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T16:47:16.477 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32084.log 2024-09-17T16:47:16.478 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33742.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33742.log.gz 2024-09-17T16:47:16.478 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32853.log 2024-09-17T16:47:16.479 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32084.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.32084.log.gz 2024-09-17T16:47:16.480 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T16:47:16.480 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31824.log 2024-09-17T16:47:16.481 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32853.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32853.log.gz 2024-09-17T16:47:16.481 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34952.log 2024-09-17T16:47:16.482 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30138.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.30138.log.gz 2024-09-17T16:47:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31824.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31824.log.gz 2024-09-17T16:47:16.483 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37504.log 2024-09-17T16:47:16.484 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34952.log.gz 2024-09-17T16:47:16.484 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35881.log 2024-09-17T16:47:16.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37504.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37504.log.gz 2024-09-17T16:47:16.486 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33534.log 2024-09-17T16:47:16.487 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35881.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.35881.log.gz 2024-09-17T16:47:16.488 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29811.log 2024-09-17T16:47:16.488 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31002.log 2024-09-17T16:47:16.489 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33534.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33534.log.gz 2024-09-17T16:47:16.489 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T16:47:16.490 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29811.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29811.log.gz 2024-09-17T16:47:16.491 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33139.log 2024-09-17T16:47:16.504 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30121.log 2024-09-17T16:47:16.507 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31002.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31002.log.gz 2024-09-17T16:47:16.507 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33165.log 2024-09-17T16:47:16.508 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33139.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33139.log.gz 2024-09-17T16:47:16.516 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29940.log 2024-09-17T16:47:16.517 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30121.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30121.log.gz 2024-09-17T16:47:16.519 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30946.log 2024-09-17T16:47:16.519 INFO:teuthology.orchestra.run.smithi160.stderr: 96.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T16:47:16.520 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33165.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33165.log.gz 2024-09-17T16:47:16.529 INFO:teuthology.orchestra.run.smithi105.stderr: 96.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35489.log 2024-09-17T16:47:16.530 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30946.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30946.log.gz 2024-09-17T16:47:16.530 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T16:47:16.531 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35541.log 2024-09-17T16:47:16.532 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29932.log 2024-09-17T16:47:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35489.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35489.log.gz 2024-09-17T16:47:16.533 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-09-17T16:47:16.533 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29940.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29940.log.gz 2024-09-17T16:47:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35541.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35541.log.gz 2024-09-17T16:47:16.534 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25520.log 2024-09-17T16:47:16.535 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37374.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-09-17T16:47:16.535 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-09-17T16:47:16.536 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.25520.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.25520.log.gz 2024-09-17T16:47:16.537 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33321.log 2024-09-17T16:47:16.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37478.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-09-17T16:47:16.538 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33425.log 2024-09-17T16:47:16.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33321.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33321.log.gz 2024-09-17T16:47:16.540 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2024-09-17T16:47:16.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33425.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33425.log.gz 2024-09-17T16:47:16.541 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35909.log 2024-09-17T16:47:16.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2024-09-17T16:47:16.542 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30405.log 2024-09-17T16:47:16.544 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35909.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35909.log.gz 2024-09-17T16:47:16.544 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34082.log 2024-09-17T16:47:16.545 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30405.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30405.log.gz 2024-09-17T16:47:16.546 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-09-17T16:47:16.547 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34082.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.34082.log.gz 2024-09-17T16:47:16.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35827.log 2024-09-17T16:47:16.548 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29393.log 2024-09-17T16:47:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33924.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-09-17T16:47:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32006.log 2024-09-17T16:47:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35827.log.gz 2024-09-17T16:47:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-09-17T16:47:16.551 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29932.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.29932.log.gz 2024-09-17T16:47:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32006.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32006.log.gz 2024-09-17T16:47:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31518.log 2024-09-17T16:47:16.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34054.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 2024-09-17T16:47:16.553 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32931.log 2024-09-17T16:47:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31518.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31518.log.gz 2024-09-17T16:47:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31388.log 2024-09-17T16:47:16.556 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32931.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32931.log.gz 2024-09-17T16:47:16.557 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37868.log 2024-09-17T16:47:16.558 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31388.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31388.log.gz 2024-09-17T16:47:16.558 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29503.log 2024-09-17T16:47:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37868.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37868.log.gz 2024-09-17T16:47:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32749.log 2024-09-17T16:47:16.560 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25431.log 2024-09-17T16:47:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29503.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29503.log.gz 2024-09-17T16:47:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33768.log 2024-09-17T16:47:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32749.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32749.log.gz 2024-09-17T16:47:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37087.log 2024-09-17T16:47:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33768.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33768.log.gz 2024-09-17T16:47:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33586.log 2024-09-17T16:47:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37087.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37087.log.gz 2024-09-17T16:47:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33399.log 2024-09-17T16:47:16.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33586.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33586.log.gz 2024-09-17T16:47:16.567 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33508.log 2024-09-17T16:47:16.568 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33399.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33399.log.gz 2024-09-17T16:47:16.568 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36636.log 2024-09-17T16:47:16.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33508.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33508.log.gz 2024-09-17T16:47:16.569 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37816.log 2024-09-17T16:47:16.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36636.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36636.log.gz 2024-09-17T16:47:16.571 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37296.log 2024-09-17T16:47:16.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37816.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37816.log.gz 2024-09-17T16:47:16.572 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32396.log 2024-09-17T16:47:16.573 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37296.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37296.log.gz 2024-09-17T16:47:16.574 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33976.log 2024-09-17T16:47:16.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32396.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32396.log.gz 2024-09-17T16:47:16.575 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35136.log 2024-09-17T16:47:16.577 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33976.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33976.log.gz 2024-09-17T16:47:16.577 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36478.log 2024-09-17T16:47:16.578 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35136.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35136.log.gz 2024-09-17T16:47:16.578 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33716.log 2024-09-17T16:47:16.579 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29393.log: 91.4% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.29393.log.gz -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T16:47:16.580 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:47:16.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36478.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36478.log.gz 2024-09-17T16:47:16.580 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31928.log 2024-09-17T16:47:16.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33716.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33716.log.gz 2024-09-17T16:47:16.582 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25443.log 2024-09-17T16:47:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31928.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31928.log.gz 2024-09-17T16:47:16.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36974.log 2024-09-17T16:47:16.584 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.25443.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25443.log.gz 2024-09-17T16:47:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32448.log 2024-09-17T16:47:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36974.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36974.log.gz 2024-09-17T16:47:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-09-17T16:47:16.588 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32448.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32448.log.gz 2024-09-17T16:47:16.588 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34692.log 2024-09-17T16:47:16.589 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29348.log 2024-09-17T16:47:16.589 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.25431.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.25431.log.gz 2024-09-17T16:47:16.590 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35801.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-09-17T16:47:16.590 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T16:47:16.591 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34692.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34692.log.gz 2024-09-17T16:47:16.591 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31206.log 2024-09-17T16:47:16.599 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-17T16:47:16.600 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31206.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31206.log.gz 2024-09-17T16:47:16.609 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30792.log 2024-09-17T16:47:16.615 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32827.log 2024-09-17T16:47:16.616 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35697.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-17T16:47:16.616 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30457.log 2024-09-17T16:47:16.618 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32827.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.32827.log.gz 2024-09-17T16:47:16.620 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31089.log 2024-09-17T16:47:16.621 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30792.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30792.log.gz 2024-09-17T16:47:16.627 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37270.log 2024-09-17T16:47:16.629 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30457.log: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30457.log.gz 2024-09-17T16:47:16.637 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29939.log 2024-09-17T16:47:16.639 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31089.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31089.log.gz 2024-09-17T16:47:16.647 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31154.log 2024-09-17T16:47:16.648 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37270.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37270.log.gz 2024-09-17T16:47:16.655 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32536.log 2024-09-17T16:47:16.656 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31154.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31154.log.gz 2024-09-17T16:47:16.661 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30349.log 2024-09-17T16:47:16.665 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29939.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.29939.log.gz 2024-09-17T16:47:16.665 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35567.log 2024-09-17T16:47:16.665 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32536.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32536.log.gz 2024-09-17T16:47:16.671 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37582.log 2024-09-17T16:47:16.672 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35567.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35567.log.gz 2024-09-17T16:47:16.676 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29388.log 2024-09-17T16:47:16.677 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30349.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.29348.log.gz 2024-09-17T16:47:16.679 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31876.log 2024-09-17T16:47:16.680 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37582.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37582.log.gz 2024-09-17T16:47:16.685 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T16:47:16.686 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-09-17T16:47:16.686 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31005.log 2024-09-17T16:47:16.687 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37608.log 2024-09-17T16:47:16.688 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31876.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31876.log.gz 2024-09-17T16:47:16.695 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31694.log 2024-09-17T16:47:16.696 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37608.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37608.log.gz 2024-09-17T16:47:16.696 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30797.log 2024-09-17T16:47:16.699 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.31005.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31005.log.gz 2024-09-17T16:47:16.700 INFO:teuthology.orchestra.run.smithi160.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.30349.log.gz 2024-09-17T16:47:16.703 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30712.log 2024-09-17T16:47:16.704 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31694.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31694.log.gz 2024-09-17T16:47:16.707 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.4.log: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.29504.log.gz 2024-09-17T16:47:16.707 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29620.log 2024-09-17T16:47:16.708 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.30797.log: /var/log/ceph/ceph-client.admin.29470.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30797.log.gz 2024-09-17T16:47:16.709 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29563.log 2024-09-17T16:47:16.711 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34378.log 2024-09-17T16:47:16.712 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29620.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.29620.log.gz 2024-09-17T16:47:16.712 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30712.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30712.log.gz 2024-09-17T16:47:16.719 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.29563.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.29563.log.gz 2024-09-17T16:47:16.719 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33846.log 2024-09-17T16:47:16.720 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34378.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34378.log.gz 2024-09-17T16:47:16.731 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36065.log 2024-09-17T16:47:16.732 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33846.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33846.log.gz 2024-09-17T16:47:16.751 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36426.log 2024-09-17T16:47:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36065.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36065.log.gz 2024-09-17T16:47:16.759 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34536.log 2024-09-17T16:47:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36426.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36426.log.gz 2024-09-17T16:47:16.765 INFO:teuthology.orchestra.run.smithi160.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.29388.log.gz 2024-09-17T16:47:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32214.log 2024-09-17T16:47:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34536.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34536.log.gz 2024-09-17T16:47:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33295.log 2024-09-17T16:47:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32214.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32214.log.gz 2024-09-17T16:47:16.791 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-09-17T16:47:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33295.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33295.log.gz 2024-09-17T16:47:16.799 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30643.log 2024-09-17T16:47:16.800 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34562.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-09-17T16:47:16.807 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29807.log 2024-09-17T16:47:16.808 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.30643.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30643.log.gz 2024-09-17T16:47:16.814 INFO:teuthology.orchestra.run.smithi160.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-09-17T16:47:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35593.log 2024-09-17T16:47:16.815 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29807.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29807.log.gz 2024-09-17T16:47:16.823 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32318.log 2024-09-17T16:47:16.824 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35593.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35593.log.gz 2024-09-17T16:47:16.831 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32723.log 2024-09-17T16:47:16.831 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32318.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32318.log.gz 2024-09-17T16:47:16.839 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36844.log 2024-09-17T16:47:16.840 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.32723.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32723.log.gz 2024-09-17T16:47:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34848.log 2024-09-17T16:47:16.852 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36844.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36844.log.gz 2024-09-17T16:47:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29469.log 2024-09-17T16:47:16.872 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34848.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34848.log.gz 2024-09-17T16:47:16.879 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33950.log 2024-09-17T16:47:16.880 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29469.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29469.log.gz 2024-09-17T16:47:16.887 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36662.log 2024-09-17T16:47:16.888 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33950.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33950.log.gz 2024-09-17T16:47:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36792.log 2024-09-17T16:47:16.896 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36662.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36662.log.gz 2024-09-17T16:47:16.903 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36452.log 2024-09-17T16:47:16.904 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36792.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36792.log.gz 2024-09-17T16:47:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29417.log 2024-09-17T16:47:16.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36452.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36452.log.gz 2024-09-17T16:47:16.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31414.log 2024-09-17T16:47:16.919 INFO:teuthology.orchestra.run.smithi160.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T16:47:16.920 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.29417.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29417.log.gz 2024-09-17T16:47:16.927 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31102.log 2024-09-17T16:47:16.928 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31414.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31414.log.gz 2024-09-17T16:47:16.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36558.log 2024-09-17T16:47:16.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.31102.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31102.log.gz 2024-09-17T16:47:16.951 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36348.log 2024-09-17T16:47:16.952 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36558.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36558.log.gz 2024-09-17T16:47:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-09-17T16:47:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36348.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36348.log.gz 2024-09-17T16:47:16.983 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.33269.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-09-17T16:47:17.053 INFO:teuthology.orchestra.run.smithi105.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T16:47:17.067 INFO:teuthology.orchestra.run.smithi160.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-09-17T16:47:17.161 INFO:teuthology.orchestra.run.smithi160.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-09-17T16:47:17.434 INFO:teuthology.orchestra.run.smithi105.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T16:47:17.581 INFO:teuthology.orchestra.run.smithi105.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T16:47:17.759 INFO:teuthology.orchestra.run.smithi105.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-09-17T16:47:17.775 INFO:teuthology.orchestra.run.smithi160.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-09-17T16:47:17.793 INFO:teuthology.orchestra.run.smithi105.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-09-17T16:47:17.933 INFO:teuthology.orchestra.run.smithi160.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-09-17T16:47:18.044 INFO:teuthology.orchestra.run.smithi160.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.29453.log.gz 2024-09-17T16:47:23.262 INFO:teuthology.orchestra.run.smithi160.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T16:47:23.416 INFO:teuthology.orchestra.run.smithi105.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T16:47:23.871 INFO:teuthology.orchestra.run.smithi160.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T16:47:23.919 INFO:teuthology.orchestra.run.smithi105.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T16:47:24.049 INFO:teuthology.orchestra.run.smithi160.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T16:47:24.050 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-09-17T16:47:24.050 INFO:teuthology.orchestra.run.smithi160.stderr:real 0m7.899s 2024-09-17T16:47:24.051 INFO:teuthology.orchestra.run.smithi160.stderr:user 0m28.767s 2024-09-17T16:47:24.051 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m1.029s 2024-09-17T16:47:24.094 INFO:teuthology.orchestra.run.smithi105.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T16:47:24.095 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-17T16:47:24.095 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m7.981s 2024-09-17T16:47:24.095 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m28.123s 2024-09-17T16:47:24.095 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m1.103s 2024-09-17T16:47:24.096 INFO:tasks.ceph:Archiving logs... 2024-09-17T16:47:24.096 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi105/log 2024-09-17T16:47:24.097 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T16:47:27.758 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi160/log 2024-09-17T16:47:27.758 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T16:47:31.268 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T16:47:31.276 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T16:47:31.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T16:47:31.279 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T16:47:31.360 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T16:47:31.368 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T16:47:31.369 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T16:47:31.375 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout:*hv01.front.sepi 67.205.162.81 3 u 23 64 377 0.058 +2.297 0.303 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout:+hv02.front.sepi 63.231.80.2 3 u 25 64 377 0.101 +2.091 0.306 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout:+hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.104 +2.545 0.556 2024-09-17T16:47:31.383 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout:============================================================================== 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout:*hv01.front.sepi 67.205.162.81 3 u 24 64 377 0.094 -3.339 0.275 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 377 0.111 -3.414 0.280 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout:+hv03.front.sepi 74.6.168.72 3 u 31 64 377 0.089 -2.532 0.274 2024-09-17T16:47:31.386 INFO:teuthology.orchestra.run.smithi160.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T16:47:31.387 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T16:47:31.396 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T16:47:31.396 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T16:47:31.405 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T16:47:31.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T16:47:31.510 INFO:teuthology.task.internal:Duration was 1792.968212 seconds 2024-09-17T16:47:31.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T16:47:31.519 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T16:47:31.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T16:47:31.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T16:47:31.560 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T16:47:31.560 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-09-17T16:47:31.561 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T16:47:31.609 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-09-17T16:47:31.609 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T16:47:31.619 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T16:47:31.619 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T16:47:31.656 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T16:47:31.686 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T16:47:31.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T16:47:31.729 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T16:47:31.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T16:47:31.801 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T16:47:31.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T16:47:31.813 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T16:47:31.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T16:47:31.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T16:47:31.857 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T16:47:31.865 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-09-17T16:47:31.880 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-09-17T16:47:31.891 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T16:47:31.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:47:31.935 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T16:47:31.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T16:47:31.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T16:47:31.951 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T16:47:31.952 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi105 2024-09-17T16:47:31.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T16:47:32.000 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-17_15:01:33-rados-wip-bharath10-testing-2024-09-17-1348-distro-default-smithi/7909060/remote/smithi160 2024-09-17T16:47:32.001 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T16:47:32.043 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T16:47:32.043 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T16:47:32.045 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T16:47:32.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T16:47:32.078 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T16:47:32.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T16:47:32.094 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T16:47:32.094 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T16:47:32.097 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T16:47:32.099 INFO:teuthology.orchestra.run.smithi105.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 16:47 /home/ubuntu/cephtest 2024-09-17T16:47:32.112 INFO:teuthology.orchestra.run.smithi160.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 16:47 /home/ubuntu/cephtest 2024-09-17T16:47:32.114 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T16:47:32.124 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T16:47:32.148 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-hybrid} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} duration: 1792.9682121276855 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-17T16:47:32.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T16:47:32.233 INFO:teuthology.run:pass